Skip to content

Remove Duplicates From a List in C#: fix the manual loops, add objects and in-place, .NET 10 - #2269

Open
vladimir-pecanac-main wants to merge 3 commits into
CodeMazeBlog:mainfrom
vladimir-pecanac-main:seo/71194-remove-duplicates-from-a-list-csharp
Open

vladimir-pecanac-main wants to merge 3 commits into
CodeMazeBlog:mainfrom
vladimir-pecanac-main:seo/71194-remove-duplicates-from-a-list-csharp

Conversation

@vladimir-pecanac-main

Copy link
Copy Markdown
Collaborator

Sample for https://code-maze.com/remove-duplicates-from-a-list-csharp/

Only collections-lists/RemoveDuplicatesFromLists changes.

What changes:

  • All three projects move from net6.0 to net10.0.
  • UsingIterationsAndShifting() and UsingIterationsAndSwapping() never wrote to the list: the inner loop assigned to a local variable, so Take(n) only truncated the input. On { 1, 1, 2, 3, 4, 5 } both returned { 1 }. Both now work on a copy of the list and write back into it (shift assigns list[k] = list[k + 1], swap exchanges list[j] and list[size]).
  • The old fixture { 1, 2, 1, 2 } could not catch that. New theory tests run every method against { 1, 1, 2, 3, 4, 5 } and { 3, 1, 3, 2, 1, 4, 2, 5, 4 }. On the old helper four of these cases fail (the two loop methods, both fixtures); on this branch all 41 tests pass.
  • WhenUsingDictionary_ThenRemovesDuplicates() called ConvertingToHashSet(); it now calls UsingDictionary().
  • UsingRecursion() returned the input list from its stop branch; it now returns the list without duplicates (same result, the outer call already discarded that value).
  • where T : notnull on RemoveDuplicatesHelper clears CS8714 and CS8602 on net10.0.
  • New: RemoveDuplicatesInPlace() (List.RemoveAll with a HashSet), Person, PersonRecord and PeopleHelper (Distinct on a class keeps equal-valued objects, Distinct on a record removes them, DistinctBy by Email), each with tests. The in-place test asserts the same list instance changed.
  • Benchmark: [Params(3, 100, 2_000)] distinct values in a 2,000 item list built in [GlobalSetup]; HashSetMethod and InitializingHashetMethod renamed to ConvertToHashSetMethod and InitializingHashSetMethod; SortMethod and the new RemoveAllInPlaceMethod start each call from a fresh copy because both change their list.

Packages (re-queried on NuGet 2026-10-04, newest stable):

  • Microsoft.NET.Test.Sdk 18.10.1
  • xunit 2.9.3
  • xunit.runner.visualstudio 4.0.0
  • coverlet.collector 10.1.0
  • BenchmarkDotNet 0.15.8

Local: SDK 10.0.302, runtime 10.0.10. dotnet build -c Release: 0 warnings, 0 errors. dotnet test: 41 passed, 0 failed. dotnet list package --vulnerable --include-transitive: none.

…in-place, net10.0

- Retarget all three projects to net10.0 and bump packages.
- Fix UsingIterationsAndShifting() and UsingIterationsAndSwapping(): both assigned
  to a local variable and never wrote to the list, so { 1, 1, 2, 3, 4, 5 } came back
  as { 1 }. Both now work on a copy and write back into it.
- Tests: a fixture with a duplicate at index 0 and one with duplicates spread through
  the list, run against every method. WhenUsingDictionary now tests UsingDictionary().
- UsingRecursion() returns the list without duplicates when it reaches the end.
- where T : notnull clears CS8714 and CS8602.
- Add RemoveDuplicatesInPlace() (RemoveAll with a HashSet<T>), Person, PersonRecord
  and PeopleHelper (Distinct on a class, on a record, and DistinctBy), with tests.
- Benchmark: [Params] for the number of distinct values in a 2,000 item list, method
  names match the article, and the two methods that change their list start every
  call from a fresh copy.
Sorting() compared every item with a start value of default, so a 0 in a
List<int> was never added: { 0, 1, 0, 2 } returned { 1, 2 }. The first item
is now always added. New theory test runs every method against { 0, 1, 0, 2 };
it fails on the old Sorting() and passes on all methods now.
With T only constrained to notnull, x.Equals(item) and list[i].Equals(list[j])
bind to object.Equals(object), so every comparison in the Any(), loop and
sorting methods boxed an int. Constraining T to IEquatable<T> makes them call
Equals(T) directly, which is what the same code does on a plain List<int>, so
the benchmark now measures the code the article shows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant