mygemsSay hi

Remove duplicate rows from a CSV

Duplicates on the columns you say identify a row.

  • Match on chosen columns
  • Preview before removing
  • Keep first or last
  • Nothing uploaded

1.Open the file

CSV or TSV. Drag it here, or choose a file.

Why it exists

"Duplicate" almost never means "every field is identical" — it means two rows are the same customer, or the same order, with different timestamps hanging off them. So this asks which columns identify a record before it removes anything, shows you what it would remove, and lets you keep either the first or the last of each group.

What it does

  • Deduplicates on the whole row, or on the subset of columns you pick as the key.
  • Counts the duplicates and shows an example group before anything is removed.
  • Keeps the first or the last row of each group, your choice.
  • Downloads the deduplicated file, and tells you exactly how many rows went.

A note on what counts as the same

Matching is exact and case-sensitive, on the values as they appear in the file. Two rows that differ only by a trailing space are two rows. If that is not what you want, the fix belongs upstream of deduplication, not inside it.

Questions

Can I deduplicate on just one column?
Yes — that is the common case. Pick the column or columns that identify a record, such as an email address or an order ID, and rows sharing those values are treated as one.
Which duplicate is kept?
The first or the last of each group, whichever you choose. First keeps the earliest row in file order; last keeps the most recent, which is usually what you want when later rows are updates.
Is the match case-sensitive?
Yes. Values are compared exactly as they appear in the file, so `[email protected]` and `[email protected]` are different.

The other tools