Remove Duplicate Lines
Delete repeated lines from a list and keep only the unique ones.
2 duplicates removed. 6 of 8 lines kept.
Runs in your browser. Nothing you enter is uploaded.
How to use the Remove Duplicate Lines
- Paste your list into the box with one item on each line.
- Choose your options: ignore case, trim spaces, remove empty lines and sort order.
- Check how many duplicates were removed in the summary.
- Copy the unique lines from the result box.
About the Remove Duplicate Lines
Got a long list with repeated entries, such as email addresses, keywords, product codes or log lines? This tool removes the duplicate lines and keeps one copy of each. You can ignore differences in capital letters, trim stray spaces, drop empty lines and sort the result, and it tells you how many duplicates it found.
The tool reads your text line by line and remembers each line it has already seen. The first time a line appears it is kept, and any later copy is dropped, so the original order is preserved unless you choose to sort. A single line break at the very end of your text is ignored instead of being counted as an empty line.
By default two lines must match exactly. If you tick 'Ignore upper and lower case', Apple and apple count as the same line and the first spelling is kept. Trimming removes spaces and tabs from both ends of each line before comparing, which catches duplicates that only differ by invisible spacing. With 'Remove empty lines' ticked, blank lines are removed and counted separately from duplicates. Sorting uses alphabetical order, from A to Z or Z to A, with numbers compared as text.
Frequently asked questions
Which copy of a duplicate line is kept?
The first one. Later copies are removed, so the order of your list is preserved unless you sort it.
Why are two lines that look identical not removed?
They may differ by a space at the end or by capital letters. Turn on 'Trim spaces' and 'Ignore upper and lower case' to treat them as the same.
Can it handle large lists?
Yes. Lists with tens of thousands of lines are processed instantly in your browser. Extremely large files may slow down older devices.
Are empty lines counted as duplicates?
Without the empty-line option, only the first blank line is kept and the rest count as duplicates. With the option on, all blank lines are removed and counted separately.
Is my list uploaded?
No. The list is processed in your browser and never leaves your device.