Remove Duplicate Lines
Clean up email lists, keyword lists, product SKUs or log files by removing repeated lines. The first occurrence of each line is kept, in its original position, and the tool tells you how many duplicates were removed.
Runs in your browser
How to use
- Paste your list, one item per line.
- Choose whether “Apple” and “apple” count as the same.
- Copy the de-duplicated list.
How it works
Each line is compared with every line seen before it using a hash set, so even lists with hundreds of thousands of lines are processed instantly. With “Ignore leading/trailing spaces” on, “ apple” and “apple ” are treated as duplicates and the output is trimmed.
Order is preserved: only later repeats are removed. If you also want the list sorted, use Sort Lines afterwards.
Examples
- Newsletter list
- Merging two exports of 2,000 addresses each often reveals several hundred duplicates. Turn off case sensitivity: email addresses are case-insensitive in practice.
Common uses
- De-duplicating email and contact lists
- Cleaning keyword research exports
- Finding unique values in a column copied from Excel
- Reducing noisy log files
Privacy & security
This tool runs entirely in your browser. Your files and text are never uploaded to our servers — processing happens on your device and results are created locally. How each tool handles files
Frequently asked questions
Which occurrence is kept?
The first one. Later identical lines are removed and the order of the remaining lines doesn't change.
Can I paste a column from Excel or Google Sheets?
Yes. Copying a column puts one cell per line, which is exactly what the tool expects.
Last updated: