CodeKitHubWhat Is This Tool?
Pasted lists — from spreadsheets, email exports, log files, scraped data — routinely carry the same five problems: duplicate entries, blank lines, inconsistent leading/trailing spaces, double spaces where there should be one, and no particular order. Fixing all five usually means bouncing between separate single-purpose tools.
This tool runs all five as one operation with checkboxes, in the order that actually matters (trim and collapse spaces before deduplicating, so "foo " and "foo" are correctly recognized as the same line).
Why Use It?
- Five real cleanup operations in one pass, not just deduplication.
- Checkbox control — turn off any step you don't want.
- Operations run in the order that produces correct results (whitespace normalized before dedup).
- Shows exactly how many lines were removed.
- Local and private — text never leaves your browser.
How to Use
- Paste your text or list into the input box.
- Check the operations you want (all but sorting are on by default).
- Click Clean — the result appears below with a count of removed lines.
- Copy the cleaned result.
Example
Input
apple
banana
apple
banana
cherryOutput
apple
banana
cherryWhitespace is normalized first, so "apple " and "apple" are recognized as duplicates and merged — a common miss in simpler tools.
Frequently Asked Questions
In what order are the operations applied?
Trim → collapse spaces → remove blank lines → deduplicate → sort. This order matters: normalizing whitespace before deduplication means lines that differ only by stray spaces are correctly treated as duplicates.
Is the deduplication case-sensitive?
Yes — "Apple" and "apple" are treated as different lines. This avoids accidentally merging genuinely different entries; if you need case-insensitive matching, lowercase your text first.
What does sorting use for Chinese text?
Pinyin-based alphabetical order, so Chinese entries sort the way they would in a phone contacts list rather than by raw character code.
Will this handle a very large list?
Yes — processing happens in your browser with no size limit beyond what your device's memory can hold, which is well beyond typical spreadsheet-export sizes.
Is my data uploaded anywhere?
No. All five operations run as local JavaScript in your browser. Nothing is sent to a server, so this is safe to use on exported customer lists or other sensitive data.