List Duplicate Remover
Data cleaning tool. Paste a messy list of items, and this script uses JavaScript Sets to filter out repetitions automatically.
Copy the Script
<script>
function cleanList() {
var raw = document.getElementById("inputArea").value;
// Split by new line
var arr = raw.split(/\r?\n/);
// Use Set to remove duplicates
var uniqueSet = new Set(arr.map(item => item.trim()).filter(item => item !== ""));
var uniqueArr = Array.from(uniqueSet);
document.getElementById("outputArea").value = uniqueArr.join("\n");
}
</script>
<textarea id="inputArea"></textarea>
<button onclick="cleanList()">Clean</button>
<textarea id="outputArea"></textarea>Frequently Asked Questions
By default, 'Apple' and 'apple' are treated as different words. You can add `.toLowerCase()` in the comparison logic to make it case-insensitive.
Yes. It is best practice to trim whitespace from the start and end of lines so that ' word' and 'word' are detected as duplicates.
It depends on the browser's memory, but JavaScript can easily handle lists with tens of thousands of lines in milliseconds.