by Richard V. (5 Submissions)
Category: String Manipulation
Difficulty: Beginner
Date Added: Wed 3rd February 2021
Rating: 
(2 Votes)
Remove duplicate items from word lists.
Assumes
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'Removes duplicates from large (milllions) wordlists quickly
'
'Sorts wordlists using the quicksort algorithm, then
'removes duplicates in a single pass, O(n), by comparing
'each item against the preceding item.
'
'Items can be "unsorted" back to the order of the original
'wordlist with the use of an array of indexes, a new
'array containing the original position of each item.
'
'Stable Quicksort algorithms by Rde.
'http://www.pscode.com/vb/scripts/ShowCode.asp?txtCodeId=63941&lngWId=1
'
'Much(!) faster Split replacement, "Quick Split," by Merri @ VBForums.com.
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'Note: memory usage issues
Download A fast method to Remove Duplicates from Word Lists (14 KB)
VBcoders