• Home
 · 
• String Manipulation
 · 
• A fast method to Remove Duplicates from Word Lists

A fast method to Remove Duplicates from Word Lists

Remove duplicate items from word lists.

Assumes
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 'Removes duplicates from large (milllions) wordlists quickly ' 'Sorts wordlists using the quicksort algorithm, then 'removes duplicates in a single pass, O(n), by comparing 'each item against the preceding item. ' 'Items can be "unsorted" back to the order of the original 'wordlist with the use of an array of indexes, a new 'array containing the original position of each item. ' 'Stable Quicksort algorithms by Rde. 'http://www.pscode.com/vb/scripts/ShowCode.asp?txtCodeId=63941&lngWId=1 ' 'Much(!) faster Split replacement, "Quick Split," by Merri @ VBForums.com. '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 'Note: memory usage issues

Rate A fast method to Remove Duplicates from Word Lists

• 1
• 2
• 3
• 4
• 5

(2(2 Vote))

Download A fast method to Remove Duplicates from Word Lists