Lexicon linguistics also referred to as text data mining, is the process of deriving high-quality information (patterns and trends) from text. Text data mining, simply put, is to derive information from text. Utilising a set of lexicon algorithms and natural language techniques, text mining yields knowledge from unstructured or semi-structured sets of text data, and provides detailed information for further analysis. This can be applied to categorised areas, including ethnicity, religion, health care, education and other areas of knowledge discovery.

• Extract, clean and create a dictionary of words, abbreviations from the text
• Text filtering, removal of stop words, i.e.: the, and, is etc.
• Transform and clean text, i.e.: removal of white spaces, punctuation, numbers etc.
• Summarise remaining words, synonyms to parent words, i.e.: walk, walking, walked > walk
• Count the frequency of parent word occurrences
• Check for negative, negative values, i.e.: has walked, has not walked
• Map summarised words to categorised classifications, i.e.: ethnicity, religion, health care etc.
• Map summarised words to positive, negative groupings and apply ratings
• Count the frequency of categorised classification groupings
• Capture trend analysis, if text data time stamped - helps to better predict movement