Effective text mining operations are predicated on sophisticated data preprocessing methodologies. In fact, text mining is arguably so dependent on the various preprocessing techniques that infer or extract structured representations from raw unstructured data sources, or do both, that one might even say text mining is to a degree defined by these elaborate preparatory techniques. Certainly, very different preprocessing techniques are required to prepare raw unstructured data for text mining than those traditionally encountered in knowledge discovery operations aimed at preparing structured data sources for classic data mining operations.

A large variety of text mining preprocessing techniques exist. All in some way attempt to structure documents – and, by extension, document collections. Quite commonly, different preprocessing techniques are used in tandem to create structured document representations from raw textual data. As a result, some typical combinations of techniques have evolved in preparing unstructured data for text mining.

Two clear ways of categorizing the totality of preparatory document structuring techniques are according to their task and according to the algorithms and formal frameworks that they use.

Task-oriented preprocessing approaches envision the process of creating a structured document representation in terms of tasks and subtasks and usually involve some sort of preparatory goal or problem that needs to be solved such as extracting titles and authors from a PDF document. Other preprocessing approaches rely on techniques that derive from formal methods for analyzing complex phenomena that can be also applied to natural language texts.