Tools for Corpus Linguistics

A hopefully comprehensive list of currently 266 tools used in corpus compilation and analysis.

This list is kept up to date by its users. Hence, please feel free to contribute by suggesting new tools. You can also make suggestions, e.g., corrections, regarding individual tools by clicking the symbol. As this is a non-commercial side (side, side) project, checking and incorporating updates usually takes some time.

Suggest a Tool

Top 25 Tags

All Tags
concordancer 44
annotation 40
visualization 27
tagging 19
pos tagger 17
text analysis 17
wordlists 16
compilation 11
statistics 11
keywords 11
collocation 10
qda 9
lexis 8
parser 8
readability 7
tokenizer 7
mixed methods 6
web-based 6
spoken 6
analysis 6
python 5
crawler 5
transcription 5
semantics 5
xml 5

There is also a comprehensive list of all tags in the database.


Tools [tokenizer]

Tool Description Tags Platforms Pricing
jTokenizer Tokenizing natural languagetokenizerFree
Natural Language Toolkit Platform for building Python programs to work with human language datatokenizer, taggerUnix, Mac, Windows (+Python 3.4)Free
SoMaJo A tokenizer and sentence splitter for German and English web and social media texts.tokenizer, sentence boundary detectorLinux, Mac, WindowsFree, Open Source
Tweet NLP Tweet tokenizer, POS Tagger, hierarchical word clusters, and a dependency parser for tweets, along with annotated corpora and web-based annotation tools. Clusters: http://www.cs.cmu.edu/~ark/TweetNLP/cluster_viewer.html pos tagger, tokenizer, parserFree
TXM XML & TEI compatible text analysis software based on TreeTagger, the CQP search engine and the R statistical environment.text analysis, concordancer, r, statistics, search tool, tokenizer, xmlWindows,Mac,Linux,TomcatFree
Unitok An annotation-aware tokenizer that splits text into line-by-line tokens.tokenizerFree
YACSI Chinese Tokeniser / PoS Tagger A Chinese tokenizer and PoS taggerchinese, tokenizer, pos taggerWindowsFree

Last Updated: June 19, 2022.

In case you are interested, the data is also available in JSON format.