R Text Analysis
r
text-analysis
topic-models
tidytext
lda
stm
quanteda
text-mining
Books, tutorials, topic modeling guides, regular expression tools, and packages for analyzing text in R.
This guide collects material for analyzing text with R, from cleaning strings and regular expressions to word frequencies and topic models. Begin with the three picks under Start here, then use the sections on tidy text, topic modeling, regular expressions, and packages as your project needs them. Installation steps are on the R page.
Start here
- Text Mining with R — Free online book on text mining with tidy data principles, by Julia Silge and David Robinson.
- quanteda tutorials — Tutorials that introduce quantitative text analysis with the quanteda package.
- Strings, R for Data Science (2e) — Chapter on working with strings in R, including regular expressions.
Tidy text and tutorials
- The tidy text format — Chapter 1 of Text Mining with R, on the tidy text format and the unnest_tokens function.
- Converting to and from non-tidy formats — Chapter 5 of Text Mining with R, on moving text between tidy and non-tidy formats.
- Term Frequency and Inverse Document Frequency (tf-idf) Using Tidy Data — Vignette from the tidytext package on tf-idf with tidy data.
- tidytext — Package for text mining with tidy tools.
- widyr — Package for widening, processing, and re-tidying datasets, such as for pairwise word counts.
- Basic Text Mining with R — Introductory text mining tutorial in R (archived copy).
- Analyzing Star Wars Movie Scripts — Kaggle notebook that analyzes Star Wars movie scripts in R.
- Getting started with stringr for textual analysis in R — Storybench tutorial on using stringr to work with text.
- Concordancing with R — Martin Schweinberger’s post on concordancing in R (archived copy).
Topic modeling
- Topic modeling — Chapter 6 of Text Mining with R, on topic modeling.
- An Introduction to Topic Modeling — Video introduction to topic models by Professor Chris Bail.
- The game is afoot! Topic modeling of Sherlock Holmes stories — Julia Silge’s post on topic modeling the Sherlock Holmes stories with stm.
- Training, evaluating, and interpreting topic models — Julia Silge’s follow-up post on training, evaluating, and interpreting topic models.
- stm — Package for the structural topic model.
- Topic Models (LDA, CTM, STM) — RPubs introduction to topic models, covering LDA, CTM, and STM.
- A topic model for movie reviews — Document that fits an LDA topic model to movie review data and explores it with LDAvis.
- Topic Modeling in R With tidytext and textmineR — Medium article on latent Dirichlet allocation with these two packages; Medium may limit free reading.
- A Link Between topicmodels LDA and LDAvis — R-Bloggers post on connecting topicmodels output to the LDAvis package.
- David M. Blei: Topic Modeling Software — David Blei’s page listing topic modeling software (archived copy).
Text mining frameworks
- Why you should stop using other text mining packages and embrace quanteda — R-Bloggers write-up of Kenneth Benoit’s talk arguing for quanteda over other text mining packages.
- Text Mining Infrastructure in R — Journal of Statistical Software article describing the tm text mining infrastructure for R.
Regular expressions
- Regular Expression Playground — Interactive site for practicing and testing regular expressions.
- RegExr: Learn, Build, & Test RegEx — Online tool for building and testing regular expressions.
- Regex Cheat Sheet — Regex cheat sheet from Pete Freitag’s site (archived copy).
- The Rebus Code: Unveiling the Secrets of Regex in R — R-Bloggers post on writing regular expressions in R.
Text packages
| Package | What it does |
|---|---|
| SnowballC | R interface to the Snowball word stemmers |
| hunspell | Stems words, tokenizes text, and checks spelling |
| readtext | Reads text files into R |
| jstor | Imports journal data from JSTOR’s Data for Research |
| textreuse | Detects text reuse and document similarity |
| spelling | Checks spelling in text and documents |
| cld3 | Detects the language of a text with Google’s Compact Language Detector 3 |
| youtubecaption | Downloads YouTube subtitles into a tidy data frame |
| textclean | Cleans and normalizes text data |
| citation-network | Builds a quick cross-citation network for a set of texts |