Step 1
Choose a corpus and sample size
Corpus composition and size determine which patterns can be observed.
Tokens analyzed—
Available tokens—
Types—
Type–token ratio—
Step 2
Separate count from rate
Raw counts answer “how many?”; normalized rates support comparisons across differently sized corpora.
| Rank | Token | Count | Per 1,000 |
|---|
Step 3
Read concordance lines
A frequency number loses the local contexts that make a word's use interpretable.
Can frequency alone establish meaning?
No. Counts identify recurring forms. Concordance lines, annotations, comparison corpora, and a sampling argument are needed to support claims about use or meaning.
Step 4
Rank neighboring words
Pointwise mutual information compares observed co-occurrence with what independent frequencies would predict.
PMI = log₂(P(x,y) ÷ [P(x)P(y)])
| Rank | Adjacent pair | Count | PMI (bits) |
|---|
Exit question
Which association would you trust enough to investigate?
Use its count, PMI, sample size, and concordance context—not just its rank.