Recursive AI

Lemmatize text in C#

OneNLP is a free, open-source natural language processing library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

NuGet GitHub Docs Download example (.zip)

Get dictionary forms such as “run” for “running” with the English model.

Terminal
dotnet add package RecursiveAI.OneNlp
dotnet add package RecursiveAI.OneNlp.Models.English

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using RecursiveAI.OneNlp;
 
var lemmas = Nlp.Lemmas(
    "The cats were running.");
Console.WriteLine(string.Join(" | ", lemmas));

The snippet prints lemmas in token order, including punctuation.

Text
the | cat | be | run | .

Unzip the example, then dotnet run.

The full program

The download’s Program.cs uses the same visible input as the snippet at the top. No input files are needed.

The download’s Program.cs is the snippet above.

dotnet run prints:

Text
the | cat | be | run | .

If the required language model is missing, ModelNotAvailableException names the package to add; nothing is downloaded.

Variations

Keep words, lemmas and parts of speech together. Analyze the text once to read each token’s annotations. This is a standalone replacement for Program.cs.

C#
using RecursiveAI.OneNlp;
 
var doc = Nlp.Analyze("The cats were running.", Language.English);
foreach (var token in doc)
    Console.WriteLine($"{token.Text}: {token.Lemma} ({token.Pos})");
Text
The: the (DET)
cats: cat (NOUN)
were: be (AUX)
running: run (VERB)
.: . (PUNCT)

The short answer

Add RecursiveAI.OneNlp.Models.English and call Nlp.Lemmas(text). For The cats were running., the complete example prints the | cat | be | run | .. The output follows token order, including the period. The model is included in the NuGet package; no service key or runtime weights download is needed.

What is lemmatization with an example?

Lemmatization assigns a base form, or lemma, to a token. In this example, cats becomes cat, were becomes be, and running becomes run. The original text is still available: a lemma is an annotation, not an instruction to rewrite the source sentence. Keeping both forms lets an application search by base form while displaying the wording the author used.

Nlp.Lemmas returns the lemma sequence. If you also need the source words or grammatical information, use Nlp.Analyze as in the variation. Each token then exposes Text, Lemma and Pos, so the output can show running: run (VERB). This avoids aligning a separately split word list with model output after the fact.

The lemmas here come from a trained English model, as described in the language model documentation. Its result is a prediction. Inspect real examples from your text domain, especially names, unusual inflections and unfamiliar vocabulary, rather than assuming every token has the base form you expect. Keep punctuation handling explicit if the next stage wants only word lemmas.

What does stemming and lemmatization mean?

Stemming cuts words down using rules for their spelling, while lemmatization assigns dictionary forms using a word’s grammatical role and context.

Both terms occur in searches about reducing word variation, but this page’s API supplies lemmas. It does not expose a stemmer in the example, and it does not produce a vocabulary of unique search terms. The output is a sequence with one entry for each token; repeated words can therefore produce repeated lemmas.

For an application that indexes text, decide separately whether to retain punctuation, lowercase terms, remove common words or deduplicate entries. Those choices are not part of the Nlp.Lemmas call. In particular, replacing a sentence with its lemmas loses the original inflection and is not a grammatical paraphrase or a translation.

What is the difference between tokenization and lemmatization?

Tokenization decides where the words and punctuation are. Lemmatization annotates those tokens with base forms. OneNLP can tokenize using the core package alone, but it needs a language model to supply lemmas. The model package documentation distinguishes these paths: Nlp.Analyze can fall back to tokens and sentence boundaries when no language model is present, while the model-dependent verbs raise an error naming the missing package.

Language detection and model availability are separate. The default overload chooses the language; the variation names English explicitly. Referencing a package for one language does not supply lemmatization for every language the detector can identify. The documented Hindi package, for example, has no real lemmas: its lemma column contains token text instead. Choose the supported language model for the actual input, and do not treat unchanged tokens as evidence that lemmatization ran successfully.

All onenlp recipes