Part-of-speech tagging labels words as nouns, verbs and other grammatical classes.
dotnet add package RecursiveAI.OneNlp.NeuralRequires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.
// train.conllu and dev.conllu: zip inputs shown below. using RecursiveAI.OneNlp; using RecursiveAI.OneNlp.Data; using RecursiveAI.OneNlp.Neural.Components; using RecursiveAI.OneNlp.Neural.Encoders; using var nlp = Pipeline.Blank(Language.English).Add( new Tagger(FastEncoder.English(7), TokenColumn.Pos) { OwnsEncoder = true }); var train = ConllUReader.Read("train.conllu") .Select(s => Example.FromReference(s.Doc)).ToList(); var dev = ConllUReader.Read("dev.conllu") .Select(s => Example.FromReference(s.Doc)).ToList(); nlp.Train(train, new TrainOptions { Epochs = 30, Seed = 7, Dropout = 0f, DevExamples = dev }); var doc = nlp.Process("Birds chase cats"); foreach (var token in doc) Console.WriteLine($"{token.Text}: {token.Pos}");
Birds: NOUN chase: VERB cats: NOUN
Unzip the example, then dotnet run.
Full program
These toy sentences demonstrate training, not real-world accuracy. The full program trains on four sentences, uses a separate sentence for development, saves model.onenlp and reloads it before predicting “Birds chase cats”. That prediction sentence is not in either input file.
train.conllu (tab-separated):
1 Cats cats NOUN NNS _ 2 nsubj _ _ 2 chase chase VERB VBP _ 0 root _ _ 3 birds birds NOUN NNS _ 2 obj _ _ 1 Dogs dogs NOUN NNS _ 2 nsubj _ _ 2 chase chase VERB VBP _ 0 root _ _ 3 cats cats NOUN NNS _ 2 obj _ _ 1 Birds birds NOUN NNS _ 2 nsubj _ _ 2 watch watch VERB VBP _ 0 root _ _ 3 dogs dogs NOUN NNS _ 2 obj _ _ 1 Cats cats NOUN NNS _ 2 nsubj _ _ 2 watch watch VERB VBP _ 0 root _ _ 3 birds birds NOUN NNS _ 2 obj _ _
dev.conllu (tab-separated):
1 Dogs dogs NOUN NNS _ 2 nsubj _ _ 2 watch watch VERB VBP _ 0 root _ _ 3 cats cats NOUN NNS _ 2 obj _ _
using RecursiveAI.OneNlp; using RecursiveAI.OneNlp.Data; using RecursiveAI.OneNlp.Neural; using RecursiveAI.OneNlp.Neural.Components; using RecursiveAI.OneNlp.Neural.Encoders; using var nlp = Pipeline.Blank(Language.English).Add( new Tagger(FastEncoder.English(7), TokenColumn.Pos) { OwnsEncoder = true }); var train = ConllUReader.Read("train.conllu") .Select(s => Example.FromReference(s.Doc)).ToList(); var dev = ConllUReader.Read("dev.conllu") .Select(s => Example.FromReference(s.Doc)).ToList(); nlp.Train(train, new TrainOptions { Epochs = 30, Seed = 7, Dropout = 0f, DevExamples = dev }); PipelineFile.Save(nlp, "model.onenlp"); using var loaded = PipelineFile.Load("model.onenlp"); var doc = loaded.Process("Birds chase cats"); foreach (var token in doc) Console.WriteLine($"{token.Text}: {token.Pos}"); Console.WriteLine("Saved model.onenlp");
dotnet run prints:
Birds: NOUN chase: VERB cats: NOUN Saved model.onenlp
Malformed CoNLL-U throws DataFormatException with a line number; a trained head rejects new labels outside its existing label set with ArgumentException.
Variations
Train language-specific tags
To learn the XPOS column instead of UPOS, replace the pipeline construction in the program with this, leaving its training call unchanged:
using var nlp = Pipeline.Blank(Language.English).Add( new Tagger(FastEncoder.English(7), TokenColumn.Tag) { OwnsEncoder = true });
Print token.Tag instead of token.Pos. With the same files, the predictions are:
Birds: NNS chase: VBP cats: NNS
Related recipes
- Use pretrained part-of-speech tagging
- Use a pretrained dependency parser
- Read CoNLL-U training data
- Save NLP results as JSON
The short answer
To train a part-of-speech tagger in C#, create a blank OneNLP pipeline, add a Tagger with a trainable encoder, read labeled sentences into Example values and call Pipeline.Train. Choose TokenColumn.Pos explicitly for UPOS: the tagger’s default column is Tag, which means XPOS. No pretrained model package is needed for this example.
How do you train a POS tagger?
Training needs words paired with the labels you want the model to predict. This recipe uses the CoNLL-U reader because each row already describes one word and its annotations. Example.FromReference makes an example from that annotated document; the reference supplies the answers used in training. The encoder starts from the seed passed to FastEncoder.English, and the tagging head learns its label set from the examples.
The example uses OwnsEncoder = true so that this head owns its encoder. The pipeline is disposed with using. If you later share one encoder across several heads, follow the training contracts for ownership and shared training rather than giving each head a separate copy by accident.
The seed in TrainOptions controls the training sequence, while dropout is disabled here to keep the small demonstration repeatable. The training documentation scopes resumed-run equivalence to the same machine and thread count; a seed alone is not a promise of identical floating-point output on every system.
What are the different types of POS tags?
The input has both UPOS and XPOS columns. In this fixture, UPOS contains NOUN and VERB; XPOS contains NNS and VBP. TokenColumn.Pos learns the former and fills token.Pos. TokenColumn.Tag learns the latter and fills token.Tag. Those are different prediction targets, not two display styles for the same output.
Use the annotation scheme your corpus actually supplies. Do not mix language-specific tag sets into one column unless that is deliberately your desired label space. The CoNLL-U column mapping also keeps lemmas, morphology and dependency relations separate. Training this tagging head does not train those other annotations.
A head without weights gets its labels from the training references. A head that already has weights has a fixed label set: adding an unseen value raises ArgumentException. To change the tag inventory, train a new head rather than assuming a loaded checkpoint will grow its outputs.
How do you perform POS tagging after training?
Call Process on the trained pipeline, or save it with PipelineFile.Save, reload it with PipelineFile.Load and call Process on the loaded pipeline as the full program does. Read the selected annotation from each token. A pipeline file stores the tokenizer, encoder, heads and labels; a document bin stores annotated text instead of those weights.
Keep training, development and final evaluation sentences separate. The development set guides model selection, so it cannot also be an untouched measure of final performance. This example holds out a different word combination for prediction, but its tiny vocabulary and repeated sentence pattern cannot establish accuracy on natural text. Use representative annotated sentences and an independent evaluation set before relying on predictions in an application.