---
title: "Chinese word segmentation in C#"
url: https://recursiveai.org/onenlp/cookbook/segment-chinese-words/
description: "Chinese word segmentation finds words in text that has no spaces."
version: "1.0.0"
---

# Chinese word segmentation in C#

**OneNLP** is a free, open-source natural language processing library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

[NuGet](https://www.nuget.org/packages/RecursiveAI.OneNlp.Models.Chinese) · [GitHub](https://github.com/recursive-ai-oss/onenlp-csharp) · [Docs](https://recursiveai.org/onenlp/docs/) · [Download example (.zip)](https://recursiveai.org/onenlp/cookbook/segment-chinese-words/segment-chinese-words.zip)

Chinese word segmentation finds words in text that has no spaces. The Chinese model supplies trained word boundaries, including in mixed Chinese and Latin text.

```shell
dotnet add package RecursiveAI.OneNlp.Models.Chinese
```

Requires [.NET 10](https://dotnet.microsoft.com/download/dotnet/10.0) (net10.0); .NET 9 and earlier aren’t supported.

```csharp
using RecursiveAI.OneNlp.Models.Chinese;

using var nlp = ChineseModel.Load();
var doc = nlp.Process("我喜欢学习中文和CSharp123。");
foreach (var word in doc)
    Console.WriteLine($"{word.Text} " +
        $"[{word.Idx},{word.EndIdx})");
```

```text
我 [0,1)
喜欢 [1,3)
学习 [3,5)
中 [5,6)
文 [6,7)
和 [7,8)
CSharp123 [8,17)
。 [17,18)
```

The input is “我喜欢学习中文和CSharp123。”. Each printed range is a half-open UTF-16 offset into that string.

“中文” comes out as “中” and “文” because the model learned from annotated sentences that split compounds into small words.

Unzip [the example](https://recursiveai.org/onenlp/cookbook/segment-chinese-words/segment-chinese-words.zip), then `dotnet run`.

## Variations

**Compare with character boundaries.** A blank pipeline starts without trained models. Here it uses character-boundary rules, so each Chinese character is a separate token.

```csharp
using RecursiveAI.OneNlp;

using var nlp = Pipeline.Blank(Language.Chinese);
foreach (var word in nlp.Process("学习中文"))
    Console.WriteLine(word.Text);
```

```text
学
习
中
文
```

## Related recipes

- [Tokenize text](https://recursiveai.org/onenlp/cookbook/tokenize-text/)
- [Split sentences](https://recursiveai.org/onenlp/cookbook/split-sentences/)

## The short answer

Install `RecursiveAI.OneNlp.Models.Chinese`, load `ChineseModel.Load()` and iterate the document returned by `Process`. Each token gives you its text, start offset (`Idx`) and exclusive end (`EndIdx`). This is the trained Chinese pipeline, not a dictionary lookup or a split at every character.

## How do I segment Chinese words in C#?

Use the model package rather than a blank Chinese pipeline. The variation above demonstrates the difference: a blank pipeline splits “学习中文” into individual ideographs. The model groups “学习” in the longer example, but still splits “中文” into “中” and “文”. Those are the actual predictions from this input, not a manually corrected word list.

That distinction matters when you store search terms or align annotations. A tokenizer can return valid character boundaries without finding the word boundaries your application expects. Before choosing a segmenter, decide what a word means for your corpus. OneNLP’s model learns word boundaries from UD Chinese-GSDSimp, a collection of Simplified Chinese sentences annotated by people (a “treebank”). Its [segmentation documentation](https://recursiveai.org/onenlp/docs/segmentation/#chinese-trained-word-segmentation-neuralwordsegmenter) says that this treebank splits compounds finely and writes personal names as separate surname and given-name tokens. A different annotation policy can disagree with those boundaries without either string being malformed.

### How does the trained segmenter work?

The segmenter assigns each input unit a label: `B` means “begin a new word” and `I` means “continue the previous word”. It is the pipeline’s word tokenizer, so downstream annotations operate on its resulting words. The loaded pipeline also supplies other Chinese analysis components; you do not need to create a separate rule tokenizer before calling it.

Each non-whitespace character is a unit, with a surrogate pair treated as one character. Runs of Latin, Greek or Cyrillic letters and runs of digits are units instead of individual characters. These are units presented to the model, not a promise that every unit becomes a separate final word. In the example, the model returns `CSharp123` as one word. Whitespace never occurs inside a word, and the first unit after whitespace begins a new word.

### How should I use the offsets?

Keep the original input string alongside your tokens. `Idx` includes the first UTF-16 code unit and `EndIdx` excludes the unit after the token. These are .NET string positions, not UTF-8 byte offsets. Do not apply them to a lowercased, normalized or otherwise edited copy of the text. This lets you highlight the exact original range without trying to locate repeated words by their spelling.

The printed ranges also make segmentation choices visible. “学习” covers `[3,5)`, while the punctuation has its own range. Do not assume a token’s position in the token sequence equals its character offset.

### Which model do I need?

The language-neutral core package does not provide trained Chinese word segmentation by itself. The explicit Chinese model load in this recipe selects the released Chinese model directly and avoids relying on automatic language detection for short mixed-language strings. Other model tiers have separate packages and resolution rules; see [model packages](https://recursiveai.org/onenlp/docs/model-packages/). Keep the model package version with your application so a change in weights does not silently change the boundaries in an existing index.