---
title: "Contextual token scoring"
url: https://recursiveai.org/onespeech/docs/contextual-biasing/
description: "OneSpeech.Decoding.PhraseTrie is an immutable, inference-only context graph. Construct it from TokenPhrase values tokenized by the model's tokenizer.…"
version: "1.0.0"
source: https://github.com/recursive-ai-oss/onespeech-csharp/blob/63002de486786163a0a7800a51811e76d511fae6/docs/contextual-biasing.md
---

# Contextual token scoring

`OneSpeech.Decoding.PhraseTrie` is an immutable, inference-only context graph.
Construct it from `TokenPhrase` values tokenized by the model's tokenizer. Include
word-boundary markers when the tokenizer uses them; this primitive matches IDs,
not words, case variants or Unicode strings. Blank and control tokens are never
phrases. The decoder validates vocabulary bounds and excludes blank.

A phrase's weight is a natural-log bonus per token, default 1, strictly positive
and at most 20. Completion receives `token count × weight`. Duplicate phrases
use the largest weight. Multiple phrases finishing at the same token receive
the largest completion reward, not their sum. Overlapping completions at
different positions each receive credit. Lists are bounded to 4096 phrases,
256 tokens per phrase and 65536 total tokens.

For search pruning, unfinished prefixes borrow credit proportional to their
depth and weight. Failure links retain suffix matches. `Advance` returns the
completion reward plus the change in prefix credit; `Finish` repays all remaining
prefix credit. Consequently `Score` of a complete sequence counts only completed
phrases. A search must keep one context state per hypothesis and advance it
only when accepting a new nonblank token. Do not advance on CTC repeated frames
or blank frames. Context states belong to the trie that created them.

`ContextualCtcBeamSearch` sums acoustic blank/nonblank path probabilities just as
ordinary CTC prefix search does. It ranks prefixes by acoustic log probability
plus context score before pruning, then removes unfinished credit before final
ranking. It reports acoustic probability and completion reward separately; the
fused score is not a probability. With an empty trie it is equivalent to
`CtcPrefixBeamSearch` at the same beam size. Pruning can discard the global winner.

`ContextualTransducerSearch` adds context deltas before greedy token selection
without renormalizing the acoustic distribution. Blank retains context, the
predictor receives only accepted nonblanks, and TDT duration logits are unchanged.
Confidence is always the original conditional acoustic probability. The greedy
search cannot recover an alternative after pruning: unfinished-prefix credit
can influence the final token. It shares symbol bounds and native frame intervals
with the non-contextual RNN-T/TDT decoders. Empty context is neutral.

This follows the inference-only shallow-fusion principle in Le et al.,
*Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep
Biasing and Shallow Fusion* (2021), §3.1. It does not implement that paper's learned
deep-biasing modules. Context bonuses can introduce false alarms; measure both
phrase recall and WER, including controls whose phrases are absent from the audio.
These primitives alone do not enable phrase requests on a recognizer family.

## Recognizer requests

`SpeechRecognizer.WithPhrases(IEnumerable<Phrase>, CancellationToken)` copies and
validates ordinary text, then returns a borrowed recognizer. No model is loaded
again. The owner must outlive wrappers/sessions; existing single-instance
concurrency restrictions are unchanged. Wrappers are not disposable owners.
Reapplying the method replaces the phrase list, including when applied to a
wrapper. Empty lists return the original recognizer. Unbiased recognizers and
other wrappers remain unchanged. `VadTranscriptionPipeline` accepts a wrapper
without any new options. File and utterance-live chunking retain its context.
Language detection is delegated to the original recognizer without bias.

Common limits are 4096 phrases, 1024 UTF-16 units per phrase, and 262144 total
units. Text must be valid Unicode and non-whitespace; outer whitespace is removed.
Weights must be finite and strictly positive, at most 20. Model-specific limits
can be smaller. No phrases or embeddings are persisted.

- **Whisper:** comma-separated phrases encoded as ordinary text with a leading
  space; special tokens refused. The entire fixed prompt must fit 223 tokens,
  otherwise configuration throws. Each window uses `startofprev`, fixed phrase
  tokens, then the newest generated-context tokens that fit the remaining
  223-token budget, followed by task/language controls. Fixed phrases survive
  fallback context resets. Only generated tokens enter the transcript. This
  uses the prompt-conditioning mechanism described in Radford et al. (2022)
  §2.2; repeating the fixed phrases at each window is this library's policy.
  This is an opt-in hint, not constrained decoding: it can substantially worsen
  WER or repeatedly hallucinate absent prompted terms. Improved term recall alone
  is not evidence that a prompt is safe.
- **Qwen3-ASR:** append comma-separated phrases to the configured `Context`,
  separated by a newline, and use the existing system-context path. Ordinary
  tokenizer encoding validates the entire combined 4096-token budget and
  refuses special tokens. No second biasing implementation is introduced.
- **Prompt weights:** explicit numeric weights are refused with
  `CapabilityNotSupportedException` for Whisper and Qwen. Repeating words or
  adding weight text would not implement a numeric score bonus.
- **Parakeet TDT:** use the publisher's pinned BPE ranks, precompiled character
  normalization and Metaspace word boundaries. Text containing control-token
  syntax or characters that encode as unknown is refused, rather than biasing
  the shared unknown ID. Ordinary token weights feed the phrase trie.
- **Custom graphs:** `TokenVocabulary.WithPhraseEncoder` attaches the caller's
  exact encoding callback to a copy of the vocabulary. No segmentation or
  normalization is guessed from piece lengths. Nonblank ordinary IDs are the
  caller's contract; blank/out-of-range IDs and trie limits are validated.
  Contextual CTC requires at most 4096 columns and uses the configured beam or
  beam 8 when unbiased decoding is greedy. RNN-T/TDT remain greedy.
- **Unsupported families:** Moonshine retains `Biasing=false`; nonempty phrases
  throw `CapabilityNotSupportedException` before recognition, never silently
  ignored. Empty requests are neutral. Invalid/canceled requests still fail.

## Targeted evaluation

`tools/OneSpeech.BiasingEval` prepares a fixed small Earnings-22 comparison. All
audio, references, hypotheses and phrase lists stay in an external state directory,
never committed (Earnings-22 CC-BY-SA-4.0). The original fetch script
`tools/scripts/fetch-earnings22.sh` pins the publisher revision and input hashes.
The diagnostic additionally verifies its five input SHA-256 pins in
`tools/scripts/pins/biasing-earnings22.sha256` before decoding.

Selection is outcome-independent: the two shortest complete recordings by the
pinned metadata's numeric duration, ordinal file-ID tie-break. These are
4474955 (875 seconds) and 4483046 (1187 seconds), about 34 minutes total. Full
human references avoid guessed crop boundaries or alignment-dependent labels.
Phrase candidates have at least five ASCII letters, 1–3 case-insensitive
occurrences in that file's full reference, and at least one capitalized occurrence
away from a sentence start. Select at most 12 in ordinal case-insensitive order.
The scorer requires each selected term to normalize to one word and fails
preparation otherwise. Controls swap the other file's selected list, removing
terms present anywhere in the normalized control reference. This reference-derived
list is a controlled diagnostic, not an estimate of how users choose phrases.

From the repository root, run `prepare CORPUS STATE`, then each
`run FAMILY MODEL STATE CASE MODE` for families `whisper`, `qwen`, `parakeet`,
cases `0`, `1`, and modes `baseline`, `phrases`, `control`. `score STATE` writes
`scores.jsonl`. Model directories are existing publisher-verified converted caches,
never modified. Shard by (family, case, mode); atomic transcripts allow bounded
jobs to resume without rerunning completed cases.

Every mode uses the same public `VadTranscriptionPipeline`, default Silero VAD,
30-second maximum chunks and English hint. Whisper-small uses fp32, temperature
zero and no DTW; Qwen3-ASR-0.6B uses fp32 greedy; Parakeet uses fp32 greedy TDT.
Each uses four graph threads. All fusion weights retain the unmodified default 1;
prompts have no explicit weights. No model parameters or score weights are fitted
on these files.

Both sides use `en/1` normalization and the same word edit alignment. Phrase
recall counts correctly aligned reference occurrences, not bag-of-word presence.
Control false alarms count normalized occurrences of absent control terms in
the hypothesis. Report WER and occurrence counts/recall for all modes per file,
including control WER and its difference from baseline. These two files are
conditional diagnostic floors, not corpus confidence intervals or a general
quality claim; do not aggregate away a per-file regression.

`bootstrap.jsonl` also records paired file-level 95% percentile intervals (1000
resamples, seed 1729) for WER and phrase miss rate (`1 - recall`), comparing each
condition with baseline. With only two files these intervals are very coarse
and conditional on this selection; they do not replace the per-file rows.

### Whisper reference diagnostic

`chunks STATE` exports the same default-Silero, 30-second-bounded PCM chunks used
by the pipeline, reading the original prepared WAV in 4096-frame blocks.
`tools/oracles/whisper-features/biasing.py STATE CASE MODE --weights DIRECTORY`
runs the pinned openai-whisper 20250625 small model through its documented
`transcribe` API, independently per chunk. The comma-space-joined phrase text
is passed as `initial_prompt`; English, fp32, temperature zero, previous-text
conditioning and default rejection thresholds are explicit. Chunk outputs are
atomic and resumable; no model source or exported implementation graph is used.
`score STATE` also scores completed `reference-whisper` results with the same
normalizer, edit alignment and paired file bootstrap.

This comparison reproduced the prompt-induced regression and absent-term
hallucination: prompt conditioning is not a guarantee of improved WER. The
native fixed-prompt retention policy and the reference's initial-prompt option
are not claimed to produce identical decoding. Keep the per-file results and
false alarms alongside term recall; do not treat reference reproduction of a
failure as evidence that the hint is safe.