--- title: "Automatic embedding and the embedding cache" url: https://recursiveai.org/oneindex/docs/auto-embedding/ description: "This note is normative for [Vector(From = ...)], the embedder an index uses, and the cache that keeps unchanged text from being embedded again." version: "0.4.0-preview.2" source: https://github.com/recursive-ai-oss/oneindex-csharp/blob/177f49bb3974e9e44df6d1a800342a9d353a0984/docs/auto-embedding.md --- # Automatic embedding and the embedding cache This note is normative for `[Vector(From = ...)]`, the embedder an index uses, and the cache that keeps unchanged text from being embedded again. ## Declaring it ```csharp public class Article { [SearchId] public string Id { get; set; } [FullText] public string Body { get; set; } [Vector(From = nameof(Body), Dimensions = 384)] public float[]? Embedding { get; set; } } ``` `From` names a text or keyword member of the same type (a build-time check in the schema: otherwise `InvalidOperationException` naming the member). `Dimensions` may be left out when `From` is set: the index then takes the embedder's `Dimension` when it is opened or created, and opening it without an embedder throws `InvalidOperationException` naming the member (`SearchTool.For()` needs no embedder). When `Dimensions` is declared, an embedder whose `Dimension` differs is refused when the index opens (`InvalidOperationException` naming both). Either way the length is recorded in the index's schema, so an index made with one length refuses to open with another. A non-nullable `float[]` property needs an initial value; an empty array means no vector was supplied. A multi-valued source is joined with a line feed. ## Which embedder There is no options object on `SearchIndex.Open`. The embedder comes from, in this order: 1. the argument: `SearchIndex.Open(path, embedder)` and `SearchIndex.InMemory(embedder)` (`null` = none); 2. `SearchEmbedders.Default`, set once at startup by the application, or by `BgeSmallEmbedder.UseAsDefault()` in `RecursiveAI.OneIndex.Models`, which loads the packed model and sets it. Nothing registers itself: no reflection, no module initializer, so referencing `.Models` alone changes nothing and the behaviour is the same under NativeAOT and trimming. `SearchIndex.Embedder` returns the one in use. An index without an embedder accepts documents without vectors and searches stay text-only; a vector the application sets on the document is always kept and never overwritten. ## When text is embedded * **Upsert** (`Upsert`, `Upsert(IEnumerable)`, `UpsertAsync`): before the document reaches the writer, with the document role. `Upsert(IEnumerable)` embeds in batches of at most 32 distinct texts (the documents are taken 64 at a time), `UpsertAsync` uses `IEmbedder.EmbedAsync`. Nothing is embedded at commit, flush or merge. * **Search**: the search text is embedded with the *query* role when a search is hybrid (`hybrid.md`), and cached in memory (the last 256 texts), so paging or repeating a search embeds once. **Failure.** If the embedder throws, returns a NaN or an infinity, or is cancelled, `Upsert` throws (`EmbeddingException` wrapping the cause and naming the model; `OperationCanceledException` for cancellation) **before** anything of the call is written: a batch call writes none of its documents' batch in that group of 64, and no half-embedded document ever exists, because the document is complete before the writer sees it. Documents of earlier groups in the same call stay buffered, as they would after any other exception. The index stays usable. ## The cache A cache entry is the SHA-256 of the model identity (`IEmbedder.ModelId`), a zero byte, and the UTF-8 source text. A document stores its entry in a hidden bytes field `#embedding` of the stored source, next to the vector (the stored source keeps the original, un-normalised vector). Writing it into the schema makes it part of the type's field list; an index created before the member had `From` gains the field through the normal additive schema evolution, and its documents count as stale until re-embedded. An upsert looks the text up in this order and embeds only on a miss: 1. an in-memory map of recent entries (up to 8,192 vectors; cleared when full), which serves identical texts of different documents and documents not yet committed; 2. the stored source of the committed document with the same identifier: same entry, so the stored vector is reused. Hence: re-upserting the same documents (a reindex from a database), reopening the index, merges, and schema-evolution rebuilds embed nothing. Merges and rebuilds move stored documents, vectors included, and never reference the embedder. Changing the text changes the entry, so only that document is embedded. Changing the model identity changes every entry: * new upserts embed again under the new identity; * `SearchIndex.ReembedStale()` (and `ReembedStaleAsync`) embeds again every committed document whose entry is missing or differs, buffering the replacements for the next commit, and returns how many; unchanged documents cost nothing. Call it after switching the embedder. A model change is not a schema difference, so it does not start the background schema rebuild by itself; `ReembedStale` is the explicit step. A model identity must change whenever its output changes (`IEmbedder.ModelId` documents this), and it also covers a changed query instruction or quantization. ## Durability Embedding happens before the write, so the commit protocol is unchanged: a document is either in a commit with its vector and entry or not in it. The kill campaign `tests/Durability` (`embed-kill`) runs a writer child that upserts through the index with a slow deterministic embedder, single and batched, commits every few operations and is killed at random moments; after each kill the directory must reopen, hold exactly the last acknowledged commit or the one in flight, every document must carry the vector its text determines, and a vector search for it must find it. ## Limits One vector field per `From` member; the source member must fit the model's input (the model cuts, it does not refuse). Texts are embedded in the caller's thread; a slow embedder slows `Upsert`. ## Sparse and multi-vector members `[Sparse(From = ...)]` (docs/sparse.md) and `[MultiVector(From = ...)]` (docs/multi-vector.md) use the same scheme with their own encoders (`ISparseEncoder`, `IMultiVectorEncoder`; defaults `SearchSparseEncoders.Default` and `SearchMultiVectorEncoders.Default`) and their own hidden cache fields (`#sparse`, `#multivector`). `ReembedStale` handles all three kinds in one pass.