---
title: "Cache LLM responses in C#"
url: https://recursiveai.org/onellm/cookbook/cache-llm-responses/
description: "Cache identical LLM requests in C# with OneLLM’s in-memory result cache. Run an offline example that shows a hit and a changed-request miss."
version: "1.0.1"
---

# Cache LLM responses in C#

**OneLLM** is a free, open-source LLM and agent library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

[NuGet](https://www.nuget.org/packages/RecursiveAI.OneLlm) · [GitHub](https://github.com/recursive-ai-oss/onellm-csharp) · [Docs](https://recursiveai.org/onellm/docs/) · [Download example (.zip)](https://recursiveai.org/onellm/cookbook/cache-llm-responses/cache-llm-responses.zip)

Reuse an identical chat request’s stored answer. The example runs offline, without an API key.

```shell
dotnet add package RecursiveAI.OneLlm
```

Requires [.NET 10](https://dotnet.microsoft.com/download/dotnet/10.0) (net10.0); .NET 9 and earlier aren’t supported.

```csharp
using OneLlm.Caching;
using OneLlm.Providers;
using OneLlm.Messages;
using OneLlm.Testing;

var provider = new ScriptedProvider(
    "offline");
provider.Enqueue(
    ScriptStep.Text("Paris"));
var cached = new CachingChatProvider(
    provider, new ResultCache());
var request = new ChatRequest {
    Model = "demo",
    Messages =
        [Message.FromUser("Capital of France?")]
};
await cached.SendAsync(request);
await cached.SendAsync(request);
Console.WriteLine($"Provider calls: {provider.Requests.Count}");
```

Unzip [the example](https://recursiveai.org/onellm/cookbook/cache-llm-responses/cache-llm-responses.zip), then `dotnet run`.

Prints the number of calls that reached the offline provider:

```text
Provider calls: 1
```

## The full program

The download queues two fixed answers, asks the France question twice, then changes the question to Italy. The answers are scripted, not generated by a model. The cache and its provider wrapper are the real OneLLM implementations.

```csharp
using OneLlm;
using OneLlm.Caching;
using OneLlm.Messages;
using OneLlm.Providers;
using OneLlm.Testing;

var provider = new ScriptedProvider("offline");
provider.Enqueue(ScriptStep.Text("Paris"));
provider.Enqueue(ScriptStep.Text("Rome"));
var cache = new ResultCache(capacity: 32);
var cached = new CachingChatProvider(provider, cache);
var request = new ChatRequest
{
    Model = "demo",
    Messages = [Message.FromUser("Capital of France?")],
    Temperature = 0
};
Console.WriteLine((await cached.SendAsync(request)).Message.GetAssistantText());
Console.WriteLine((await cached.SendAsync(request)).Message.GetAssistantText());
Console.WriteLine($"Provider calls: {provider.Requests.Count}");
var changed = request with
{
    Messages = [Message.FromUser("Capital of Italy?")]
};
Console.WriteLine((await cached.SendAsync(changed)).Message.GetAssistantText());
Console.WriteLine($"Provider calls: {provider.Requests.Count}");
Console.WriteLine($"Cached answers: {cache.Count}");
```

`dotnet run` prints:

```text
Paris
Paris
Provider calls: 1
Rome
Provider calls: 2
Cached answers: 2
```

The repeated France request uses the stored answer. The changed message reaches the provider and adds another entry. Provider exceptions propagate to the caller; the wrapper stores an answer only after `SendAsync` succeeds.

## Variations

**Clear stored answers.** Append this to the full program when your source data changes:

```csharp
cache.Clear();
Console.WriteLine($"Cached answers: {cache.Count}");
```

```text
Cached answers: 0
```

**Wrap a real provider.** This standalone setup needs your OpenAI API key in `OPENAI_API_KEY`. It was compiled against OneLLM 1.0.1, not run against a live model; there is no live output shown.

```csharp
using OneLlm;
using OneLlm.Caching;
using OneLlm.Providers.OpenAI;

string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")
    ?? throw new InvalidOperationException("Set OPENAI_API_KEY.");
var provider = new OpenAIProvider(
    new OpenAIOptions { ApiKey = new Secret(key) },
    TimeProvider.System);
var cached = new CachingChatProvider(provider, new ResultCache());
```

Send requests through `cached.SendAsync` using a model available to your account. Keep the wrapper and cache alive for subsequent calls; the first uncached request goes to the service. The [provider guide](https://recursiveai.org/onellm/docs/providers-openai/) describes the default Responses protocol.

## Related recipes

- [Test LLM code without an API key](https://recursiveai.org/onellm/cookbook/test-llm-code-without-api-key/)
- [Save and resume a chat](https://recursiveai.org/onellm/cookbook/save-and-resume-a-chat/)

## The short answer

Wrap an `IChatProvider` in `CachingChatProvider`, give it a `ResultCache`, and keep that wrapper for later requests. Repeated `SendAsync` calls with the same key return the stored `ChatResponse` without calling the wrapped provider. You must opt in: creating a normal provider does not enable this cache.

The cache lives in your process. It does not install a database, create a disk cache, or configure a provider’s server-side cache. The complete example runs offline so you can check request counts without credentials. In an application, wrap the chat provider you already use instead of `ScriptedProvider`.

## What LLM caching techniques does this use?

This is exact request-response caching: the application stores a completed answer and reuses it for the same request. OneLLM computes a SHA-256 key from the provider identity and the request’s answer-affecting fields. Those include the model, system prompt, messages, tools, response format, reasoning settings, output limit, sampling settings, stop sequences, service tier and extra parameters. The wrapper includes both provider name and deployment in that identity.

Keep the same cache instance alive across calls. Creating a new `ResultCache` for every request leaves no stored answers to reuse. Its capacity is a count of answers, not a byte limit; once full, it drops the least recently used entry. The constructor defaults to 256 entries, while the full example chooses 32. These behaviors are defined in the [result-cache implementation](https://github.com/recursive-ai-oss/onellm-csharp/blob/ea17c434f4e469046ea583ad334deb6a8974e28b/src/OneLlm/Caching/ResultCache.cs).

Use caching when returning the earlier answer is acceptable. A repeated request at a nonzero temperature will still get the stored answer, not a new sample. The example sets temperature to zero to make that reuse explicit. Clearing the cache is useful when external facts change without changing the prompt. There is no expiration setting on this cache; decide when your application should clear it or replace it.

## Is this a semantic cache for an LLM?

No. The built-in result cache does not search embeddings or accept a similar question as a match. “Capital of France?” and “What is France’s capital?” are different request text and therefore different keys. This is useful when you need an exact repeat, not an approximate answer selected by similarity.

The full program’s Italy request is a deliberate miss. It demonstrates that keeping the same model and sampling settings does not make different messages share an answer. Changing tool definitions or the requested response format also changes the key. Do not use this cache as a retrieval system for related questions.

## Is LLM prompt caching the same as response caching?

No. Provider prompt caching and local result caching operate at different boundaries. OneLLM’s result-cache wrapper can return a complete response without sending a request. Provider caching is configured on requests sent to a service. The library documents those separate controls in [pricing and counting](https://recursiveai.org/onellm/docs/pricing-and-counting/).

`ChatRequest.Cache` and `UserId` are excluded from the result-cache key: the implementation describes them as settings that steer provider caching or abuse monitoring rather than the answer. They are not a way to partition cached responses between users. Choose cache ownership deliberately when callers should not share stored answers.

Streaming is another separate path. `CachingChatProvider.StreamAsync` passes directly to the inner provider and never caches the stream. Use `SendAsync` for this recipe; wrapping a provider alone does not make streamed calls reuse earlier answers.
