Recursive AI

Unit test LLM calls in C# without an API key

OneLLM is a free, open-source LLM and agent library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

NuGet GitHub Docs

Code that calls a model is hard to test: every run costs money, needs a key and gives a different answer. OneLLM ships a scripted provider that answers in the model’s place, so tests run offline and give the same result every time. It works for code that calls models through OneLLM; for code on another client, wrap the client in an interface or run a fake HTTP server (both are compared below).

Terminal
dotnet add package RecursiveAI.OneLlm

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using OneLlm;
using OneLlm.Testing;
 
var openai = new ScriptedProvider("openai");
var runtime = new Runtime();
runtime.Providers.Set("openai", openai);
Llm.Default = runtime;
 
openai.EnqueueText("billing");
string team = await Llm.AskAsync(
    "gpt-6-luna", "Route: charged twice");
Console.WriteLine(team);

Download this example (.zip): unzip, then dotnet run.

Prints billing. No key is read and nothing goes over the network: the scripted provider is registered under the name openai, so gpt-6-luna resolves to it instead of the OpenAI API. gpt-6-luna is a model name from OneLLM’s model catalog, which maps names and aliases to a provider and its prices.

The full program

This is the Program.cs in the download. Triage.RouteAsync is the code under test: it asks gpt-6-luna to route a support ticket and checks the answer. The program runs four tests against it.

C#
using OneLlm;
using OneLlm.Testing;
 
// The library's scripted provider stands in for OpenAI: no key, no network, no spend.
var openai = new ScriptedProvider("openai");
var runtime = new Runtime();
runtime.Providers.Set("openai", openai);
Llm.Default = runtime;
 
// Test 1: the answer is parsed.
openai.Enqueue(ScriptStep.TextWithUsage("billing", 52, 1));
string team = await Triage.RouteAsync("I was charged twice for my order.");
Check("routes a double charge to billing", team == "billing");
 
// Test 2: the prompt holds the ticket and the allowed teams.
string sent = openai.Requests[^1].Messages[^1].GetUserText();
Check("prompt contains the ticket", sent.Contains("charged twice"));
Check("prompt lists the teams", sent.Contains("billing, orders, other"));
 
// Test 3: an answer outside the list falls back to "other".
openai.Enqueue(ScriptStep.TextWithUsage("Sure! I think this is about shipping.", 50, 9));
Check("unexpected answer becomes other", await Triage.RouteAsync("Where is my parcel?") == "other");
 
// Test 4: a rate limit reaches the caller as OneLLM's typed exception.
openai.Enqueue(ScriptStep.Fail(new RateLimitException("openai", "Too many requests", TimeSpan.FromSeconds(20))));
try { await Triage.RouteAsync("Hello?"); Check("a rate limit is thrown", false); }
catch (RateLimitException e) { Check($"a rate limit is thrown (retry after {e.RetryAfter?.TotalSeconds}s)", true); }
 
// The calls were priced from the catalog as real gpt-6-luna calls would be.
Console.WriteLine($"{openai.Requests.Count} calls, model {openai.Requests[0].Model}, spent ${runtime.Spent}");
 
static void Check(string name, bool ok) => Console.WriteLine($"{(ok ? "PASS" : "FAIL")}  {name}");
 
// The code under test: it calls a model through OneLLM's static API.
static class Triage
{
    static readonly string[] Teams = ["billing", "orders", "other"];
 
    public static async Task<string> RouteAsync(string ticket)
    {
        string answer = await Llm.AskAsync("gpt-6-luna",
            $"Route this support ticket to one team ({string.Join(", ", Teams)}). " +
            $"Answer with the team name only.\n\nTicket: {ticket}");
        string team = answer.Trim().ToLowerInvariant();
        return Teams.Contains(team) ? team : "other";
    }
}

dotnet run prints:

Text
PASS  routes a double charge to billing
PASS  prompt contains the ticket
PASS  prompt lists the teams
PASS  unexpected answer becomes other
PASS  a rate limit is thrown (retry after 20s)
3 calls, model gpt-6-luna, spent $0.0000152

Each Enqueue adds the answer for the next request, in order. openai.Requests holds every request the code sent, so a test can check the prompt as well as the result. The last line shows that the scripted calls were priced like real ones: 102 input and 10 output tokens of gpt-6-luna at the catalog’s $0.10 and $0.50 per million.

If the code makes more requests than the script holds, the provider throws InvalidOperationException, so a test catches an unexpected extra call. On a first request with nothing queued, the message reads “The scripted provider 'openai' received request 1 but its script is empty.” The same code in production calls OpenAI: drop the four setup lines and set OPENAI_API_KEY.

Variations

Each snippet runs after the four setup lines of the snippet at the top.

A streamed answer. Each string is sent as its own piece, as a real stream would arrive:

C#
openai.Enqueue(ScriptStep.Text("bill", "ing"));
await foreach (string piece in Llm.StreamTextAsync("gpt-6-luna", "Route: charged twice"))
    Console.Write($"[{piece}]");
Console.WriteLine();
Text
[bill][ing]

The same answer for any number of calls, for code that calls the model in a loop:

C#
openai.Fallback = request => ScriptStep.Text("other");
for (int i = 0; i < 3; i++)
    Console.Write(await Llm.AskAsync("gpt-6-luna", $"Ticket {i}") + " ");
Console.WriteLine($"({openai.Requests.Count} requests)");
Text
other other other (3 requests)

Check the model, the system prompt and the message your code sent:

C#
openai.EnqueueText("billing");
using (var chat = Llm.Chat("gpt-6-luna", "You route support tickets."))
    await chat.AskAsync("Charged twice for order 1182");
 
var request = openai.Requests[^1];
Console.WriteLine(request.Model);
Console.WriteLine(request.System);
Console.WriteLine(request.Messages[^1].GetUserText());
Text
gpt-6-luna
You route support tickets.
Charged twice for order 1182

Test cost logic. Give the step a token usage and the catalog prices it: 1,000 input and 200 output tokens of gpt-6-luna cost $0.0002. GenerateAsync returns the answer with its usage and cost; its third argument is a list of attachments (images, PDFs), and [] sends none.

C#
openai.Enqueue(ScriptStep.TextWithUsage("billing", 1_000, 200));
AskResult r = await Llm.GenerateAsync("gpt-6-luna", "Route: charged twice", []);
Console.WriteLine(r.Cost.Total == 0.0002m ? "cost ok" : $"cost {r.Cost.Total}");
Text
cost ok

Simulate a failure. The step throws the exception you give it, so you can test timeouts and provider errors:

C#
openai.Enqueue(ScriptStep.Fail(new TimeoutException("model too slow")));
try { await Llm.AskAsync("gpt-6-luna", "Route: charged twice"); }
catch (TimeoutException e) { Console.WriteLine($"caught: {e.Message}"); }
Text
caught: model too slow

As an xUnit test. The same setup inside a [Fact], run with dotnet test in a project made by dotnet new xunit:

C#
using OneLlm;
using OneLlm.Testing;
using Xunit;
 
[Collection("Llm.Default")] // tests that replace Llm.Default must not run in parallel
public class TriageTests
{
    [Fact]
    public async Task Routes_a_double_charge_to_billing()
    {
        var openai = new ScriptedProvider("openai").EnqueueText("billing");
        var runtime = new Runtime();
        runtime.Providers.Set("openai", openai);
        Llm.Default = runtime;
 
        string team = await Llm.AskAsync("gpt-6-luna", "Route: charged twice");
 
        Assert.Equal("billing", team);
        Assert.Contains("charged twice", openai.Requests[0].Messages[^1].GetUserText());
    }
}
Text
Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 124 ms - TriageTests.dll (net10.0)

The short answer

To unit test LLM calls in C#, put a fake provider where the real one would be. With OneLLM, create a ScriptedProvider, register it on a Runtime under the provider’s name (openai, anthropic, gemini), and set that runtime as Llm.Default. Your code keeps calling Llm.AskAsync("gpt-6-luna", …). The provider answers with the steps you queued, records every request, and needs no key and no network.

The project’s docs describe it this way: “OneLlm.Testing.ScriptedProvider replays a script of answers (text, tool calls, usage, failures, pauses) and records the requests it received. Streaming and whole answers go through the same events, so they build equal messages.” It ships in the main package, RecursiveAI.OneLlm, which is free and open source under MIT or Apache 2.0. The rest of this page answers the questions people ask next.

How do you mock the OpenAI API in C#?

There are three common ways, and they test different things.

  • Wrap the client in your own interface and mock the interface. This tests your logic, but not how your prompt, your types or your parsing meet the library.
  • Run a fake HTTP server that speaks the OpenAI API. This tests the whole HTTP path, but you have to write the server and keep it in step with the API.
  • Swap the provider inside the library. OneLLM’s statics (Llm.AskAsync, Llm.GenerateAsync, Llm.StreamTextAsync, Llm.Chat, Llm.RunAgentAsync) all go through Llm.Default, a Runtime. Register a ScriptedProvider on it and every layer above the provider runs as it would in production: model names resolve through the catalog, prompts are built, structured output is parsed, agent tools run, and cost is priced.

The third needs no interface of your own and no server. Because the scripted provider is registered under the name openai, the model name in your code doesn’t change between the test and production.

One rule from the library’s own documentation of Llm.Default: tests that replace it “must not run in parallel with others that use the statics.” In xUnit, put those tests in one collection, as in the variation above.

What should a unit test of LLM code check?

A unit test can’t judge whether a model’s answer is good: the scripted provider returns exactly what you queued. What it can check is everything your code does around the model, and that is where most bugs are:

  • The request. openai.Requests holds each ChatRequest: Model, System, Messages, Tools, ResponseFormat, Temperature and the rest. Check that the prompt holds the data it should, and only that.
  • Parsing. Queue answers your parser must handle: the expected one, extra words around it, the wrong case, empty text. In the full program, “Sure! I think this is about shipping.” must become other.
  • Failures. ScriptStep.Fail throws whatever you pass. OneLLM’s own types are worth testing against: RateLimitException (with RetryAfter), OverloadedException, AuthException, TransportException, BudgetExceededException.
  • Cost. ScriptStep.TextWithUsage reports token counts, and the shipped catalog prices them, so a test can check a budget or a cost report with exact decimal amounts.
  • Agents. A step can carry tool calls (ScriptedToolCall), so a test can drive an agent through its tools and check what each one returned. The agent recipe does this.

How do you test LLM responses themselves?

That’s a different job from unit testing. Judging whether a model answers well needs real model calls on a set of example inputs, scored by rules or by people, and it costs money each time it runs. Keep those runs out of the unit test suite and run them on purpose, for example before changing the model or the prompt.

Two things from this page still help there. The same code runs against a real model when you drop the scripted setup and set the provider’s key. And every call reports its cost, so you can see what an evaluation run spent (see Track OpenAI API cost per call). For free real answers during development, point the same code at a local model with Ollama.

About live coverage: OneLLM’s results say every provider is tested by replaying recorded fixtures, and a live smoke test at release passed against OpenAI (8 of 8 cases) and Gemini (7 of 7). Anthropic, Azure OpenAI / Foundry, Amazon Bedrock and Vertex AI were not run live.

All onellm recipes