Recursive AI
onellmDocs

Getting started with OneLLM

OneLLM 1.0.0Markdown

All onellm docs

Install OneLLM 1.0.0 for .NET 10, ask a model, get typed JSON back, stream, run a script, an agent with a C# tool and a decider, and see what each call cost. Every sample on this page was run on the 1.0.0 packages from NuGet, without an API key, and shows its real output.

Install

The packages target net10.0, so you need the .NET 10 SDK.

Terminal
dotnet new console -o OneLlmStart
cd OneLlmStart
dotnet add package RecursiveAI.OneLlm --version 1.0.0

Seven packages are on NuGet at 1.0.0, all on one version:

  • RecursiveAI.OneLlm: the core. Provider clients, the model catalog and pricing, templating, the runtime, agents, tools, deciders, file and memory stores, JSONL and event sinks, OpenTelemetry. It brings in OneYaml and OneImage, and no native code.
  • RecursiveAI.OneLlm.Web: the fetch_webpage_markdown tool, on OneHtml.
  • RecursiveAI.OneLlm.JavaScript: JavaScript tools, hooks and decider logic in a sandbox, and run_javascript, on OneEcma.
  • RecursiveAI.OneLlm.Extensions.AI: OneLLM models as IChatClient and friends, and any IChatClient as a OneLLM provider.
  • RecursiveAI.OneLlm.EntityFrameworkCore: an instance store and a record sink on your DbContext.
  • RecursiveAI.OneLlm.OneLite: an instance store and a record sink on a OneLite file.
  • RecursiveAI.OneLlm.Cli: the onellm command-line tool.

Keys

OneLLM reads a provider’s key from the environment when the first call to that provider is made:

ProviderVariable
AnthropicANTHROPIC_API_KEY
OpenAIOPENAI_API_KEY
GeminiGEMINI_API_KEY
TypeSafe Jev (deciders)TYPESAFE_API_KEY

Without one, the call fails before anything is sent:

Text
AuthException: anthropic: No API key. Set the environment variable ANTHROPIC_API_KEY.

Cloud deployments (Azure OpenAI / Foundry, Amazon Bedrock, Vertex AI) take explicit credentials; see deployments. Coverage of providers, from the project’s results: every provider and deployment is replayed from recorded fixtures; a live smoke at release passed against OpenAI (8 of 8) and Gemini (7 of 7); Anthropic, Azure OpenAI / Foundry, Amazon Bedrock and Vertex AI were not run live.

Run it without a key

To try every sample below with no key and no spend, start Program.cs with this block. It registers the library’s own test provider, OneLlm.Testing.ScriptedProvider, under the real provider names, so model names resolve and the shipped catalog prices each call as it would a real one. Each sample then queues the answer the scripted model gives. With a real key, delete this block and the Enqueue lines; the rest is unchanged.

C#
using System.Text.Json.Serialization;
using OneLlm;
using OneLlm.Agents;
using OneLlm.Deciders;
using OneLlm.Messages;
using OneLlm.Testing;
using OneLlm.Tools;
 
var anthropic = new ScriptedProvider("anthropic");
var openai = new ScriptedProvider("openai");
var noKey = new Runtime();
noKey.Providers.Set("anthropic", anthropic);
noKey.Providers.Set("openai", openai);
noKey.Providers.Set("typesafe", new ScriptedDecisionProvider("typesafe"));
noKey.Tools.Add<DocketTools>();
Llm.Default = noKey;

A first call

A model is a catalog name or alias, such as claude-sonnet, gpt-6-luna or gemini-flash; provider:model says which provider serves it (openai:gpt-6-luna, ollama:llama3). GenerateAsync returns the same call with its usage and cost:

C#
anthropic.Enqueue(ScriptStep.TextWithUsage("A monad is a type that wraps a value and chains computations on it.", 14, 15));
string answer = await Llm.AskAsync("claude-sonnet", "In one sentence, what is a monad?");
Console.WriteLine(answer);
 
openai.Enqueue(ScriptStep.TextWithUsage("2, 3 and 5.", 13, 7));
AskResult r = await Llm.GenerateAsync("gpt-6-luna", "Name three prime numbers.", []);
Console.WriteLine($"{r.Text} [{r.Usage?.InputTokens} in, {r.Usage?.OutputTokens} out, {r.Cost.Total}]");
foreach (CostLine line in r.Cost.Lines)
    Console.WriteLine($"  {line.Counter}: {line.Quantity} x {line.UnitPrice} per {line.Per} = {line.Amount}");
Text
A monad is a type that wraps a value and chains computations on it.
2, 3 and 5. [13 in, 7 out, 0.0000048]
  inputTokens: 13 x 0.1 per 1000000 = 0.0000013
  outputTokens: 7 x 0.5 per 1000000 = 0.0000035

Cost is a list of line items plus a decimal total, priced from the catalog (here gpt-6-luna at $0.10 and $0.50 per million tokens, as of 2026-11-05). A model with no price in the catalog gives zero cost and a warning, never a guess. To see a model’s values and where each came from, run onellm catalog <model> (below).

Structured output

The schema comes from your type and is sent through the provider’s native structured output. Passing the source-generated type info keeps the call trim and NativeAOT safe.

C#
openai.Enqueue(ScriptStep.TextWithUsage("{\"customer\":\"ACME Ltd\",\"total\":120.50}", 60, 12));
Invoice invoice = await Llm.AskAsync("gpt-6-luna", "ACME Ltd billed 120.50 for consulting.", AppJson.Default.Invoice);
Console.WriteLine(invoice);
Text
Invoice { Customer = ACME Ltd, Total = 120.50 }

Invoice and AppJson go at the end of Program.cs, with the tool class used further down:

C#
public sealed record Invoice(string Customer, decimal Total);
 
[JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase)]
[JsonSerializable(typeof(Invoice))]
public sealed partial class AppJson : JsonSerializerContext;
 
public sealed partial class DocketTools
{
    /// <summary>Looks up a docket entry.</summary>
    /// <param name="caseNo">Case number, for example 24-1234.</param>
    [Tool]
    public static string GetDocket(string caseNo) => "Docket " + caseNo + ": open";
}

Streaming

Streamed events build the same result a whole call returns.

C#
anthropic.Enqueue(ScriptStep.Text("Green lights at midnight, ", "the queue drains one job at a time, ", "dawn finds the build red."));
StreamingAnswer stream = Llm.StreamAsync("claude-haiku", "Write a haiku about build servers.");
await foreach (string piece in stream.TextAsync()) Console.Write(piece);
AskResult done = await stream.Result;
Console.WriteLine();
Console.WriteLine($"result text matches: {done.Text == "Green lights at midnight, the queue drains one job at a time, dawn finds the build red."}");
Text
Green lights at midnight, the queue drains one job at a time, dawn finds the build red.
result text matches: True

A script

A .flow file is a script: its prompts are templates in Go’s text/template language, rendered in order, one model call per user prompt, no tools. Save this as prompts/commit-message.flow:

YAML
model: gpt-6-luna
systemPrompt: You write git commit messages.
initialPrompts:
  - user:
      - text: "Summarize this change in one sentence: {{ .inputs.diff }}"
  - user:
      - text: "Now write the commit message from your summary: {{ (index .turns 0).text }}"
C#
string diff = "- retries = 1\n+ retries = 3";
openai.Enqueue(ScriptStep.TextWithUsage("Raises the retry count from 1 to 3.", 40, 10),
               ScriptStep.TextWithUsage("Raise retries from 1 to 3\n\nTransient failures now get two more attempts.", 70, 16));
ScriptResult script = await Llm.RunScriptAsync("prompts/commit-message.flow", new Dictionary<string, object?> { ["diff"] = diff });
Console.WriteLine(script.Text);
Console.WriteLine($"calls: {script.Turns.Count}, cost: {script.Cost.Total}");
Text
Raise retries from 1 to 3
 
Transient failures now get two more attempts.
calls: 2, cost: 0.0000240

See templating for the language.

An agent with a C# tool

A tool is a method marked [Tool] on a partial class (DocketTools above). The generator writes the JSON schema from the XML comments and an invoker that uses no reflection. Save the agent as agents/clerk.yaml:

YAML
name: clerk
model: claude-sonnet
systemPrompt: You look up court dockets. Call task_complete with your answer.
inputs:
  caseNo: ""
initialPrompts:
  - user:
      - text: "Find docket {{ .inputs.caseNo }} and report its status."
references:
  - tools: [get_docket, task_complete]

The scripted model calls get_docket, then task_complete:

C#
anthropic.Enqueue(
    new ScriptStep { ToolCalls = [new ScriptedToolCall("c1", "get_docket", "{\"caseNo\":\"24-1234\"}")], Usage = new Usage { InputTokens = 300, OutputTokens = 20 } },
    new ScriptStep { ToolCalls = [new ScriptedToolCall("c2", "task_complete", "{\"result\":\"Docket 24-1234 is open.\",\"summary\":\"Looked up the docket.\"}")], Usage = new Usage { InputTokens = 340, OutputTokens = 25 } });
AgentResult agent = await Llm.RunAgentAsync("agents/clerk.yaml", new Dictionary<string, object?> { ["caseNo"] = "24-1234" });
Console.WriteLine($"{agent.StopReason}: {agent.Text} ({agent.Cost.Total})");
Text
Completed: Docket 24-1234 is open. (0.001730)

Two turns of claude-sonnet (640 tokens in, 45 out) at the catalog’s $2 and $10 per million. The loop ends when task_complete runs, the model answers with no tool call, a call needs approval, or maxIterations is reached. Llm.StartAgentAsync returns a run you can follow, steer and approve; approvals are saved in the instance. See agents.

A decider

A decider asks typed questions of a model and returns the answers. Save this as deciders/triage-ticket/decider.yaml:

YAML
model: jev-latest
state:
  title: "{{ .inputs.title }}"
questions:
  topic:
    type: choice
    instructions: Which team should handle the ticket?
    criteria:
      billing: Charges and refunds
      orders: Delivery and returns
  angry:
    type: noul
    instructions: Is the customer angry?
C#
DecisionResult verdict = await Llm.DecideAsync("deciders/triage-ticket", new Dictionary<string, object?> { ["title"] = "Refund please" });
Console.WriteLine(verdict.Answers.GetRawText());
Text
{"topic":{"type":"choice","choice":"billing","confidence":0.5,"probabilities":{}},"angry":{"type":"noul","noul":0.5}}

These are the scripted engine’s placeholder answers (the first choice, 0.5); a real engine fills in its own. The engine comes from the model’s catalog capabilities: TypeSafe Jev, OpenAI Decisions or any chat model. See deciders.

Grants and spend caps

A definition names tools; the host decides what they may touch. The default runtime allows files under the working directory, with the shell and the network off. To grant more, build a Runtime. The web tool here is in its own package:

Terminal
dotnet add package RecursiveAI.OneLlm.Web --version 1.0.0
C#
var runtime = new Runtime
{
    Capabilities = Capabilities.Files("work")
                 + Capabilities.ReadOnlyFiles("reference")
                 + Capabilities.ShellCommands("git status")
                 + Capabilities.Network("api.example.com"),
    Store = new OneLlm.Instances.FileInstanceStore("runs"),
    SpendCap = 5m,             // dollars, for the whole runtime
    InstanceSpendCap = 1m,
};
runtime.Tools.Add<DocketTools>();
runtime.Tools.Add(new OneLlm.Web.FetchWebpageMarkdownTool());   // RecursiveAI.OneLlm.Web; needs the Network grant

Set it as Llm.Default to use it from the statics. A call that could pass a cap throws BudgetExceededException before anything is sent. See capabilities and the runtime.

From Microsoft.Extensions.AI

With the bridge package, a OneLLM model is an IChatClient, with its cost in AdditionalProperties:

Terminal
dotnet add package RecursiveAI.OneLlm.Extensions.AI --version 1.0.0
C#
anthropic.Enqueue(ScriptStep.TextWithUsage("Hello from the bridge.", 9, 5));
Microsoft.Extensions.AI.IChatClient client = OneLlm.Extensions.AI.RuntimeExtensions.AsChatClient(noKey, "claude-sonnet");
var response = await client.GetResponseAsync([new Microsoft.Extensions.AI.ChatMessage(Microsoft.Extensions.AI.ChatRole.User, "Say hello.")]);
Console.WriteLine($"{response.Text} {response.AdditionalProperties?["onellm.cost"]}");
Text
Hello from the bridge. $0.000068

With using OneLlm.Extensions.AI; the same is runtime.AsChatClient("claude-sonnet"). See the bridge, and coming from another library for call-for-call tables.

The onellm tool

Terminal
dotnet tool install --global RecursiveAI.OneLlm.Cli --version 1.0.0
onellm catalog claude-sonnet
Text
model: anthropic:claude-sonnet-5-5
provider: anthropic
name: "claude-sonnet-5-5"	[shipped]
displayName: "Claude Sonnet 5.5"	[shipped]
contextWindow: 1000000	[shipped]
…
pricing.inputPerMillion: 2	[shipped]
pricing.outputPerMillion: 10	[shipped]
pricing.asOf: "2026-11-05"	[shipped]

onellm run runs an agent, a .flow or a decider. Here the flow above, with its model changed to ollama:llama3, runs against a local OpenAI-compatible server on Ollama’s port (for this page, a stand-in that answers with fixed text). The result goes to standard output; the cost line to standard error:

Terminal
onellm run commit-message.flow --input diff=$'- retries = 1\n+ retries = 3'
onellm cost
Text
Raise retries from 1 to 3
 
Transient failures now get two more attempts before the job fails.
script i-8d48971163424f3ebc6fa8efe5c2a255 · 2 turns · cost $0.000000
llama3	2 calls	80 in	24 out	$0.000000
1 instance: $0.000000

Presets for servers on your own machine are priced at zero. The run is saved as a readable YAML instance under .onellm/instances/. Without a key for a hosted model the tool exits with code 3 and names the variable to set. See the onellm tool and the example agents it embeds.

Next