Recursive AI

Call an OpenAI-compatible server from C#

OneLLM is a free, open-source LLM and agent library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

NuGet GitHub Docs Download example (.zip)

Connect to a server that accepts OpenAI’s Chat Completions request format. Use your server’s model name (e.g. llama.cpp --alias local-model); set CHAT_BASE_URL to its API base URL, including /v1.

Terminal
dotnet add package RecursiveAI.OneLlm

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using OneLlm.Messages;
using OneLlm.Providers.OpenAI;
 
var url = Environment.GetEnvironmentVariable(
    "CHAT_BASE_URL")!;
var server = new OpenAICompatibleProvider("local",
    new() { BaseUrl = new Uri(url) }, TimeProvider.System);
var result = await server.SendAsync(new()
{
    Model = "local-model",
    Messages = [Message.FromUser("Say hello.")],
    MaxOutputTokens = 32, Temperature = 0
});
Console.WriteLine(result.Message.GetAssistantText());

Prints the local model’s answer; no API key is used.

Unzip the example, then dotnet run.

One run with SmolLM2-135M-Instruct on a local llama.cpp server printed:

Text
Hello! I'm here to help you with your questions and concerns.

The full program

For a local server listening on port 8080, set the base URL in a macOS/Linux shell:

Terminal
export CHAT_BASE_URL="http://localhost:8080/v1"

The download contains the client, not a server or model weights. Start your server before dotnet run and set CHAT_BASE_URL in the same terminal. The program also reads CHAT_MODEL, defaulting to local-model; change it to the model identifier your server accepts.

The only request input is the prompt “Say hello.” A temperature of zero requests deterministic sampling, but other models and server settings can still produce different words.

C#
using OneLlm.Messages;
using OneLlm.Providers.OpenAI;
 
var url = Environment.GetEnvironmentVariable("CHAT_BASE_URL")
    ?? throw new InvalidOperationException("Set CHAT_BASE_URL.");
var model = Environment.GetEnvironmentVariable("CHAT_MODEL")
    ?? "local-model";
var provider = new OpenAICompatibleProvider("local",
    new() { BaseUrl = new Uri(url) }, TimeProvider.System);
var result = await provider.SendAsync(new()
{
    Model = model,
    Messages = [Message.FromUser("Say hello.")],
    MaxOutputTokens = 32,
    Temperature = 0
});
Console.WriteLine(result.Message.GetAssistantText());

With the same local model, the full program printed:

Text
Hello! I'm here to help you with your questions and concerns.

If the server cannot be reached, the provider throws TransportException; check the running server and API base URL.

Variations

Read response metadata

Use the same endpoint and model, but inspect the model name and raw stopping reason instead of the answer text.

C#
using OneLlm.Messages;
using OneLlm.Providers.OpenAI;
 
var url = Environment.GetEnvironmentVariable(
    "CHAT_BASE_URL")!;
var server = new OpenAICompatibleProvider("local",
    new() { BaseUrl = new Uri(url) }, TimeProvider.System);
var result = await server.SendAsync(new()
{
    Model = "local-model",
    Messages = [Message.FromUser("Say hello.")],
    MaxOutputTokens = 32, Temperature = 0
});
Console.WriteLine($"Model: {result.Model}");
Console.WriteLine($"Stopped: {result.FinishReasonRaw}");
Text
Model: local-model
Stopped: stop

Set an API key from an environment variable

If your server requires authentication, add this property to the OpenAIOptions initializer (new() { BaseUrl = … }). This configuration line is shown without making an authenticated request; the local example above uses no key.

C#
ApiKey = new OneLlm.Secret(Environment.GetEnvironmentVariable("CHAT_API_KEY")!),

Call Ollama · Test without an API key · Save and resume a chat

The short answer

Use OpenAICompatibleProvider with an explicit OpenAIOptions.BaseUrl, then call SendAsync with the server’s model identifier and your messages. This provider uses Chat Completions. It does not require the remote server to implement the separate Responses protocol.

The example was run against a real local inference server using SmolLM2-135M-Instruct, not a canned response fixture or a paid API. The model produced the answer shown above.

How do I call an OpenAI-compatible server in C#?

There are three separate names to configure. BaseUrl is the server’s API root. Model is the identifier accepted by that server. The provider name, local in the example, labels results and errors; it is not the model name or the address.

Use the API root rather than the complete chat endpoint. OneLLM appends chat/completions to the base URL. If the root ends in /v1, do not append /chat/completions yourself or the path will be repeated. Likewise, a model’s filename is not necessarily its API identifier: this example starts the server with the alias local-model and sends that alias.

Message.FromUser creates the user message. SendAsync returns a ChatResponse; Message.GetAssistantText() reads its text. The variation shows two other response fields without changing the request: the returned model name and the provider’s raw reason for stopping. Keep these fields when diagnosing whether you reached the expected model or hit an output limit.

What is an OpenAI-compatible API?

For this recipe, compatibility means accepting the Chat Completions HTTP request shape and returning an answer in that format. It does not mean that every optional feature is implemented. A text request can succeed even when a server lacks tools, strict JSON schema output, streamed usage or embeddings.

OneLLM’s provider documentation distinguishes OpenAIProvider, which defaults to Responses, from OpenAICompatibleProvider, which speaks Chat Completions. Choose the latter for this task. Changing only a model name does not change the wire protocol.

The library also has catalog presets for known compatible servers. Presets record base URLs and protocol quirks; they do not discover capabilities at runtime. A custom provider gives you explicit control over the endpoint without implying that the server supports every OpenAI feature. Test the particular request features you intend to use against your deployed server.

Can I use a custom OpenAI-compatible endpoint?

Yes. The constructor used above takes OpenAIOptions, including the base URL, optional API key and HTTP client. A missing key sends no authorization header. This example leaves the key unset because the local server does not require authentication. It does not read a cloud key from your environment.

The full program reads CHAT_BASE_URL itself and reports a missing setting before making a request. That variable is a convention of this example, not an environment variable automatically read by the provider. CHAT_MODEL similarly belongs to the example.

A successful connection does not establish that the answer is correct. The tiny model used here is enough to exercise an HTTP request and response, not to establish suitability for your own document analysis or chat workload. Run representative prompts with the model you plan to serve.

Source: the pinned provider guide and the compatible provider constructor.

All onellm recipes