Recursive AI

Configure LLM retries in C#

OneLLM is a free, open-source LLM and agent library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

Set bounded retries on a provider to handle temporary rate limits and connection failures.

Terminal
dotnet add package RecursiveAI.OneLlm

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using OneLlm.Http;
using OneLlm.Providers.OpenAI;
 
var options = new OpenAIOptions
{
    Retry = new() { MaxRetries = 2 }
};
Console.WriteLine(
    options.Retry.MaxRetries);

Unzip the example, then dotnet run.

The zip holds the full program below. To run only the snippet at the top, paste it into Program.cs of a new console app and install the package above. That snippet prints:

Text
2

The snippet configures the options; retries happen inside the provider’s SendAsync when those options are passed to its constructor, as in the full program.

The same RetryPolicy type is accepted by AnthropicOptions.Retry and GeminiOptions.Retry, as well as OpenAIOptions.Retry for OpenAI and compatible providers.

The full program

The download sends a chat request through a provider with an in-memory HTTP handler. The handler returns an error once, then a successful JSON response; no URL is contacted and no API key is used. It exercises rate limiting, overload, authentication and bad requests. The 1 ms base delay is for this offline example, not a recommended production setting.

A non-retryable error, or one that remains after the retry budget is used, throws a typed ProviderException.

C#
using System.Net;
using System.Text;
using OneLlm;
using OneLlm.Http;
using OneLlm.Messages;
using OneLlm.Providers.OpenAI;
 
foreach (int status in new[] { 429, 503, 401, 400 })
{
    using var handler = new ReplyHandler(status);
    using var http = new HttpClient(handler);
    var provider = new OpenAICompatibleProvider("demo", new()
    {
        BaseUrl = new Uri("https://example.invalid/v1"),
        HttpClient = http,
        Retry = new RetryPolicy
        {
            MaxRetries = 2,
            BaseDelay = TimeSpan.FromMilliseconds(1),
            MaxDelay = TimeSpan.FromMilliseconds(4),
            JitterFraction = 0
        }
    }, TimeProvider.System);
    try
    {
        var result = await provider.SendAsync(new()
        {
            Model = "demo-model",
            Messages = [Message.FromUser("Say hello.")]
        });
        Console.WriteLine($"{status}: {result.Message.GetAssistantText()} ({handler.Calls} attempts)");
    }
    catch (ProviderException error)
    {
        Console.WriteLine($"{status}: {error.GetType().Name} ({handler.Calls} attempt)");
    }
}
 
// Entirely in memory: the URL is never contacted.
sealed class ReplyHandler(int firstStatus) : HttpMessageHandler
{
    public int Calls { get; private set; }
    protected override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request, CancellationToken cancellationToken)
    {
        int status = ++Calls == 1 ? firstStatus : 200;
        string body = status == 200
            ? """{"id":"demo","object":"chat.completion","model":"demo-model","choices":[{"index":0,"message":{"role":"assistant","content":"Hello."},"finish_reason":"stop"}]}"""
            : """{"error":{"message":"Try again","type":"demo_error"}}""";
        return Task.FromResult(new HttpResponseMessage((HttpStatusCode)status)
        {
            Content = new StringContent(body, Encoding.UTF8, "application/json")
        });
    }
}

Running dotnet run from the unzipped example prints:

Text
429: Hello. (2 attempts)
503: Hello. (2 attempts)
401: AuthException (1 attempt)
400: BadRequestException (1 attempt)

Variations

Turn retries off when another layer already owns retrying:

C#
using OneLlm.Http;
using OneLlm.Providers.OpenAI;
 
var options = new OpenAIOptions { Retry = RetryPolicy.None };
Console.WriteLine(
    options.Retry.MaxRetries);
Text
0

The short answer

Set OpenAIOptions.Retry to a RetryPolicy. MaxRetries counts extra attempts, not the initial request: a value of 2 allows at most three attempts. The provider retries transient failures; it doesn’t treat every unsuccessful response as something another attempt can fix. The full example uses the real provider transport with scripted HTTP responses, rather than simulating retries around a model call.

How do I retry LLM rate-limit errors in C#?

Use the provider’s built-in retry policy for rate-limit responses. The retry documentation lists HTTP 408, 429, 500, 502, 503, 504 and 529, plus transport failures, as retryable. A quota-exhausted 429 is different from a temporary rate limit and is not retried.

The full program distinguishes these cases by returning a plain rate-limit response once, then a valid completion. Its attempt counter increases twice for 429 and 503. It increases only once for 401 and 400, because retrying those statuses does not repair the credentials or request. AuthException and BadRequestException let the caller decide what to fix instead.

Keep retries bounded. After the allowed retries are exhausted, the error reaches your application; the library does not keep the request pending forever. Treat that as a normal failure path in your caller. Do not put an unbounded retry loop around it, which would undo the bound you just configured.

How does exponential backoff handle Retry-After?

OneLLM’s default policy allows 2 retries, starting with a 500 ms base delay, a 30 s backoff cap and 25% jitter. The delay doubles between attempts. Jitter adds a random component to the wait rather than making every caller use an identical interval.

A provider can ask for a longer wait through Retry-After, including an HTTP date, or retry-after-ms. The longer provider-requested delay wins over the calculated backoff. MaxDelay caps calculated backoff; MaxRetryAfter separately bounds the server-requested wait. The default accepted maximum is 60 s. A longer request is returned as an error immediately, not slept through and not silently shortened.

These settings bound different things. Choose the number of extra attempts for the operation you are performing, and choose how long the caller can reasonably remain waiting. A backoff cap alone is not a total timeout across all attempts.

How can I test LLM retries without an API key?

Inject an HttpClient backed by a custom HttpMessageHandler, as the download does. The provider still constructs a request, reads the JSON response and classifies the HTTP error. Only the network is replaced. The complete response bodies and the handler are shown in Program.cs, so you can change the first status and rerun the example.

This is a transport test, not a live service check. A scripted model exception tests your application’s error handling, but it does not by itself exercise HTTP retry behavior. Keep those two tests separate.

Cancellation ends a request or a retry wait with OperationCanceledException. The documented timeout applies per attempt up to response headers; for streaming responses it is not an idle timeout for the rest of the stream. See the error and retry reference when choosing those bounds.

All onellm recipes