---
title: "Detect the encoding of an HTML page"
url: https://recursiveai.org/onehtml/cookbook/detect-encoding/
description: "Decode HTML bytes the way a browser does, using the byte order mark, the HTTP Content-Type header, the meta charset prescan and the windows-1252 fallback…"
version: "1.0.0"
---

# Detect the encoding of an HTML page

Decode HTML bytes the way a browser does, using the byte order mark, the HTTP Content-Type header, the meta charset prescan and the windows-1252 fallback, and override the fallback when you know better.

`Document.Parse(bytes)` implements the WHATWG Encoding Standard and the HTML encoding-sniffing algorithm. The input here is a French menu saved as windows-1252, which declares itself with `<meta charset="windows-1252">`:

```html
<!DOCTYPE html>
<html lang="fr">
<head>
<meta charset="windows-1252">
<title>Menu</title>
</head>
<body>
<h1>Café crème – 3,50 €</h1>
</body>
</html>
```

```csharp
using System.Text;
using RecursiveAI.OneHtml;
using RecursiveAI.OneHtml.Encodings;

// A windows-1252 page that declares its charset in a <meta>
byte[] declared = File.ReadAllBytes("menu-1252.html");
Document doc = Document.Parse(declared);
Console.WriteLine($"{doc.Source.Encoding.Name} ({doc.Source.Confidence}): {doc.Root.QuerySelector("h1").TextContent}");

// Sniff without parsing: BOM, then transport Content-Type, then the <meta> prescan
EncodingSniffResult sniffed = EncodingSniffer.Sniff(declared);
Console.WriteLine($"Sniffed: {sniffed.Encoding.Name}, {sniffed.Confidence}");

// The HTTP header wins over the <meta> when you pass it
Document fromHeader = Document.Parse(declared, new ParseOptions { ContentType = "text/html; charset=utf-8" });
Console.WriteLine($"With a UTF-8 header: {fromHeader.Source.Encoding.Name}, {fromHeader.Source.DecodeErrorCount} decode errors");

// A byte order mark beats everything
byte[] bom = [0xEF, 0xBB, 0xBF, .. Encoding.UTF8.GetBytes("<meta charset=windows-1252><p>café")];
Console.WriteLine($"With a BOM: {Document.Parse(bom).Source.Encoding.Name}");

// No BOM, no header, no <meta>: the last-resort default is windows-1252
byte[] bare = Encoding.UTF8.GetBytes("<p>café");
Document guessed = Document.Parse(bare);
Console.WriteLine($"Undeclared: {guessed.Source.Encoding.Name} ({guessed.Source.Confidence}): {guessed.Root.QuerySelector("p").TextContent}");

// Choose the fallback yourself when you know your sources
Document utf8 = Document.Parse(bare, new ParseOptions { DefaultEncoding = WhatwgEncoding.Utf8 });
Console.WriteLine($"Default set to UTF-8: {utf8.Root.QuerySelector("p").TextContent}");

// Labels resolve the way browsers resolve them
Console.WriteLine($"latin1 -> {WhatwgEncoding.ForLabel("latin1")}, gb2312 -> {WhatwgEncoding.ForLabel("gb2312")}");
```

```text
windows-1252 (Certain): Café crème – 3,50 €
Sniffed: windows-1252, Tentative
With a UTF-8 header: UTF-8, 4 decode errors
With a BOM: UTF-8
Undeclared: windows-1252 (Tentative): café
Default set to UTF-8: café
latin1 -> windows-1252, gb2312 -> GBK
```

The order of precedence:

1. **A byte order mark** (UTF-8, UTF-16LE, UTF-16BE) decides, whatever the page declares.
2. **The transport `Content-Type`**, which you pass as `ParseOptions.ContentType` because the library doesn’t make HTTP requests. Pass the header your HTTP client received. Here a wrong UTF-8 header produced 4 decode errors on the windows-1252 bytes, as it would in a browser.
3. **The prescan** of the first 1,024 bytes for a `<meta charset>`, a `<meta http-equiv="Content-Type">`, or an XML declaration. Change the length with `ParseOptions.PrescanLimit`.
4. **The fallback**, `ParseOptions.DefaultEncoding`, which is windows-1252 unless you set it. The HTML standard recommends windows-1252 for most locales. That is why undeclared UTF-8 bytes show up as `café`. If you know your sources are UTF-8, set `DefaultEncoding = WhatwgEncoding.Utf8`.

`ParseOptions.OverrideEncoding` forces an encoding, though a byte order mark still wins. `doc.Source` tells you what was used: `Encoding`, `Confidence` (`Tentative` for a guess or a prescan result, `Certain` once confirmed), `BomLength` and `DecodeErrorCount`. `EncodingSniffer.Sniff` runs the same steps without parsing.

Labels follow the Encoding Standard’s table, so `latin1` and `iso-8859-1` mean windows-1252, and `gb2312` means GBK. When you edit a page, the output is written back in the encoding it was decoded with (see [Edit HTML without reformatting it](https://recursiveai.org/onehtml/cookbook/edit-html-csharp-lossless/)).

More on loading real pages: [Parse HTML in C#](https://recursiveai.org/onehtml/articles/parse-html-csharp/).

Encoding in a full scraper: [Web scraping in C#](https://recursiveai.org/onehtml/articles/web-scraping-csharp/).