# onemail docs, all pages (OneMail 0.3.0-preview.2)

Every page of https://recursiveai.org/onemail/docs/ and https://recursiveai.org/onemail/cookbook/ as markdown, getting started first. Index: https://recursiveai.org/onemail/llms.txt


<!-- https://recursiveai.org/onemail/docs/getting-started/ -->

# Getting started with OneMail

Install OneMail 0.3.0-preview.2 for .NET 10, parse an email, and open an mbox, an Outlook MSG file and a PST store with the same call. Pure C#, with no Outlook, COM or native library.

## Install

OneMail is a preview: the API can change before 1.0. The packages target `net10.0`, so you need the .NET 10 SDK, and `dotnet add package` needs `--prerelease` or the exact version.

```shell
dotnet new console -o OneMailStart
cd OneMailStart
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
```

Three packages are on NuGet at 0.3.0-preview.2:

- **`RecursiveAI.OneMail`**: the mail formats, the read view, conversion and display. It brings in `RecursiveAI.OneMail.Calendar`, `RecursiveAI.OneArchive` and `RecursiveAI.OneHtml`.
- **`RecursiveAI.OneMail.Calendar`**: ICS and vCard (and jCal, jCard, xCal, xCard), recurrence and time zones. It depends only on .NET.
- **`RecursiveAI.OneMail.Cli`**: the `onemail` command-line tool. Install it with `dotnet tool install -g RecursiveAI.OneMail.Cli --prerelease`.

## A first program

Download four public test files: an `.eml` and an mbox from Apache Tika, an Outlook message from Apache POI, and Tika’s PST store.

```shell
curl -LO https://raw.githubusercontent.com/apache/tika/4.2.0/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-mail-module/src/test/resources/test-documents/testRFC822_multipart_attachments.eml
curl -LO https://raw.githubusercontent.com/apache/tika/4.2.0/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-mail-module/src/test/resources/test-documents/complex.mbox
curl -LO https://raw.githubusercontent.com/apache/poi/REL_5_4_1/test-data/hsmf/attachment_test_msg.msg
curl -LO https://raw.githubusercontent.com/apache/tika/4.2.0/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/test/resources/test-documents/testPST.pst
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneMail;

// One message from bytes
IMailItem eml = Mail.Parse(File.ReadAllBytes("testRFC822_multipart_attachments.eml"));
Console.WriteLine($"{eml.Subject} from {eml.From[0].Address}");
Console.WriteLine($"{eml.Attachments.Count} attachments");

// Any file: the content decides the format, not the extension
string[] files = ["complex.mbox", "attachment_test_msg.msg", "testPST.pst"];
foreach (string path in files)
{
    using MailContainer mail = Mail.Open(path);
    Console.WriteLine($"{path}: {mail.Format}");
    foreach (IMailItem item in mail.Enumerate().Take(3))
        Console.WriteLine($"  {item.Sent?.Value:yyyy-MM-dd}  {item.Subject}");
}
```

```text
EML 1 from Bugs.Bunny@acme.com
2 attachments
complex.mbox: Mbox
  2009-06-01  Re: question about when shuffle/sort start working
  2009-05-31  Re: question about when shuffle/sort start working
  2009-06-01  Re: question about when shuffle/sort start working
attachment_test_msg.msg: Msg
  2009-04-22  test pièce jointe 1
testPST.pst: Pst
  2014-02-26  Re: Feature Generators
  2014-02-25  Re: init tokenizer fails: "Bad type in putfield/putstatic"
  2014-02-25  Feature Generators
```

The second mbox date reads May 31 because the line prints each date in the sender’s own time zone; `{item.Sent?.Value:u}` prints UTC.

## The two entry points

- **`Mail.Parse(bytes)`** parses one MIME message (`.eml`) and returns an `IMailItem`.
- **`Mail.Open(path)`** reads the first bytes to decide the format, then returns a `MailContainer`. A single message (`.eml`, `.msg`, `winmail.dat`) holds one item. A store (mbox, Maildir, PST, OST, `.dbx`) holds many, read as you enumerate.

Every item has the same read view, `IMailItem`: `Subject`, `From`, `ToRecipients`, `Cc`, `Sent`, `TextBody`, `HtmlBody`, `RtfBody`, `Attachments`, `Defects` and `Provenance`. Accessors never throw on malformed input: each repair is a typed defect with its byte offset.

Under the view, every layer is public: the MIME reader and writer, the mailbox stores, the PST layers, MSG, TNEF and the MAPI property model.

## Next

- Task by task: the [cookbook](https://recursiveai.org/onemail/cookbook/), with recipes for [MSG](https://recursiveai.org/onemail/cookbook/read-msg-file/), [PST](https://recursiveai.org/onemail/cookbook/read-pst-file/), [OLM](https://recursiveai.org/onemail/cookbook/read-olm-file/), [MSG to EML](https://recursiveai.org/onemail/cookbook/convert-msg-to-eml/), [PST to mbox](https://recursiveai.org/onemail/cookbook/convert-pst-to-mbox/), [attachments](https://recursiveai.org/onemail/cookbook/extract-attachments/) and [calendars](https://recursiveai.org/onemail/cookbook/read-ics-file/).
- What is checked, and every known gap: [Fidelity and known gaps](https://recursiveai.org/onemail/docs/compatibility/).
- Not in this preview: writing MSG or PST, SMTP, IMAP and POP3, and DKIM, S/MIME and OpenPGP.


<!-- https://recursiveai.org/onemail/docs/overview/ -->

# OneMail

An email library for .NET with no dependency beyond the .NET base class library and its sibling libraries (OneArchive, OneHtml, OneRender). It reads and writes the formats mail lives in, keeps what it reads lossless, and says what it repaired instead of throwing.

> **Early preview.** MIME, the mailbox stores, ICS/vCard trees with recurrence and time zones, and Outlook items on read (MSG/OFT, PST/OST, TNEF, compressed RTF, MAPI↔MIME conversion) work today; Outlook writing, protocols and cryptography are still to come. Nothing here is a stable API yet. What is checked, and the known gaps: [docs/compatibility.md](https://recursiveai.org/onemail/docs/compatibility/).
>
> **Display HTML is not yet independently reviewed.** The sanitiser behind `ToDisplayHtml` (`RecursiveAI.OneMail.Display`) has not had its independent security review. Do not rely on it to make untrusted mail safe to show until a release says the review has passed.

```csharp
using RecursiveAI.OneMail;

IMailItem message = Mail.Parse(File.ReadAllBytes("message.eml"));
Console.WriteLine($"{message.Subject} from {message.From[0].Address}");

using MailContainer mbox = Mail.Open("archive.mbox");
foreach (IMailItem item in mbox.Enumerate())
{
    Console.WriteLine(item.Subject);
}
```

## Goals

- **Reads:** MIME/EML, MHTML, TNEF, MSG/OFT, PST and OST, MBOX variants, Maildir, EMLX, OLM, DBX, ICS and vCard (and their JSON/XML forms).
- **Writes:** the exchange formats, MSG/OFT and Unicode PST.
- **Lossless and lenient:** untouched MIME, ICS and vCard write back byte for byte; Outlook data round-trips property for property; every repair of malformed input is a typed defect with its byte offset.
- **Every layer public:** random-access sources, the MIME reader and writer, the PST layers and the raw protocol layer, with a one-line happy path on top.
- **Protocols and cryptography** in separate packages: SMTP, IMAP, POP3, SASL; DKIM, ARC, S/MIME and OpenPGP.
- **NativeAOT, trimming and browser WebAssembly** (format core) are supported targets from the start.

## Packages

All packages target `net10.0` and share one version.

| Package | Contents | Depends on |
|---|---|---|
| `RecursiveAI.OneMail.Calendar` | ICS, vCard, JSON/XML forms, recurrence, time zones | .NET only |
| `RecursiveAI.OneMail` | the formats, the models and the one-line entry points | `.Calendar`, `RecursiveAI.OneArchive`, `RecursiveAI.OneHtml` |
| `RecursiveAI.OneMail.Cli` | the `onemail` command-line tool | `RecursiveAI.OneMail` |

These three are on nuget.org. `RecursiveAI.OneMail.Client` (SMTP, IMAP, POP3, SASL), `.Cryptography` (DKIM, ARC, S/MIME, OpenPGP) and `.Render` (a message to PDF or an image) come in later previews.

## Clean room

OneMail is written from public specifications (RFCs, Microsoft's Open Specifications, WHATWG) and from observing released programs as black boxes. The source of other mail libraries is not read.

## Not affiliated

This project is unaffiliated with other products named OneMail.

## Building

See [CONTRIBUTING.md](https://github.com/recursive-ai-oss/onemail-csharp/blob/bb4c9543917ab4ad466b5200d132e1899cb3ca14/CONTRIBUTING.md).

## Licence

MIT OR Apache-2.0, at your option: [LICENSE-MIT](https://github.com/recursive-ai-oss/onemail-csharp/tree/bb4c9543917ab4ad466b5200d132e1899cb3ca14/LICENSE-MIT), [LICENSE-APACHE](https://github.com/recursive-ai-oss/onemail-csharp/tree/bb4c9543917ab4ad466b5200d132e1899cb3ca14/LICENSE-APACHE). Copyright (c) Recursive AI, LLC.


<!-- https://recursiveai.org/onemail/docs/shared-view/ -->

# The shared read view and `Mail.*`

Every item OneMail reads (a MIME message, an MSG, a PST/OST message; calendar items and others as their readers arrive) implements one
read-only interface, `IMailItem`, in the root namespace `RecursiveAI.OneMail`. It is a **projection**: it copies nothing from
the model underneath, parses only what the caller asks for, and never throws on malformed content. Code that only needs to read
mail depends on the view; code that needs every byte and every defect goes to the model (for MIME, `MimeMailItem.Entity`).

## The view

| Member | Meaning |
|---|---|
| `Kind` | `Message`, `Appointment`, `Contact`, `Task`, `Note`, `Journal`, `DistributionList`, `Other`. MIME is always `Message`. |
| `Subject` | Decoded subject, or null. |
| `Sender`, `From`, `ToRecipients`, `Cc`, `Bcc`, `ReplyTo` | `MailAddress(Name, Address, X500Address)` (`X500Address` is only set for Outlook items whose address is an Exchange address); groups flattened (V7). Only the first field of each name is read (H2). |
| `Sent`, `TryGetDate(source, out value)` | A date and where it comes from: `DateHeader`, `ClientSubmit`, `Delivery`, `Creation`. MIME has `DateHeader`; a store adds `Delivery` (mbox envelope date, Maildir modification time, Apple Mail `date-received`). `ZoneKnown` is false when the source states no zone. MAPI items add the other two. |
| `MessageId`, `InReplyTo`, `References` | Without angle brackets. |
| `ConversationIndex`, `ConversationTopic` | MAPI properties; for MIME the `Thread-Index` (base64) and `Thread-Topic` headers when present, else empty/null. |
| `TextBody`, `HtmlBody`, `RtfBody` | `MailBody`: `ContentType`, `Charset`, `Open()` (decoded bytes), `ReadText()`. **Raw candidates, not the best body**: choosing between them is later work. For an Outlook item whose RTF was written as RTF, `HtmlBody` and `TextBody` are converted from it when the item has no such body of its own (`MailBody.Origin` is `RtfConverted`; [`rtf.md`](https://recursiveai.org/onemail/docs/rtf/)). MIME has no RTF. |
| `Attachments` | `IMailAttachment`: `FileName`, `ContentType`, `ContentId`, `IsInline`, `Size` (decoded; computed on first use by decoding once), `Open()`, `Embedded` (an `IMailItem` for `message/rfc822`). Built on first use. `ContentLocation` (`Content-Location`, or `PidTagAttachContentLocation`) is the other name HTML can use for a part (RFC 2557). `cid:` and location references are resolved against `ContentId`/`ContentLocation` by `ToDisplayHtml` ([`display-html.md`](https://recursiveai.org/onemail/docs/display-html/)). |
| `Defects` | One shape (`MailDefect`: family, code, offset, ambiguity flag) over each family's type: `mime` (the tree's defects, then those of the structured headers read), `mailbox` (the store reader's bits for this message). Evaluated on each enumeration. |
| `Provenance` | Format of the container, location, folder, byte offset and length of the item in its container file (-1 when not a range), index. |

Offsets in a `mime` defect are relative to the item's own first byte (what the MIME reader sees), not to the container file;
add `Provenance.Offset` for a file position (for an mboxrd store whose body lines were unquoted, positions after the first
unquoted line shift by the removed `>` bytes).

### Bodies and attachments for MIME

- **Text/HTML**: `multipart/alternative`: the last matching child (RFC 2046 5.1.4); `multipart/related` and `multipart/signed`: the
  first child only; any other multipart: the first matching child in order; a `message/*` part is not searched. A text part with
  `Content-Disposition: attachment` is never a body. Text decoding: the stated charset; if absent or unknown, UTF-8 when valid, else
  Windows-1252.
- **Attachments**: every leaf part and `message/*` part that is not the chosen text/HTML body, in document order, except a text part
  directly in a `multipart/alternative` that is not marked `attachment` (that is another rendering of the body). A message that is
  one leaf part is its own attachment. `IsInline` is true for a part that is not marked `attachment` and has a `Content-ID`.
  The resolutions are V1–V8 in [`ambiguities.md`](https://recursiveai.org/onemail/docs/ambiguities/).
- An embedded `message/rfc822` is an `IMailItem` with the same view, parsed from the same bytes.
- **A TNEF part** (`application/ms-tnef`, winmail.dat, found by content) that qualifies under MS-OXCMAIL 2.2.3.8 (`conversion.md`, TNEF parts) is
  shown the way Outlook shows it, not as a file (V9): the part is not listed as an attachment; the attachments the TNEF holds (embedded messages
  included, each an item with its own view) take its place in document order; `RtfBody` is the TNEF's compressed RTF decompressed; `TextBody`
  and `HtmlBody` are the TNEF's (an HTML property, or the HTML or text de-encapsulated from its RTF) when the MIME side has none of that kind.
  The MIME parts always win. A part that does not qualify (damaged stream, correlation key mismatch) stays one ordinary
  attachment. `MimeMailItem.TnefParts` lists the unpacked parts (the raw bytes are `part.GetDecodedContent()`; the tree is untouched) and
  `TnefMessages` their MAPI messages. A stream ending in CR LF is unpacked and the item reports the defect `AmbiguousTnefTrailingCrLf` (family `tnef`). Header fields are never taken from a TNEF: the view of a MIME item reads its headers only.

## MAPI items (MSG, PST, OST)

`MapiMailItem` projects a `MapiMessage` (`docs/mapi.md`) onto the same view. Nothing is looked up outside the item.

| Member | From |
|---|---|
| `Kind` | `PidTagMessageClass` by whole dot-separated segment, case-insensitive: `IPM.Appointment` Appointment, `IPM.Contact` Contact, `IPM.Task` Task, `IPM.StickyNote` Note, `IPM.Activity` Journal, `IPM.DistList` DistributionList; `IPM.Note`, `IPM.Post`, `IPM.Schedule`, `IPM.TaskRequest`, `IPM.Recall`, `IPM.Outlook.Recall`, `IPM.Sharing` and `REPORT` Message; no class Message; anything else Other. So `IPM.TaskRequest` is a message and not a task. |
| `Subject` | `PidTagSubject` with the leading U+0001 plus one-character pair removed (the marker of the normalised-subject split, MS-OXPROPS). |
| `From` | `PidTagSentRepresenting*`, else `PidTagSender*` (MS-OXCMAIL 2.1.3.1). |
| `Sender` | `PidTagSender*`, only when a representing sender exists and has a different address; else null. |
| address of a person | the SMTP address (`Address`). An Exchange (`EX`) address is resolved without a directory, in the order of [`conversion.md`](https://recursiveai.org/onemail/docs/conversion/) (OX1): its own `PidTagSmtpAddress` family property; another sender or recipient of the message with the same distinguished name; the transport `From`/`Sender` field's single mailbox for the sender groups; a unique display-name match in the transport headers. The distinguished name stays in `X500Address`. When nothing resolves, `Address` is that distinguished name (as before) and `X500Address` is the same. An address of another type (X.400, fax) is the stored address. The name is the display name (`PidTagRecipientDisplayName` for a recipient row without one). |
| `ToRecipients`, `Cc`, `Bcc` | recipient table rows by `PidTagRecipientType` 1, 2, 3, in table order. `ReplyTo` is read from `PidTagReplyRecipientEntries`: one-off EntryIDs (MS-OXCDATA 2.2.5.1) give name, type and address; other entry kinds need a directory and are left out. |
| `Sent`, `TryGetDate` | `PidTagClientSubmitTime` (`ClientSubmit`), `PidTagMessageDeliveryTime` (`Delivery`), `PidTagCreationTime` (`Creation`). FILETIME in UTC, so `ZoneKnown` is false. |
| `MessageId`, `InReplyTo`, `References` | `PidTagInternetMessageId` (0x1035), `PidTagInReplyToId` (0x1042), `PidTagInternetReferences` (0x1039), angle brackets removed; when the property is absent, the same-named field of `PidTagTransportMessageHeaders`. |
| `ConversationIndex`, `ConversationTopic` | the two properties. |
| `TextBody` | `PidTagBody`; when absent and the RTF body is a plain-text encapsulation (MS-OXRTFEX), the text from the RTF (UTF-8). `HtmlBody`: `PidTagHtml` (binary) or `PidTagBodyHtml` (string); when absent and the RTF body encapsulates HTML, the HTML from the RTF (UTF-8). `RtfBody`: `PidTagRtfCompressed` **decompressed** (MS-OXRTFCP), absent when it does not decompress. A body is absent when nothing gives it. |
| body text and charset | A string property is UTF-16LE (`Charset` utf-16le). An 8-bit text body uses the message's code page rule (`docs/msg.md`); binary HTML uses `PidTagInternetCodepage` (1200 means UTF-16LE), else the same rule; RTF has no charset. |
| `Attachments` | the attachment table in order. `FileName` long name, else short name, else display name. `ContentType` `PidTagAttachMimeTag` lower-cased, `message/rfc822` for an embedded message, else `application/octet-stream` unless the file name's extension is in the built-in table (see Attachment content types). `ContentId` without angle brackets. `IsInline`: has a content ID and `PidTagAttachContentDisposition` is not `attachment`. `Size`: length of `PidTagAttachDataBinary`, else `PidTagAttachSize`. `Embedded`: the embedded message as a `MapiMailItem`. |
| `Defects` | the model's `MapiDefect`s (message, recipients, attachments, embedded messages), family `mapi`; ambiguity flag from the kind. |
| `Provenance` | PST/OST: format, file, folder path, index in enumeration order; offset and length are -1 (a message is not a byte range). MSG: format, file, offset 0, the file's length. |

PST/OST enumeration through `Mail.Open` yields ordinary messages of non-search folders in folder order (`docs/pst-messaging.md`).

## Attachment content types

`IMailAttachment.ContentType` is the stored type whenever the item stored a specific one. Only a missing type or the generic `application/octet-stream` (MIME part, `PidTagAttachMimeTag`, OLM `OPFAttachmentContentType`; TNEF attachments are read as MAPI attachments) is replaced, and only when the file name has an extension in a small built-in table: office and PDF, text, calendar and contact, message, image, audio, video and archive types. The last extension counts, case-insensitively; a directory part of the name is ignored. There is no operating-system lookup, so the answer is the same on every platform and in the browser. When the type was guessed this way `ContentTypeInferred` is true; otherwise (including when nothing in the table matches and the type stays `application/octet-stream`) it is false. An embedded message is always `message/rfc822`. The raw entity, property bag or element is untouched: only the view infers, so round-trips and conversions that read the model keep the stored bytes.

## Opening things: `Mail.*`

| Call | Result |
|---|---|
| `Mail.Parse(bytes \| stream, MimeReaderOptions?)` | One `IMailItem`. |
| `Mail.Open(path \| stream \| source, MimeReaderOptions?, MboxReaderOptions?)` | A `MailContainer`: the sniffed `Format`, `IsStore`, `Enumerate()` and `await foreach`. A single message (EML, MSG) is a container of one; a PST or OST is a store. A PST/OST or MSG that is such by content but cannot be read at all throws `InvalidDataException`. |

Each `Mail.*` method is one call into the layer that does the work; the options are those layers' own option types.

`MailContainer` lifetime: the container owns the file it opened from a path (a stream or source passed in stays the caller's). Items
read their bytes when first used, so they are valid until the container is disposed, after which reading an unread item throws
`ObjectDisposedException`. Enumeration holds one message's state at a time. A source without synchronous reads is handled by the
async enumeration (mbox messages are loaded before the item is yielded).

### Sniffing

`MailFormatSniffer.Sniff(source)` and `.SniffDirectory(path)` decide by content; the file name is never read.

| Evidence | Format |
|---|---|
| `D0 CF 11 E0 A1 B1 1A E1`, and the UTF-16 name `__properties_version1.0` in the first 16 MiB | `Msg` (MS-OXMSG); otherwise `CompoundFile` |
| `!BDN`, client magic `SO` at offset 8 / other | `Ost` / `Pst` |
| `78 9F 3E 22` | `Tnef` |
| first line is an mbox envelope line (`From ` + sender + date) | `Mbox`; `EMLX`: byte count, message, then end of file or a property list |
| `BEGIN:VCALENDAR` / `BEGIN:VCARD` (any case, after an optional BOM and blanks) | `Ics` / `VCard` |
| first line is a header field: printable ASCII name, optional blanks, `:` | `Eml` |
| directory: `cur` and `new`; dot-prefixed Maildir subfolders; `Messages` directories of `.emlx` | `MaildirFolder`, `MaildirRoot`, `AppleMailBundle` |

A message with no header block (starts with a blank line or with text that is not a header field) cannot be told from any text
file and is `Unknown`; `Mail.Parse` still reads it. A recognised format whose reader is not in the library yet (`Ics`, `VCard`,
`CompoundFile`) makes `Open` throw `UnsupportedMailFormatException` with its `Format`; `Unknown`
throws the same exception with `Format == Unknown`. DBX files and DBX store folders (`Dbx`, `DbxStore`) and Outlook for Mac archives (`Olm`) are recognised and opened.

## The CLI: `onemail inspect`

`onemail inspect [--json] [--sizes] [--defects] <file-or-directory>...` prints the format and every item. A directory that is not a
store is walked and each file inspected on its own (a tree of loose messages); a file that is not readable is reported, not fatal.
Text: one line per item (date, first sender, subject, attachment names, defect count). `--json`: one object per line, a `container`
record then an `item` record per message with the whole view and provenance. `--sizes` decodes attachments to measure them.
It streams: one item is held at a time.

## How it is checked

- `tools/oracles/viewcheck.sh`: runs `onemail inspect --json`, cuts each item's bytes out of its container, runs the OneMail oracle
  driver (built on the tree API, not the view) and compares subject, addresses, date, Message-ID/In-Reply-To/References and
  attachment lists per item; then compares attachment names and counts with Python `email`, grouping differences by signature, each
  tied to a V entry or an N entry.
- Allocation: enumerating an mbox of small messages creates one ~150-byte item per message and reads nothing; reading subject,
  from and date of each costs about 5.4 KB per message, almost all of it the MIME tree (~2.7 KB) and the address and date parsers,
  not the view.


<!-- https://recursiveai.org/onemail/docs/mime-reading/ -->

# Reading MIME

This is the behaviour of `MimeReader` (`RecursiveAI.OneMail.Mime.Reading`) and the entity tree built on it
(`RecursiveAI.OneMail.Mime`). It is written to be implemented the same way in another language.

## Events cover the input exactly

The reader turns a message into a flat sequence of events. Every input byte belongs to exactly one event, in order, so
concatenating the bytes of all events reproduces the input — including malformed input. The tree stores the same ranges,
so an unchanged tree writes back byte for byte.

| Event | Bytes it covers |
|---|---|
| `EntityStart`, `EntityEnd`, `Defect` | none (a position) |
| `EnvelopeLine` | an mbox `From ` line at the very start of the message, with its line end (not a header field) |
| `Header` | one header field: name, colon, value, folded lines, and the field's own line end |
| `HeadersEnd` | the empty separator line (`CRLF` or `LF`), or nothing if the input has none |
| `Preamble` / `Body` / `Epilogue` | a chunk of text; chunk sizes depend on buffering and carry no meaning |
| `Delimiter` | the line end that precedes the delimiter line, the delimiter line `--boundary[--]`, its transport padding, and its own line end |

A multipart body is: preamble, then for each part a `Delimiter`, the part, and finally the closing `Delimiter` and the epilogue.
The line end before a delimiter belongs to the delimiter (RFC 2046 §5.1.1), so a part's body does not include it.

## Structure

- A header block ends at the first empty line. A line with no colon also ends it: the line starts the body and
  `MissingHeaderBodySeparator` is reported. Any line with a colon is a header field, even when the name has spaces, control
  bytes or bytes above 0x7E (`InvalidHeaderName`; MimeKit and Go do the same). A folded line (starting with space or tab)
  continues the field before it (RFC 5322 §2.2.3); one with no field before it ends the header block. A first line that starts
  with `From ` is an mbox envelope line: the reader reports it as its own event and the header block starts after it.
- Two delimiter lines in a row (`--b` then `--b`) leave an empty part with no headers between them. The reader reports it; the
  entity tree folds it into the delimiters (Python and MimeKit produce no part there), so writing is unchanged.
- An entity is a **multipart** when `Content-Type` is `multipart/*` with a usable `boundary`, and the transfer encoding is
  7bit, 8bit or binary. It is a **message** when the type is `message/rfc822` or `message/global` (or it has no `Content-Type`
  inside `multipart/digest`, RFC 2046 §5.1.5), under the same encoding rule. Everything else is a **leaf**.
- A delimiter line is `--` + boundary, optionally followed by `--`, then optional spaces and tabs, then the line end or the end of
  the input. Active boundaries are tried innermost first. A delimiter of an enclosing multipart ends everything nested inside it
  (RFC 2046 §5.1.2); each multipart cut off this way reports `MissingFinalBoundary`.
- The first `Content-Type` and the first `Content-Transfer-Encoding` of an entity decide; later ones are reported and ignored.

## Defects

Defects are events, reported after the event that raised them, with the byte offset. They never stop reading.

| Kind | Meaning | Reading used |
|---|---|---|
| `MissingHeaderBodySeparator` | header block ended by end of input or a non-header line | the offending line starts the body |
| `HeaderContinuationWithoutHeader` | folded line with no field to continue | ends the header block; the line starts the body |
| `BareCarriageReturnInHeader` | CR not followed by LF inside a field | left in place |
| `NulInHeader` | NUL inside a field | left in place |
| `Utf8Header` | bytes above 0x7F that are valid UTF-8 (RFC 6532) | informational |
| `AmbiguousEightBitHeader` | bytes above 0x7F that are not UTF-8 | kept raw; the text layer picks a charset |
| `InvalidHeaderName` | a field name with a space, a control byte or a byte above 0x7E | still a header; `Name` is read as Latin-1, a leading UTF-8 BOM is ignored |
| `ControlCharacterInHeader` | a control character other than NUL, tab, CR, LF inside a field | left in place |
| `MixedLineEndings` | LF and CRLF both used in the headers and delimiters | each line ends either way; reported once per message |
| `MissingStartBoundary` | a multipart whose body has no delimiter line | no parts; the whole body is its preamble |
| `EpilogueHasDelimiter` | after the closing delimiter, a line that looks like a delimiter | stays epilogue text; reported once per entity |
| `BoundaryHasTrailingSpace` | a `boundary` parameter ending in blanks | matched trimmed, which also matches the literal one |
| `InvalidParameter` | a `Content-Type` or `Content-Disposition` parameter without `=` | dropped |
| `WhitespaceBeforeHeaderColon` | `Name : value` (obsolete syntax, RFC 5322 §4.2) | name is trimmed |
| `AmbiguousDuplicateContentType` | more than one `Content-Type` | first wins |
| `AmbiguousDuplicateTransferEncoding` | more than one `Content-Transfer-Encoding` | first wins |
| `InvalidContentType` | not `type/subtype` | `text/plain` |
| `UnknownTransferEncoding` | not 7bit/8bit/binary/base64/quoted-printable | identity |
| `MultipartWithoutBoundary`, `EmptyBoundary`, `BoundaryTooLong` | unusable boundary | the entity is a leaf |
| `AmbiguousDuplicateBoundary` | a multipart reuses an enclosing multipart's boundary | innermost wins |
| `AmbiguousBoundaryLikeLine` | a line starts with `--boundary` but continues with other text | body text, not a delimiter |
| `AmbiguousEncodedComposite` | a multipart or message type declared base64 or quoted-printable | a multipart is still split into parts from the raw body; a message stays a leaf |
| `MissingFinalBoundary` | multipart without closing delimiter | the multipart ends at the enclosing delimiter or the end of input |

Header decoding reports the text layer's issues as defects too (`InvalidCharsetBytes`, `AmbiguousUnknownCharset`,
`MalformedEncodedWord`, `InvalidEncodedWordBase64`, `InvalidEncodedWordQuotedPrintable`, `EncodedWordTooLong`,
`AmbiguousEncodedWordInQuotedString`, `AmbiguousSplitMultibyte`, `InvalidParameterEscape`, `MalformedExtendedParameter`,
`AmbiguousParameterSegments`, `DuplicateParameterSegment`, `AmbiguousParameterForms`, `DuplicateParameter`,
`TextLimitExceeded`), and so do the structured-header parsers (addresses: `AmbiguousUnquotedCommaInPhrase`,
`MissingAngleBrackets`, `ObsoleteRoute`, `EmptyAddress`, `MissingDomain`, `UnclosedAngle`, `UnterminatedQuote`,
`AmbiguousAdjacentAngleAddrs`, `UnparsedAddressText`, `ObsoleteLocalPart`, `InvalidAddressSyntax`; dates: `WeekdayMismatch`,
`AmbiguousTwoDigitYear`, `ThreeDigitYear`, `UnknownTimeZone`, `InvalidTimeZone`, `LeapSecond`, `FractionalSeconds`,
`InvalidDate`; identifiers: `InvalidMessageId`). Their offsets are into the parsed input. The resolutions are in
[ambiguities.md](https://recursiveai.org/onemail/docs/ambiguities/).

`Ambiguous*` marks places where mail clients disagree. The reading above is the library's choice; it has not yet been confirmed
against how particular clients behave, and may change once it is.

Reports, MHTML and authentication results add the defects below (details in [reports-and-mhtml.md](https://recursiveai.org/onemail/docs/reports-and-mhtml/)):
`ReportMalformedLine`, `ReportMissingField`, `ReportInvalidFieldValue`, `AmbiguousReportDuplicateField`, `ReportMissingGroup`,
`AmbiguousReportExtraBlocks`, `AmbiguousReportBlankLines`, `ReportStructureInvalid`, `ReportLimitExceeded`,
`MhtmlStartNotFound`, `InvalidContentLocation`, `AmbiguousMhtmlDuplicateLocation`, `AmbiguousMhtmlDuplicateContentId`,
`AuthenticationResultsMissingId`, `AuthenticationResultsInvalidVersion`, `InvalidAuthenticationResults`,
`AmbiguousAuthenticationResultsNone`, `AmbiguousAuthenticationResultsProperty`, `InvalidReceivedSpf`.

## Resource limits (on by default)

Reaching a limit reports a defect and keeps the rest of the input as body text, so nothing is lost and nothing is allocated for it.

| Limit | Default | Defect | Effect |
|---|---|---|---|
| `MaxDepth` | 64 | `DepthLimitExceeded` | a deeper multipart/message is a leaf |
| `MaxEntityCount` | 10,000 | `EntityLimitExceeded` | no more delimiters are recognised; the rest of the input is body of the current entity |
| `MaxHeaderBlockBytes` | 1 MiB | `HeaderLimitExceeded` | the header block ends; the rest is body |
| `MaxHeaderFieldBytes` | 256 KiB | `HeaderFieldLimitExceeded` | the header block ends; the rest is body |
| `MaxHeadersPerEntity` | 10,000 | `HeaderLimitExceeded` | the header block ends; the rest is body |
| `MaxBoundaryLength` | 255 | `BoundaryTooLong` | the entity is a leaf |
| `MaxTotalBytes` | 2 GiB − 1 | `TotalSizeLimitExceeded` | reading stops; `IsTruncated` is set |

The read buffer is pooled and grows only to hold one header field, never past twice `MaxHeaderBlockBytes`.

## Transfer encodings

`Base64Decoder` and `QuotedPrintableDecoder` are lenient and report what they repaired as `TransferIssues`.

- base64: whitespace is ignored; characters outside the alphabet are skipped; padding ends a group and data after it starts a
  new one (concatenated base64 blocks decode); a final partial group yields its complete bytes.
- quoted-printable: `=XX` in either hex case; soft breaks (`=` + line end, also with blanks in between) vanish; blanks before a
  hard line end are kept (they are data; Python and MimeKit keep them); a malformed `=` stays literal, together with the blanks
  after it. Hard line ends are copied as they are.
- uuencode (`x-uuencode`, `uuencode`, `x-uue`, `uue`): text before the `begin` line is ignored and decoding stops at `end`.

## Writing

`MimeWriter` copies reader events byte for byte, and generates headers (folded at word boundaries at 78 columns) and delimiters. Non-ASCII text in free-text fields (`Subject`,
`Comments`, `Keywords`, `Content-Description` and unknown fields) is written as RFC 2047 encoded words; fields with their own
syntax (addresses, identifiers, dates, `Content-*`) keep it as UTF-8 (RFC 6532), and parameters are built with
`MimeHeaderValue.Parameter`, which writes RFC 2231 extended values. A header name or value that could start a new header (a colon in the name, a line break in the value) is
refused. A modified tree re-serialises only the entities on the path to a change; untouched entities, bodies, preambles and
epilogues are copied from the source as ranges.


<!-- https://recursiveai.org/onemail/docs/text-encodings/ -->

# Text encodings: charsets, encoded words, parameter values

Normative behaviour of the text layer (`RecursiveAI.OneMail.Text`). It sits under the MIME code and knows nothing about
messages: it turns bytes into text and text into bytes, and says what it had to repair.

## Contract

- Decoders never throw on malformed input. They return the best text and report each repair or guess as a `TextIssue`
  (kind, byte offset, byte length). Offsets are into the span given to the decoder; for parameters the offset is the one the
  caller passed with the parameter.
- Kinds named `Ambiguous*` are the documented resolutions in the table below. Others are plain damage.
- Decoding is lossless where the input allows it: CR and LF survive, a byte order mark is kept as U+FEFF, U+FFFD is written only for bytes the charset cannot map.
- Passing no issue list costs nothing: the paths allocate the result string, and for encoded words one pooled byte buffer.

## Charsets

`MailEncoding.ForLabel` ignores ASCII case and surrounding white space and looks, in order, at

1. the mail-only labels: `utf-7` and its aliases, `utf-32*`, `iso-2022-kr`, `hz-gb-2312`, `iso-2022-jp-1/-2/-3`, `cp50220/50221/50222`
   (also `windows-` and `x-windows-` forms), the DOS code pages (`cp437`, `ibm850` ...), `johab`;
2. a code page number in a Windows-style spelling (`cp1252`, `windows-1251`, `codepage-1250`) that the table knows;
3. the WHATWG Encoding Standard label table, decoded by OneHtml's tables.

`replacement` and the labels the standard maps to it (ISO-2022-CN, ISO-2022-CN-EXT) are unknown (`null`); `unknown-8bit` and anything else not in
the lists are unknown. Callers apply the *unknown charset* rule below.

Because the Encoding Standard is the table, `us-ascii`, `iso-8859-1` and `iso-8859-9` decode as windows-1252 / windows-1254 and the bytes 0x80..0x9F
map to the printable characters, as every mail client in practice does. A strict Latin-1 reading would turn those bytes into C1 controls.

`MailEncoding.ForCodePage` is the reverse map for MAPI's `PidTagInternetCodepage`/`PidTagMessageCodepage`: 874, 932, 936, 949, 950, 1200, 1201, 1250-1258, 12000,
12001, 20127, 20866, 21866, 28591-28606, 38598, 437/737/775/850/852/855/857/858/860-865/869, 10000, 10007, 866, 50220-50222, 50225, 51932, 51949, 52936, 54936, 65000, 65001, 1361.
`Name` is the spelling to write in a header (`windows-1252`, `shift_jis`, `gb2312` for 936, `ks_c_5601-1987` for 949, `iso-8859-1` for 28591).
The name, not the decoder, differs for aliases: 20127 and 28591 decode like windows-1252.

Encoding (`GetBytes`) uses the runtime's code-page tables (UTF-8 and UTF-7 are ours). A character the charset lacks becomes `?` and is reported (`Unencodable`).
The stateful ISO-2022 charsets are encoded by the runtime's 50220/50221/50222/50225/52936 encoders, one word at a time.

### Decoding details

- Table-driven charsets are decoded by OneHtml. Two properties of its HTML-oriented API are worked around here so mail text is not changed: a leading
  byte order mark would override the charset (each piece is decoded with a leading space that is removed afterwards) and CR/CRLF would become LF (the input is cut at every CR byte,
  which is a character on its own in all these charsets, and the CRs are put back). Pure ASCII input skips OneHtml.
- UTF-8: the Encoding Standard's maximal-subpart replacement. UTF-16/32: the runtime's, with replacement.
- Unknown charset (`AmbiguousUnknownCharset`): the bytes are decoded as UTF-8 when they are valid UTF-8, otherwise as windows-1252.

### UTF-7 (RFC 2152) and IMAP modified UTF-7 (RFC 3501 §5.1.3)

- UTF-7 decoding accepts any ASCII byte in direct mode; `+-` is `+`; a base64 run ends at `-` (absorbed) or at a non-base64 byte (kept). A byte of 0x80 or above, a run with
  unpaired surrogates, non-zero leftover bits, a leftover of six or more bits, or an empty run (`+` followed by a non-base64, non-`-` byte) is `InvalidUtf7`; the damaged part becomes U+FFFD where it carries text.
- UTF-7 encoding writes letters, digits, `'(),-./:?`, space, tab, CR and LF directly and everything else in base64 (also the "optional direct" set); `-` closes a run only
  when the next character could continue it.
- Modified UTF-7 decoding: `&-` is `&`; a run is base64 with `,` for `/` and ends at `-`. A run without its `-` still decodes (`InvalidImapUtf7`). Bytes outside 0x20..0x7E in direct mode become U+FFFD.
  Encoding writes printable ASCII directly (`&` as `&-`) and other runs as `&base64-`, always closed.

### ISO-2022 family (ISO-2022-JP, -1, -2, -3, CP50221/50222, ISO-2022-KR, HZ)

One tolerant decoder serves the whole ISO-2022-JP family. A text that uses a character set outside the label's repertoire is still decoded and reported as `AmbiguousIso2022Repertoire`.

| Label | Repertoire |
|---|---|
| `iso-2022-jp` (cp50220) | ASCII, JIS-Roman (`ESC ( J`), JIS C 6226-1978 (`ESC $ @`), JIS X 0208-1983 (`ESC $ B`) |
| `iso-2022-jp-1` | + JIS X 0212 (`ESC $ ( D`) |
| `iso-2022-jp-2` | + GB 2312 (`ESC $ A`), KS C 5601 (`ESC $ ( C`), ISO-8859-1 and ISO-8859-7 high parts as G2 (`ESC . A`, `ESC . F`, `ESC N`) |
| `iso-2022-jp-3` | base set + JIS X 0213 plane 1 (`ESC $ ( O` / `Q`, read as JIS X 0208) and plane 2 (`ESC $ ( P`: each pair becomes U+FFFD with `UnsupportedCharacterSet`) |
| `cp50221`, `cp50222` | base set + half-width katakana (50221: `ESC ( I`; 50222: SO/SI) |
| `iso-2022-kr` | ASCII + KS C 5601 through SO/SI; the `ESC $ ) C` header is consumed |
| `hz-gb-2312` | `~{ ... ~}`, `~~`, `~` LF |

The two-byte sets are rewritten to their EUC form and decoded by OneHtml's EUC-JP / GBK / EUC-KR tables. A line end (CR or LF) returns to ASCII in every variant, so a missing reset cannot
swallow the following line. Unknown escapes, SO/SI where the variant has none, bytes ≥ 0x80 and unpaired bytes become U+FFFD (`InvalidIso2022`).
JIS X 0213 beyond plane 1 and ISO-2022-CN are not decoded.

## Encoded words (RFC 2047)

`Rfc2047.DecodeText` decodes header text, `Rfc2047.Encode` produces it.

Decode rules:

- A fold (CRLF, or a bare CR/LF, followed by space or tab) is removed first; the white space after it stays (RFC 5322 §2.2.3).
- White space between two adjacent encoded words is dropped; between a word and plain text it stays (RFC 2047 §6.2).
- `charset*language` is accepted; the language is ignored (RFC 2231 §5).
- B: characters outside the alphabet and data after padding are skipped; wrong or missing padding and a one-character tail are `InvalidBase64`; the bytes that can be decoded are kept.
- Q: `_` is a space; `=XX` is a byte; `=` without two hex digits stays a literal `=` (`InvalidQuotedPrintable`).
- A word is `=?charset?B|Q?text?=` with the text ending at the first `?=`. White space (space, tab) inside the text is accepted and reported (`MalformedEncodedWord`); CR/LF inside, an encoding other than B/Q (case-insensitive)
  or a missing `?=` make it not a word: the characters stay literal (`MalformedEncodedWord`) and scanning resumes two bytes later. An empty or space-containing charset, or a missing `?` structure, is not
  recognised as a word at all: literal text, no issue.
- Longer than 75 characters: decoded, `EncodedWordTooLong`.
- Plain 8-bit text: UTF-8 if valid (RFC 6532), else the fallback charset (default windows-1252), `AmbiguousRaw8Bit`.

Ambiguities and their resolutions:

| Case | Resolution | Issue |
|---|---|---|
| Charset not known | UTF-8 if valid else windows-1252 | `AmbiguousUnknownCharset` |
| Adjacent words with the same charset (case-insensitive) | the bytes of the run are decoded together, so a multibyte character or an ISO-2022 shift split between words survives; words with different charsets are decoded separately | `AmbiguousSplitMultibyte` (UTF-8 only: reported when a word alone is not valid UTF-8) |
| Encoded word in a quoted string or parameter value | decoded | `AmbiguousEncodedWordInQuotedString` |
| White space inside an encoded word | decoded | `MalformedEncodedWord` |
| Invalid base64 / Q escape | repaired as above | `InvalidBase64` / `InvalidQuotedPrintable` |
| `=?` that does not start a word, tried again inside | the scan resumes at the next `=?`, so `=?=?utf-8?Q?abc?=?=` decodes the inner word | none |

Encode rules: text without characters outside printable ASCII (and without `=?`) is returned unchanged. Otherwise each maximal group of tokens that need encoding, together with the
white space between them (a decoder drops it), becomes encoded words of at most 75 characters, never cutting a character (or a surrogate pair), B or Q whichever is shorter. Q uses only
`A-Za-z0-9!*+-/` plus `_` for space, so the result is valid in phrases, comments and unstructured text. Words are separated by one space; folding there is legal.

## Parameters (RFC 2231)

`ParameterCollector` takes the parameters of one header in source order (name as written, value unquoted) and `Resolve` returns one `DecodedParameter` per name (case-insensitive, first spelling kept).

- `name*` is an extended value `charset'language'percent-encoded`; `name*N` and `name*N*` are continuation segments; only segment 0 carries the charset; `*N*` segments are percent-decoded, `*N` segments are literal.
- Segments are ordered by number (stable) and joined **as bytes**, then decoded once with the charset, so a character split between segments survives.
- No charset (value without two `'`, or an empty charset): the percent-decoded bytes are read as UTF-8 if valid else the fallback (`MalformedExtendedValue` when the quotes are missing; `AmbiguousRaw8Bit` for the fallback).
- Percent escape that is not `%XX`: kept literally (`InvalidPercentEscape`).
- Plain `name=value`: UTF-8 if valid else fallback; a value containing an encoded word is decoded as one (`AmbiguousEncodedWordInQuotedString`).

| Case | Resolution | Issue |
|---|---|---|
| Segment numbers have a gap or do not start at 0 | every segment present is joined in numeric order | `AmbiguousParameterSegments` |
| Same segment number twice | both are kept, in source order | `DuplicateParameterSegment` |
| `name=` and `name*=`/`name*0=` both present | the RFC 2231 form wins, wherever it appears | `AmbiguousParameterForms` |
| Same plain name twice | the first | `DuplicateParameter` |
| More than 1024 parameters, segment number ≥ 256 | the excess is ignored | `LimitExceeded` |

`Rfc2231.Encode` returns the parts to write: a token value as `name=value`; other printable ASCII as a quoted string; anything else as `name*=charset''%XX...`, or
`name*0*`, `name*1*` ... for values longer than the part length (default 50), each part holding whole characters; the charset appears in the first part only.

## How this is checked

`tools/text-diff` generates ≥ 10k encoded-word and parameter cases (valid, adjacent, split, malformed), plus charset cases (UTF-7, IMAP UTF-7, ISO-2022 family, HZ) and values harvested from the CMU Enron maildir,
runs them through this library and through Python `email` (`decode_header`, `Message.get_param` with `collapse_rfc2231_value`), Go `mime` (`WordDecoder`, `ParseMediaType`, with `golang.org/x/text` charsets) and MimeKit's
public decoding API, and compares decoded values. Each difference must match a rule in `explain.py` that names the resolution above; anything else is a failure. The Python and Go oracles cannot
decode IMAP modified UTF-7, so those cases check against an independent reference encoder in the generator.


<!-- https://recursiveai.org/onemail/docs/reports-and-mhtml/ -->

# Reports, MHTML and authentication results

Typed views over the MIME entity tree (`RecursiveAI.OneMail.Mime.Reports`, `.Mhtml`, `.Authentication`). The tree stays the
lossless store: a view reads it, never changes it, so an untouched message still writes back byte for byte. Every repair is a
typed defect with the byte offset. The ambiguity entries (RP, AR, RS, MH) are in [ambiguities.md](https://recursiveai.org/onemail/docs/ambiguities/).

## multipart/report (RFC 6522)

`MimeReport.Find(tree)` returns the first `multipart/report` (searching nested message/rfc822 as well); `TryCreate(entity)`
tests one entity. The view exposes the human-readable part, the machine-readable part, and the original (a message, or
headers only for `text/rfc822-headers` and `message/rfc822-headers`). `GetDeliveryStatus`, `GetDispositionNotification` and
`GetFeedbackReport` parse the machine part and add their defects to a caller's list.

| Report | Media type | Spec | Class |
|---|---|---|---|
| Delivery status | `message/delivery-status`, `message/global-delivery-status` | RFC 3464, 6533 | `DeliveryStatusReport` (per-message block, per-recipient `RecipientStatus`) |
| Disposition notification | `message/disposition-notification` | RFC 8098 (3798) | `DispositionNotification`, `MdnDisposition` |
| Feedback | `message/feedback-report` | RFC 5965, 6591, 6650 | `FeedbackReport` |

- **Field syntax** (`ReportFieldParser`): RFC 5322 header syntax, a block ends at an empty line. A line of only blanks is a
  fold, not an empty line. Values are not RFC 2047-decoded (the RFCs do not use encoded words here); bytes are read as UTF-8
  if valid, else windows-1252. A lone CR stays in the value (`BareCarriageReturnInHeader`).
- **Typed values**: `DsnTypedValue` (`type; value`), `EnhancedStatusCode` (RFC 3463; trailing comments tolerated),
  `DsnAction`, dates as `MimeDate`. A value that does not fit stays readable as text and raises `ReportInvalidFieldValue`.
- **Kind**: the machine part's own media type decides; a `report-type` that disagrees or is missing raises
  `ReportStructureInvalid`.
- **Limits**: a report body over 16 MiB or more than 100,000 fields raises `ReportLimitExceeded`; the rest is not parsed.
- **Writing**: `MimeReport.CreateDeliveryStatus / CreateDispositionNotification / CreateFeedbackReport` build the
  `multipart/report` (boundary overridable for reproducible output); `ToBytes` writes fields folded at 78 columns, UTF-8 for
  non-ASCII (the caller picks the `global` media type with `RequiresGlobal`).

## MHTML (RFC 2557)

`MhtmlDocument.Find / TryCreate / Open / Load` view a `multipart/related` (or a lone HTML part) as a root resource plus
resources. `Resolve(reference, htmlBase)` returns the resource a reference in the HTML names; `ResolveToAbsolute` gives the
URI. `MhtmlBuilder` writes a `.mht`. `MhtmlUri` is RFC 3986 §5.2 resolution, with `thismessage:/` as the base of last resort.

Base order (RFC 2557 §5): the HTML's own base argument, the legacy `Content-Base`, an absolute `Content-Location` of the
part and then of its enclosing entities, else `thismessage:/`. Lookup is exact first, then after `Normalize` (scheme and host
case, percent-hex case, space as `%20`, default ports, dot segments). `cid:` matches `Content-ID` only (RFC 2557 §8.3: a
`Content-Location` holding a `cid:` URL never satisfies it), with angle brackets stripped and percent-decoding; `mid:` needs
both a content id and the message id. A lookup falls through to enclosing `multipart/related` entities. Duplicates: the first
wins, and the ambiguity is reported.

## Authentication-Results and Received-SPF

`MimeEntity.GetAuthenticationResults()` and `GetReceivedSpf()` parse every such field (RFC 8601; RFC 7208 §9.1) into
`AuthenticationResults` (authserv-id, version, `AuthenticationMethodResult` with method, version, result, comment, reason and
`AuthenticationProperty` ptype.name=value) and `ReceivedSpf`. This is parsing only: nothing is evaluated or trusted, and
callers must still check the authserv-id against their own boundary. `MimeEntity.GetAuthenticationResults` takes the header
text as the tree stores it (comments and folds kept in the raw value).

## Method

`tools/oracles/reports` runs ours, MimeKit 4.18.2 (public API) and Python 3.13 `email` over `testdata/reports` (119 messages,
generated by `tools/reports/make_fixtures.py`, with RFC 3464, 8601, 7208 and 2557 examples) and over the corpus messages that
carry the headers (`corpus.sh`). Python has no Authentication-Results parser; Received-SPF has no oracle (RFC examples only).
MHTML reference resolution is also run against Python `urllib.parse.urljoin` on 218 base/reference pairs. A difference must
match a line of `explained.tsv` (naming an ambiguity entry or a documented oracle limit) or the run fails.


<!-- https://recursiveai.org/onemail/docs/mailboxes/ -->

# Mailbox stores

How OneMail enumerates and writes mbox files, Maildir folders and Apple Mail (EMLX) stores, and reads Outlook Express (DBX) stores. The code is in
`RecursiveAI.OneMail.Stores` (shared types, file-system abstraction, content sniffing) and its `.Mbox`, `.Maildir`, `.Emlx` and `.Dbx`
namespaces. Text here is written to be implemented the same way in another language. The cases where programs disagree are
listed with their resolutions in [`ambiguities.md`](https://recursiveai.org/onemail/docs/ambiguities/) (S1–S18).

## The store contract

A store reader does not parse messages. It yields **entries**: where a message is (a byte range of a source, or a file of a
directory store), what the store knows about it (flags, keywords, dates, UID) and what was irregular (**defects**). The MIME
layer parses the bytes on demand.

- An entry is plain data and holds no reference to its source. Opening the message is a call on the reader, taking the entry.
- File stores (mbox, EMLX) read from an `IRandomAccessSource`. Directory stores (Maildir, Apple Mail bundles) read from an
  `IStoreFileSystem`: list, open for reading, create new, rename, delete. A directory on disk, memory (tests, browser WebAssembly)
  and any storage mapped onto those calls all work. Paths use `/` and are relative to the file system's root.
- Enumeration: `Read()`/`ReadAsync(ct)` on the mbox reader, `Enumerate`/`EnumerateAsync(ct)` everywhere else. The synchronous and
  asynchronous paths are the same code.
- Nothing throws on malformed content. Defects are `StoreDefects` bits on the entry and, with the reader's
  `DefectHandler`, `(kind, byte offset)` pairs. `Ambiguous*` marks a choice between valid readings.
- Sniffing is by content: `StoreSniffer.Sniff(source)` (mbox, EMLX), `StoreSniffer.SniffDirectory(fs, path)` (Maildir folder,
  Maildir++ root, Apple Mail bundle), `MboxSniffer.Detect(source)` (variant). Extensions are never consulted.

## mbox

### Records tile the file

A file is `[preamble][record]…`. A record is `[envelope line][message][separator]`.

| Part | Bytes |
|---|---|
| Preamble | everything before the first envelope line (`PreambleLength`; empty for a normal file) |
| Envelope line | from `From ` through its line ending (LF or CRLF); the last line of a file may lack one |
| Message | the stored bytes, headers first, **still quoted** |
| Separator | one empty line (LF or CRLF), only if the message ends with one: the file's last record included |

`Offset + RecordLength` is the next record's `Offset` (or the file's end), so copying every record (and the preamble) reproduces the
file. `MessageLength` excludes the separator; a record whose message ends in an empty line and has no separator after it loses that
line to the separator rule (an unavoidable ambiguity of the format), and a message that does not end in a line ending has no
separator.

### Where a message ends

1. **Envelope lines.** A line starting `From ` at a line start begins a message when it reads as an envelope line (RFC 4155 §3):
   after the sender there is a month name next to a one- or two-digit day, and later a time containing a colon. That accepts
   `From a@b Mon Jan  1 00:00:00 2024`, `From 123@xxx Mon Jan 01 00:00:00 +0000 2024` (Gmail Takeout) and `From - Wed Jan 1 …`.
   Any other `From ` line is message text and sets `AmbiguousFromLine` (S1, S5). `MboxFromLineMode.Any` takes every `From ` line.
   Envelope lines longer than `MaxEnvelopeLineBytes` (4096) are text in strict mode, an envelope with `OverlongEnvelopeLine` in `Any` mode.
2. **`Content-Length`.** Unless the variant is mboxo or mboxrd, the reader looks at the header block (at most `MaxHeaderBytes`,
   64 KiB) for the first `Content-Length`. The message body is the length that many bytes after the empty line ending the headers.
   If what follows is the end of the file, or an envelope line, or one empty line and then either, the length is **valid**: the message
   ends there and its body is not scanned (so an unquoted envelope-looking line inside is kept). Otherwise the length is ignored and the
   message ends at the next envelope line, with `ContentLengthMismatch` (S2). Two differing values: the first wins, `AmbiguousContentLength`.
3. An envelope line with no empty line before it still begins a message: `MissingSeparatorLine` on the message before it (S8).

### Variants

| Variant | Body line quoting when writing | Reading | Notes |
|---|---|---|---|
| mboxo | `From ` → `>From ` | unquote exactly one `>` | not reversible: a stored `>From ` could have been either |
| mboxrd | `>*From ` gets one more `>` | unquote one `>` from `>+From ` | reversible; the writer's default |
| mboxcl | as mboxo, plus `Content-Length` | as mboxo; length found by the rule above | length counts the stored body |
| mboxcl2 | none, plus `Content-Length` | stored bytes as they are | only the length protects the text |
| Unknown (reading) | | non-length entries unquoted as mboxrd; entries ended by a valid length returned as stored (S9) | |

`MboxSniffer.Detect` guesses the variant from the first messages (up to 8 MiB or 2,000): a valid `Content-Length` means mboxcl, or
mboxcl2 when a length-delimited body holds an unquoted envelope line; a `>>From ` line means mboxrd; a lone `>From ` is taken as mboxrd.
No evidence leaves it Unknown. Reading never depends on this call.

### Memory

One pooled window (default 64 KiB) holds the bytes being scanned. A scan looks for the vectorised pattern `\nFrom ` and
checks each hit; a length-delimited message is skipped without being read. Enumeration allocates no objects per message (entries are
structs); the measured numbers are in the benchmark notes.

### Writing

`MboxWriter` appends one message at a time: the envelope line (`From <sender> <asctime date>`, UTC; sender and date default to
`MAILER-DAEMON` and the clock, both overridable per call), the message with the variant's quoting, a line ending if the message lacks
one, and an empty line. The file is complete after every call. For mboxcl and mboxcl2 an existing `Content-Length` header is
removed and a correct one added as the last header; a body without a final line ending gets one (the length counts it). A header
block over 1 MiB is refused.

`AppendToExisting` looks at the last bytes of a seekable file and adds a missing line ending or separator before the first new envelope
line. `AppendRecord` and `WritePreamble` copy bytes of another file verbatim, so read→write of untouched records is byte-identical.

### Metadata

`MboxMetadata` reads hints from the header block: `Status` `R` (seen); `X-Status` `A F D T` (answered, flagged, deleted, draft);
`X-Gmail-Labels` (comma separated, quoted labels may hold commas, encoded words decoded); `X-GM-THRID`; `X-Keywords`;
`Content-Length`. `MboxEnvelope` parses sender and date from the envelope line (no zone means UTC).

## Maildir

A Maildir folder holds `cur`, `new` and `tmp`. `MaildirStore` reads `cur` then `new`, each by file name (ordinal); `tmp` is
never listed; names starting with `.` inside `cur` and `new` are ignored.

- **Names.** `unique[:2,flags[,extra]]`. The separator before `2,` is `:` in the Maildir specification, which Windows file names cannot
  hold; Windows tools use `;` (isync's `InfoDelimiter` defaults to `;` on Windows and `:` elsewhere). **Reading accepts both on every
  platform** (a `;` counts only when `2,` follows it, so other semicolons stay in the unique part; a colon wins when both occur). **Writing**
  uses `MaildirStoreOptions.InfoSeparator`, default `;` on Windows and `:` elsewhere; only those two are allowed (S19). `dovecot-uidlist`
  file names are matched with either separator. Standard flags `D F P R S T` map to draft, flagged, forwarded ("passed"), answered, seen,
  trashed (`Deleted`). Lowercase `a`–`z` are Dovecot keyword slots. A file in `new` also carries `Recent`. `,S=<size>` and `,W=<vsize>` in the
  unique part are read as sizes. Any other info version (`:1,…`) is kept in `RawInfo` and not interpreted. Names are built with
  flags in ASCII order (S10).
- **Keywords.** Dovecot: `dovecot-keywords` lines `<index> <name>`, letter `a` is index 0. Courier: `courierimapkeywords/:list` (S13).
  A letter without a name appears as itself with `UnknownKeywordLetter`. A folder holds at most 26 Dovecot keywords.
- **UIDs.** `dovecot-uidlist`: a header line, then `<uid> [fields] :<file name>`; the entry's `Uid` is matched by the unique part.
- **Folders.** Maildir++: the root is the inbox (folder name `""`); a subfolder is a directory `.Name` holding `cur`/`new`, and
  the dot is the hierarchy separator (`.Archive.2024` is folder `Archive.2024`). Nested layouts (a folder inside a folder
  directory, as Python's `add_folder` makes on a subfolder) are not Maildir++ and are not read.
- **Delivery** writes the message under `tmp` with a unique name, then renames it into `new`, or into `cur` with `<separator>2,flags` when it has
  flags or keywords. Default unique names are `seconds.M<microseconds>P<pid>Q<counter>.<host>` with `/` and `:` in the host escaped; supply a
  factory for reproducible names. The message bytes are written unchanged.
- **`SetFlags`** renames into `cur` with the new info section; the unique part and the content do not change.
- **`CopyToAsync`** copies a store through the model: every message file under its exact name, and the folder control files
  byte for byte.

## EMLX and Apple Mail bundles

An `.emlx` file is a decimal byte count and a line feed, that many bytes of message, and (usually) an XML property list.
`EmlxReader.Read` returns the three ranges; a bad count line gives no message and the whole file as trailer (`BadByteCount`), a count
past the end of the file is cut there (`TruncatedMessage`). The property list gives the flag word, dates, remote id and conversation id
(S12); `Plist.TryParse` reads XML property lists only, ignores DTDs (nothing is fetched or expanded), and is bounded (1 MiB,
32 levels, 100,000 values). `.partial.emlx` files hold messages without their attachments, which Mail keeps in
`Attachments/<number>/` beside the `Messages` directory; the bundle reader names that directory and does not read it.

`AppleMailBundle` finds `Messages` directories at any depth up to eight below the bundle (older layouts hold `Messages` directly, newer
ones nest it under `<GUID>/Data/…`) and lists their `.emlx` files by number. `EmlxWriter.Write` writes a message with a new property
list; `WriteFrom` rewrites a file from its parts (canonical count line, message, original trailer bytes), so a file with a canonical
count line is reproduced exactly.

## Outlook Express (DBX)

An Outlook Express 5/6 store is a directory of `.dbx` files: `Folders.dbx` (the folder hierarchy), one message database per folder
(`Inbox.dbx`, `Sent Items.dbx`, user folders) and helpers (`Pop3uidl.dbx`, `Offline.dbx`) that hold no mail. Reading is read-only and
never writes. Facts are from the oedbx DBX format notes (Arne Schloh, updated by Zvonko Tesic); the constants and structures here are
our own statement of them.

**File.** Every database starts with `cf ad 12 fe` and a class id: `c5 fd 74 6f` message database, `c6 fd 74 6f` `Folders.dbx`,
`c7 fd 74 6f` `Pop3uidl.dbx`, `30 9d fe 26` `Offline.dbx`. Sniffing reads those eight bytes only (never the extension). The fixed header is
0x24bc bytes, the file info follows (length at header 0x1c), then the middle sector of objects. Every object starts with its own absolute
file offset (its *marker*); a pointer is an absolute offset. A file shorter than the header reads as nothing (`DbxTruncatedHeader`).

**Index tree.** Header 0xe4 points at the root of a B-tree of 0x27c-byte nodes: 0x18 bytes (marker, unused, left child, parent, node id,
entry count, unused, left subtree count) and up to 0x33 entries of (value, right child, subtree count). Values are pointers to records.
The walk is in order: left child, then each value followed by its child, which gives the records in index order. Header 0xc4 repeats the
entry count. The walk keeps one 0x27c buffer per level on an explicit stack, stops at `MaxTreeDepth` (32) and at a node budget of
`file length / 0x27c + 1` visits (`DbxTreeLoop`), skips a node whose address is outside the file or whose marker is wrong
(`DbxBadTreeNode`), clamps an entry count above 0x33, and reports a final count different from the header's (`DbxEntryCountMismatch`,
the tree wins). The other two trees (watched or ignored threads; sorted-by-parent folder lists) are not read: the main tree already holds
every record.

**Records.** A record (indexed info) is a 12-byte header (marker, body length, object length, field count, change count), an index of
field-count 4-byte entries, and a data area. An entry is a field byte (low 7 bits the field number, top bit "value is in the entry") and
three value bytes: a direct value is that 24-bit integer; otherwise it is an offset into the data area, and the field runs to the next
stored offset or the end of the area. Strings end at a zero byte and are decoded as windows-1252 (S15). Times are Windows FILETIMEs.
A body length over `MaxInfoBytes` (1 MiB) or past the end of the file is a defect (`DbxBadIndexedInfo`; a record cut by the end of the
file keeps what is there), and a field offset outside the data area reads as absent. Message record fields used: 0x00 index, 0x01 flag
word, 0x02 sent time, 0x03 line count, 0x04 pointer to the first text segment, 0x05 subject without prefixes, 0x06 saved time, 0x07
message id, 0x08 subject, 0x09/0x0d/0x0e sender, 0x0a message id answered, 0x10 priority, 0x11 stated length, 0x12 received time,
0x13/0x14 first recipient, 0x1a account. Any field is reachable by number (`DbxRecord.TryGetInt32`, `GetString`, `GetBytes`).
Folder record fields: 0x00 index, 0x01 parent (-1 for the root "Outlook Express"), 0x02 name, 0x03 file name, 0x06 flags, 0x07 and 0x08
counts, 0x09 special-folder kind (unverified, S18).

**Messages.** The text is a singly linked chain of 0x210-byte segments: marker, body length (0x200), text length, next pointer, then the
text; the last has next 0. Concatenating the text lengths gives the message exactly as Outlook Express stored it, with the line ends it
received (usually CR LF); nothing is converted. The chain is followed lazily: `OpenMessage` is a forward-only stream, `MeasureMessage` walks
only the 16-byte headers (3 % of the bytes), `ReadMessage` fills an array of exactly the measured length. A bad pointer or marker ends the
chain (`DbxChainBroken`), a text length over the body length or the file is cut (`DbxBadSegmentLength`), and a chain that revisits a segment
(found with Brent's method: two variables, no set) or exceeds the file or `MaxMessageBytes` (2 GiB) ends with `DbxChainLoop`; a loop is
noticed within one more lap, so up to a lap of its text may repeat. The stated length (field 0x11) is advisory (S14). A record with no
chain pointer yields an empty message (S17). Free (deleted) objects are never followed; their recovery is a forensic mode.

**Store folders.** `DbxStoreFolder` reads `Folders.dbx` (found ignoring case), pairs each folder with its message file by the stored name
(ignoring case) and enumerates folder by folder in index order, then message files no folder names, as `<file name>` folders with
`DbxUnlistedFile`. The folder path joins names up the parent chain and leaves out the top-level folders (those with parent -1: the "Outlook Express" root, "Local Folders" and each account), so `Inbox` and `alt.2600` are paths, climbing at most 64 levels; two accounts holding a folder of the same name give the same path (the record itself says which account through `ParentIndex`).
`Mail.Open` takes a `.dbx` file (`MailFormat.Dbx`; `Folders.dbx` and the helpers enumerate as empty) or a store directory
(`MailFormat.DbxStore`, recognised by a `Folders.dbx` with the folder class id). An item holds no open file: it reopens its message file to
read.

**Memory.** Enumeration holds one record (its few hundred bytes) and the tree stack; messages are read when asked. The source is read
synchronously (an async-only source is refused); `EnumerateMessagesAsync` runs the same code between cancellation checks.

## Defects

| Defect | Where | Meaning |
|---|---|---|
| `PreambleBeforeFirstMessage` | mbox | bytes before the first envelope line, kept as the preamble |
| `MissingSeparatorLine` | mbox | no empty line before the next envelope line |
| `AmbiguousFromLine` | mbox | a `From ` line that is not an envelope line, kept as text (strict) |
| `ContentLengthMismatch` | mbox | `Content-Length` does not end at an envelope line; ignored |
| `AmbiguousContentLength` | mbox | two different `Content-Length` values; the first is used |
| `OverlongEnvelopeLine` | mbox | envelope line over the limit |
| `UnrecognisedFileName` | Maildir | info section is not version 2 |
| `UnsortedFlags` | Maildir | flag letters out of order or repeated |
| `UnknownKeywordLetter` | Maildir | a keyword letter with no name in the folder's keyword file |
| `MalformedControlFile` | Maildir | a keyword, UID or Courier list line that could not be read |
| `BadByteCount`, `TruncatedMessage` | EMLX | as described above |
| `DbxTruncatedHeader` | DBX | the file is shorter than the 0x24bc-byte header; nothing is enumerated |
| `DbxBadPointer`, `DbxBadObjectMarker` | DBX | a record pointer outside the file or inside the header; an object not starting with its own offset. The object is skipped |
| `DbxBadTreeNode`, `DbxTreeLoop`, `DbxEntryCountMismatch` | DBX | an unusable index node (skipped); a tree deeper than the limit or revisiting nodes (the walk stops); a count different from the header's (the tree wins) |
| `DbxBadIndexedInfo` | DBX | a record whose lengths do not fit (skipped when unreadable, kept when only cut by the end of the file) |
| `DbxChainBroken`, `DbxChainLoop`, `DbxBadSegmentLength` | DBX | a message chain ending at a bad object; a cycle or an over-long chain; a text length larger than the body or the file |
| `AmbiguousDbxDeclaredLength` | DBX | the record's stated length differs from the chain's; the chain wins (S14) |
| `AmbiguousDbxNoMessageText` | DBX | the record has no chain; the message is empty (S17) |
| `DbxUnlistedFile` | DBX | a message file `Folders.dbx` does not name (S16) |

## Checks

`tools/mailbox/run.sh` is the whole check (see its header). The method:

- **Boundaries.** `compare_mbox.py` runs the file through OneMail in an oracle-compatible mode (`Any`, Content-Length ignored), Python's `mailbox`
  and MimeKit's mbox parser (`tools/oracles/mailbox-driver`), and compares the message count, each envelope offset and message end. Each
  difference is tied to a table entry (S3, S4, S5); OneMail's default mode is compared with its own compatible mode, and every
  difference must carry the defect that explains it (S1, S2).
- **Ground truth.** `gen_mboxes.py` writes every variant, LF and CRLF, and edge shapes (no separators, preamble, wrong lengths, Takeout
  headers), with independent writers (hand-written quoting, Python's `mailbox.mbox`) and a sidecar holding the SHA-256 of each message as
  given. `OneMailRanges verify` reads them back.
- **Round trips.** Record copy of every mbox, a message-level rewrite (unquote, append again with the same variant, sender and date),
  Maildir copy through the model, EMLX rewrite from parts. Files that a message-level rewrite changes are listed with the reason
  (no separators or no final separator in the source, a preamble, a wrong length corrected, an unquoted writer, a Takeout envelope with a zone).
- **Writers.** `OneMailRanges write` makes every variant and a Maildir with flags and keywords; `py_check_written.py` reads them with
  Python's `mailbox` (mboxcl2 excepted, S6).
- **DBX against undbx.** `tools/dbx/check.sh` writes stores with the test-side builder (`DbxTestBuilder`: header, file info, conditions, shuffled
  or contiguous chains, trees of any fan-out, record variants), extracts every message with OneMail and with undbx 0.21 (a distribution
  package run as a program), and compares each folder's messages as a multiset of SHA-256 hashes (`compare.py`). Real stores
  (`tools/corpus/fetch-dbx.sh`) go through the same comparison. Unit tests cover the builder round trip over fan-outs, shuffles and sizes, the
  hostile cases and the happy path.
- **Fuzz.** Targets `mbox`, `emlx` and `dbx` (`fuzz/`): the window size must not change the entries, entries must tile the file, copying records
  must reproduce it, mboxrd quoting must be reversible, a DBX message read into an array must equal the streamed read and the measured chain, and nothing may throw.


<!-- https://recursiveai.org/onemail/docs/composition/ -->

# Replies and forwards

`RecursiveAI.OneMail.Composition` makes the replies and forwards a mail client would, from any item in the shared view (MIME, MSG,
PST/OST, TNEF, OLM, DBX). `item.CreateReply()`, `item.CreateReplyAll()` and `item.CreateForward()` are one call each over
`MessageComposer`. The result (`ComposedMessage`) holds the new message as a MIME entity tree, and, for Outlook originals, the same
message in the MAPI property model. Nothing is sent, and the original is not changed. The same inputs and options always give the same bytes
when the clock (`TimeProvider`) and the random source (`RandomBytes`) are fixed.

## Subject

One prefix, never stacked. Every leading prefix of the original is removed (`Re: RE: Fwd: AW: topic` becomes `topic`), then the new
prefix is put on: `Re: ` for replies, `Fwd: ` for forwards (both can be replaced in the options). A prefix is a label from
`SubjectPrefixes.Labels`, ignoring case, an optional counter in square brackets or parentheses (`Re[2]:`, `Re(3):`), optional blanks, and
a colon (ASCII or full-width). A word that is not in the list (`Status: x`) is part of the subject. A subject that is only prefixes
becomes empty. Control characters and line separators in the original's subject become spaces, and trailing white space is dropped, so a
decoded subject can never start a new header.

The labels are the ones language packs of common clients write; they are an agent default and not a specification: Re, Fw, Fwd (English); AW, WG
(German); SV, Svar, VS, VB, ATB (Scandinavian); Antw, Doorst (Dutch); TR, Ré, Réf, Rép (French); RV, PD, Reenv (Spanish); ENC, RES, Resp
(Portuguese); R, RIF, Inoltro (Italian); Odp (Polish); VL (Finnish); YNT, ILT (Turkish). Labels in non-Latin scripts are not recognised.

MAPI: the MS-OXCMAIL import splits the subject (MS-OXCMAIL 2.2.3.2.6.1), so `PidTagSubjectPrefix` is `Re: ` or `Fwd: ` and
`PidTagNormalizedSubject` is the rest.

## Threading headers (RFC 5322 §3.6.4)

Replies only. A forward starts no reply thread: it has no `In-Reply-To` or `References` (agent default; clients differ), but keeps the
conversation topic and index.

- `In-Reply-To`: the original's `Message-ID`.
- `References`: the original's `References`, or its `In-Reply-To` when it has none, then its `Message-ID` (not repeated when it is already
  last). Identifiers that cannot be written (white space, angle brackets, control characters, longer than 512 characters) are dropped with
  `InvalidMessageIdDropped`.
- Too long a chain is cut to `MaxReferences` (default 20, at least 2) by keeping the first identifier and the most recent ones, and raises
  `ReferencesTruncated`.
- An original without a `Message-ID` gets no `In-Reply-To` (`ParentHasNoMessageId`); `References` holds what it had.
- The new `Message-ID` comes from `MessageIdFactory`, or `random@MessageIdDomain`; with neither, none is written and the sending server adds one.

## Conversation topic and index (MS-OXOMSG 2.2.1.3)

`Thread-Topic` and `Thread-Index` (MAPI `PidTagConversationTopic`, `PidTagConversationIndex`) are written when the original is an Outlook item
or already has an index, or when `ConversationIndexMode.Always` is set; MIME messages from other clients get none by default.

- The topic is the original's topic, else its normalised subject.
- A valid index (six bytes of FILETIME, a 16-byte GUID, then five-byte levels) gets one more level: a delta code bit and 31 bits of time
  delta from the header's time, then a random byte. The delta is `TimeDiff >> 18` when the high 32 bits of the difference AND
  `0x00FE0000` are zero (under about 1.7 years), else `TimeDiff >> 23` with the delta code set. A clock before the header's time gives
  delta 0 (`ConversationIndexTimeClamped`).
- No index, or one that is not in the form (`ConversationIndexInvalid`), starts a new thread: a header block with the original's delivery or
  sent time (never later than now) and a new GUID, then the level.
- An index that has reached 255 bytes is kept as it is (`ConversationIndexFull`).

`ConversationIndex.TryParse`, `CreateHeader` and `AppendLevel` are public and decode and extend indexes by themselves.

**Observed against the spec.** MS-OXOMSG draws the header as a reserved byte 0x01 followed by a 40-bit time (the high 32 bits of the FILETIME and
the top 8 bits of its low half). Indexes in real Outlook items (45 MSG files and the CFReDS OST) carry the time from the first byte on: `01 CD 50
A5 70 16` is the FILETIME's top six bytes, whose first byte is 0x01 for every date between 1829 and 2057. OneMail reads and writes that form, so the
header time has 2^16 FILETIME ticks (6.5 ms) of resolution. The spec's delta rules are followed as written.

## Recipients

- **Reply**: `Mail-Reply-To` if present (MIME items), else `Reply-To`, else `From` (RFC 5322 §3.6.2). Replying to a message the user sent
  (every address is one of `OwnAddresses`) goes to that message's `To` instead.
- **Reply all**: the reply target plus the original's `To`, then `Cc`; the user's own addresses are removed; each address appears once, `To`
  taking precedence over `Cc`. A `Mail-Followup-To` header (MIME items) replaces the whole list. If only `Cc` is left it becomes `To`. If
  nobody is left, the people the user wrote to are used.
- Addresses are compared ignoring case (the local part is technically case-sensitive; no real server treats it so).
- Recipients without an SMTP form (an Exchange distinguished name only) are never written and never lost silently: they are in
  `ComposedMessage.Unaddressable` with `UnaddressableRecipient`. An address that is not usable (no single `@`, white space, specials, longer
  than 320 characters) is dropped with `InvalidAddressDropped`. More than `MaxRecipients` (default 500) stops the list with `TooManyRecipients`.
  No recipient at all raises `NoReplyTarget`; the message is still made so the caller can address it.
- Display names are written as atoms, quoted strings or RFC 2047 encoded words as needed; control characters become spaces.

## Quoting

Top posting: the author's `Text`/`Html` (if given), then the quote.

- **Text**: the attribution line (`On Mon, 4 Mar 2024 09:00:00 +0100, Ann <a@x.example> wrote:` by default; `Attribution` replaces it), then
  every line of the original's text body prefixed with `> ` (`>` alone for an empty line). A `format=flowed` original gives a `format=flowed`
  reply (RFC 3676 §4.5): quoted lines get `>` with no space, soft line breaks (trailing space) survive, our own lines are space-stuffed, and
  `delsp` is kept.
- **HTML**: `<div>attribution</div><blockquote type="cite">…</blockquote>`, the original's content between `<body>` and `</body>` (all of it when
  there is no body element). The quoted HTML is not sanitised: it is as hostile as the original, and anything that shows it goes through the display sanitiser.
- A body larger than `MaxQuoteCharacters` (default 1,048,576) is cut at a line end (`QuoteTruncated`).
- Both representations are quoted when the original has both (`multipart/alternative`); one when it has one. `QuoteBodyFormat.TextOnly` or
  `HtmlOnly` quotes only that one; if it does not exist, nothing is quoted (`BodyNotAvailable`): HTML is not turned into text here.
- Inline images (`cid:`) that the quoted HTML refers to are carried in a `multipart/related`. A reply carries other attachments only with
  `IncludeAttachments`.
- **Inline forward**: the banner `---------- Forwarded message ----------` and the header block (From, Date, Subject, To, Cc), then the
  original's body unquoted, with all attachments carried. Embedded messages (`message/rfc822`) are carried as messages.
- **Attachment forward**: a `message/rfc822` part with `Content-Disposition: attachment; filename="<subject>.eml"`. MIME originals are
  embedded as their own bytes (a fresh parse of the item's message gives back exactly the source bytes, malformed input included). Outlook
  items are exported by the MS-OXCMAIL mapping. An original that is neither (an OLM record) is rebuilt from the shared view with
  `ParentSynthesized`.
- At most `MaxCarriedBytes` (default 64 MiB) of attachments and embedded originals are carried; the rest is skipped with `AttachmentSkipped`.

## MAPI result

When the original is an Outlook item (or `CreateMapi` is true) the finished MIME message is imported by `MimeToMapiConverter`, so the
MAPI message is exactly what the MS-OXCMAIL mapping makes of the MIME one, and `ComposedMessage.Report` is that import's loss report.
`PidTagInReplyToId` holds the one identifier, `PidTagInternetReferences` the chain, `PidTagConversationTopic` and `PidTagConversationIndex` the topic and
the index. An attachment forward carries the original as an embedded message.

## Not covered

Sending, signing, sanitising the quote, HTML to text, meeting responses (accept/decline), drafts' `X-Unsent`, `Disposition-Notification-To`, and
translated attribution or banner text (pass your own).

## How it is checked

`tools/composition/run.sh`: replies, reply-alls and both forwards of every sampled item, parsed back by OneMail, MimeKit, Python `email` and Go
`net/mail`, with subject, From, To, Cc, In-Reply-To and References compared to what the composer says it wrote. See `docs/compatibility.md` for the
latest numbers. `fuzz/` has the `composition` target.


<!-- https://recursiveai.org/onemail/docs/msg/ -->

# MSG and OFT reading

`RecursiveAI.OneMail.Msg` reads Outlook message files (`.msg`) and templates (`.oft`) into the MAPI model (`docs/mapi.md`).
The container is MS-CFB, read through OneArchive's compound file reader; the content is MS-OXMSG. An OFT is an MSG whose
message class is a template; the same reader opens it.

## Using it

`MsgFile.TryOpen(path | memory | stream | source, ...)` returns false with typed defects when nothing can be read, otherwise a
`MsgFile` with `Message`. `ReadEverything()` loads every recipient, attachment and embedded message and returns the defects
found, so a caller can validate a file in one call. `MsgFile.IsMsg(source)` is the content sniff used by `MailFormatSniffer`:
a compound file whose root has a `__properties_version1.0` stream. The file extension is never consulted.

Opening reads the directory and the top-level property stream only. Value streams, attachment data and children are read on
demand: opening a 2 MB file with 1,321 recipients allocates 4.7 MB; a 3.5 KB file about 21 KB.

## Layout read (MS-OXMSG 2.1-2.4)

- Root: `__properties_version1.0`, `__substg1.0_<tag>` value streams, `__recip_version1.0_#XXXXXXXX` and
  `__attach_version1.0_#XXXXXXXX` storages, `__nameid_version1.0`.
- Property stream headers: 32 bytes for the top-level message, **24 for an embedded message** (2.4.1.2), 8 for recipients and
  attachments. Entries are 16 bytes: tag, flags, then 8 bytes of value (fixed types) or size and reserved (variable types).
- Embedded messages and object attachments are in `__substg1.0_3701000D`; as a storage it is read as a message (or raw
  storage for other objects), as a stream its bytes are the value.
- Named properties: the GUID, entry and string streams of `__nameid_version1.0` map IDs from 0x8000 to names (2.2.3). The
  entry's own index is ignored in favour of its position (`NamedPropertyIndexMismatch`). Embedded messages use the top-level
  map; their own is ignored (`NestedNamedPropertyMap`).
- Multi-valued properties: fixed-length elements in one stream; variable-length ones as a length stream plus one value stream
  per element (2.1.3). The length stream is read at open.

## Code page for 8-bit strings

Resolution order, first hit wins: the object's own `PidTagMessageCodepage`, the ANSI page of `PidTagMessageLocaleId`,
`PidTagInternetCodepage` (unless the strings are Unicode), the enclosing message's answer, then 1252. Reported as
`AmbiguousCodePage` when there is no `PidTagMessageCodepage` and 8-bit strings exist. The order was fixed by the Apache POI
and Tika files that carry only a locale or an Internet code page (a Big5 file, a Cyrillic file, a German file in UTF-8 form).
`MapiObject.String8EncodingSource` says which rule applied. An unknown page falls back to 1252 (`UnsupportedCodePage`).

## Size field of strings

MS-OXMSG says a string's size is the stream length plus the terminator for Unicode (2.4.2.2). ANSI files written by some
producers store the terminating NUL inside the stream. Either is accepted; any other mismatch is `ValueSizeMismatch`, and the
stream's bytes are the value.

## Damage policy

| Situation | Result |
|---|---|
| Not a compound file / damaged container | Open fails: `NotACompoundFile`, `CompoundFileInvalid` (OneArchive's reader is strict, see below) |
| No `__properties_version1.0` | `NotAMessage` |
| Entry without its value stream | property kept, empty: `ValueStreamMissing` |
| Value stream without an entry | read as a property: `AmbiguousOrphanValueStream` |
| Property stream truncated | whole entries kept: `PropertyStreamTruncated` |
| Same tag twice | both kept, last wins: `AmbiguousDuplicateProperty` |
| Recipient/attachment counts differ from header | storages found win: `RecipientCountMismatch`, `AttachmentCountMismatch` |
| Strings of the other kind than the store mask says | read by their own type: `AmbiguousStringType` |
| Unknown streams | ignored: `UnreferencedStream` |

The full list is `MapiDefectKind`. Truncated or damaged compound files that OneArchive's reader rejects cannot be opened at
all; two minimised fuzzer files in the Apache POI set are in that class.

## Limits (`MsgReaderOptions`)

Properties per object 1,048,576; recipients and attachments 65,536 each (the specification says 2,048; real files exceed it);
embedding depth 32; values per multi-valued property 1,048,576; bytes per value 256 MiB; kept defects 10,000. Sizes are checked
before anything is allocated. The compound-file reader has its own total-size limit (1 GiB by default).

## Property-exact re-encoding

`MsgStreams.Encode` re-encodes an object's property stream and value streams from the model. For all 53 openable corpus files
and 7 generated fixtures, every object's property stream and every value stream re-encode byte for byte (see below), so a
writer can be built on the model without loss.

## How it is checked

- `tests/OneMail.Tests/Msg`: generated fixtures (test-side builder, `testdata/msg/generated`, regenerate with
  `tools/msg/MsgTool gen`) and the public corpus when fetched (`tools/corpus/fetch-msg.sh`).
- `tools/msg/MsgTool streams <file>` compares the re-encoded streams with the file's.
- `tools/msg/compare_extract_msg.py` compares, per object, property tags, flags, entry bytes, value stream bytes, decoded
  strings and named-property names against the Python package extract-msg, run as a released package (version in
  `tools/msg/README.md`). Differences are either listed with a reason in the script or reported as unexplained.
- Fuzz target `msg` (`fuzz/run.sh msg <seconds>`): patches a valid fixture and opens it, enumerating everything.

Results (62 files: 55 corpus + 7 generated): 60 open, 60 re-encode identically; 58 compared, 1,516 objects, 15,609
properties, 9,822 value streams, 6,131 decoded strings, 955 named properties, **0 unexplained**. Explained: 36 strings in files
with no message code page (the oracle uses a fixed default; ours follows the rule above), one orphan stream, one signed
attachment the oracle exposes no properties for. Not compared: the two refused files, a generated empty file and a generated
file with an OLE attachment (the oracle cannot open them).


<!-- https://recursiveai.org/onemail/docs/pst-messaging/ -->

# PST/OST messaging layer

What `RecursiveAI.OneMail.Pst.Messaging` reads, how it treats damage, and what was checked. The structures are MS-PST 2.4
(store, folders, messages, attachments, recipients, named properties) on the lists-tables-properties layer
(`docs/pst-ltp.md`). The same code reads ANSI, Unicode, 4 KiB and OST files; the differences live in the node database
(`docs/pst-ndb.md`). Every object it returns is a `RecursiveAI.OneMail.Mapi` property bag, so a PST message, a folder and an
MSG message are read with the same accessors (`docs/mapi.md`).

## Types

| Type | Spec | What it is |
|---|---|---|
| `PstStore` | MS-PST 2.4.3 | `TryOpen` over a path, stream, byte array or random-access source. `Root`, the store's `Properties`, `PasswordCrc`, `TryGetNamedProperty`, `EnumerateFolders`, `EnumerateMessages(PstMessageSet)` (and `Async` twins), `TryOpenMessage(nid)`, `ReadEverything()` (walks and reads every object, returns `PstWalkCounts`). |
| `PstFolder` | 2.4.4 | Properties, `Name`, `Path`, `Depth`, `ParentFolder`, `EnumerateSubfolders`, `EnumerateMessages(PstMessageSet)` and `EnumerateMessageIds`, `IsSearchFolder`. |
| `PstMessage` | 2.4.5 | A `MapiMessage` with `Recipients`, `Attachments`, defects, `IsEmbedded`, `IsAssociated`. |
| `PstRecipient`, `PstAttachment` | 2.4.5.3, 2.4.6 | Rows of the recipient and attachment tables with their properties; an attachment may carry an embedded `PstMessage`. |
| `PstReaderOptions` | | Limits (below) and the options of the layers underneath. |

`MapiMailItem` (root namespace) adapts any `MapiMessage` to the shared view (`docs/shared-view.md`); `Mail.Open` uses it for
PST, OST and MSG files.

## How things are loaded

- **Names.** The named-property map (NID 0x61, MS-PST 2.4.7) is read once at open; property IDs of 0x8000 and above are
  resolved to GUID plus numeric ID or string name through it. A missing or unreadable map leaves those properties unnamed and
  raises `NodeMissing` or `ContextUnreadable`; nothing is dropped.
- **Folders.** From the root folder's hierarchy table, depth first. A folder reached twice, or deeper than
  `MaxFolderDepth`, is not entered again (`FolderLoop`). Contents tables list the messages; a message listed in a table
  whose node is missing is reported (`NodeMissing`) and skipped.
- **Search folders** (their contents are links to other folders' messages) and **associated (FAI) messages** (rules, views,
  settings) are not enumerated by default: `PstMessageSet.Normal` is the ordinary contents of ordinary folders; `Associated` and
  `All` add the hidden messages. Search folders are listed by `EnumerateFolders` (`IsSearchFolder`) but their contents are not
  enumerated, so no message appears twice. `Mail.Open` uses `Normal`.
- **Messages.** The property context is read when the message is opened. Values held in the heap are copied at that point;
  values held in subnodes (large bodies, attachment data, big tables) stay on disk and are read when asked, by range
  (`PstValueStream` for streams). Items therefore need no disposal and stay valid until the store is disposed.
- **Recipients and attachments.** Each table is a table context in a subnode of the message. The row's identifier is checked
  against the subnode the object is stored under (`RowIdMismatch`). A row whose properties are in a separate subnode
  (attachments) is merged with it; the table's bookkeeping columns (0x67F2, 0x67F3) are not exposed as properties.
- **Embedded messages.** An attachment of method `ATTACH_EMBEDDED_MSG` holds its message as an object property: an 8-byte
  structure (subnode NID, size) in the heap (MS-PST 2.3.3.5). A bare subnode NID is accepted too (ambiguity P2). The embedded
  message is read the same way as a top-level one, to `MaxEmbeddingDepth`.
- **Large tables.** A recipient or attachment table whose row matrix is in a data tree is read through the node database's data
  tree, row by row.
- **Password and obfuscation.** `PasswordCrc` returns the stored CRC. It is not protection: the store reads whatever the
  value. Block obfuscation is decoded by the node database for every file.

## Limits

| Limit | Option | Default |
|---|---|---|
| Folder depth | `MaxFolderDepth` | 256 |
| Properties in one object | `MaxProperties` | 1,048,576 |
| Recipients of a message | `MaxRecipients` | 65,536 |
| Attachments of a message | `MaxAttachments` | 65,536 |
| Embedded-message depth | `MaxEmbeddingDepth` | 32 |
| Named properties | `MaxNamedProperties` | 32,768 |
| Defects kept (the rest are counted in `DroppedDefects`) | `MaxDefects` | 10,000 |

Reading never throws on malformed content. Beyond the limits, rows and objects are skipped and `LimitReached` is raised (`DefectLimitReached` when the defect list itself is full).
`TryOpen` returns false with defects only when no usable header or root exists.

## Defects

`MapiDefect` carries a kind, a message, the node where known, and an offset when the lower layer gave one. Added by this layer:
`NodeMissing`, `ContextUnreadable`, `ValueUnreadable`, `LowerLayerDefect` (a node database or LTP defect met while reading an
object), `FolderLoop`, `RowIdMismatch`, and the two ambiguities `AmbiguousAttachmentTableMissing` and
`AmbiguousObjectReference` (P1, P2 in `docs/ambiguities.md`). Store-wide node database defects are on `DatabaseDefects`.

## How it is checked

Worker, libpff `pffexport` 20180714 (black box), `tests/PstOracle`:

- Real files (testPST, testPST_variousBodyTypes, the CFReDS OST): folders, items, recipients, attachments (names, sizes,
  content), embedded messages, subject, sender, body text/HTML/RTF values and times, against `pffexport -f all -m all`:
  0 differences. Items libpff does not export (31 folder-associated messages of the OST) are counted separately.
- Property-exact: every property bag against the raw LTP entries (tag, type, value bytes): 0 differences on 27 objects /
  959 properties (testPST), 16 / 796 (variousBodyTypes), 109 / 3,763 (OST).
- Generated files from the test builder in ANSI, Unicode and 4 KiB layouts (up to 128 items, a 300-recipient message,
  110-row tables in data trees, 3-deep embedded messages, large attachments in subnodes): 0 differences from `pffexport`
  (except the 4 KiB case below) and 0 from the property-exact check.
- **libpff limitation (4 KiB files).** libpff 20180714 does not read a table whose row matrix lies in a data tree in a 4 KiB
  file: on the generated 4 KiB store of 215 items it reads 18 items, and the 300-recipient message gives 0 recipients (tried with
  data block sizes of 8,168 and 65,512 bytes). Our reader reads all of them, and the property-exact check agrees with the raw
  entries. The same libpff agrees with us on every small table of the real 4 KiB OST. So the 4 KiB large-table case has no
  differential until another oracle (Outlook) exists.
- Fuzzing: the `pst-messaging` target (bounded run recorded in the unit report).
- Memory: `PstOracle pstalloc <file> <passes>` prints bytes allocated per message and the working set after each pass.


<!-- https://recursiveai.org/onemail/docs/pst-ltp/ -->

# PST/OST lists, tables and properties (LTP) layer

What `RecursiveAI.OneMail.Pst.Ltp` reads, how it treats damage, and what was settled by looking at real files. The structures
are MS-PST 2.3 (heap-on-node, BTree-on-Heap, property context, table context). The layer sits directly on the node database
layer (`docs/pst-ndb.md`) and gives values no meaning: it returns property IDs, property types and **references to bytes**.
Strings, names, folders, messages and attachments belong to the layers above. LTP is the same in ANSI, Unicode and 4 KiB
files; the differences (block sizes, compression, encoding) are the node database's.

## Types

| Type | Spec name | What it is |
|---|---|---|
| `HeapNode` | HN (2.3.1) | A node's data read as heap blocks of numbered allocations. Resolves HIDs, owns the subnode tree, reads values. |
| `Hid`, `Hnid` | HID, HNID (2.3.1.1, 2.3.3.2) | 32-bit heap address; heap-or-subnode address. |
| `Bth`, `BthCursor` | BTH (2.3.2) | The tree: `TryFind` by key, a cursor over leaf records in key order. |
| `PropertyContext`, `PcProperty` | PC (2.3.3) | Properties by ID: `TryGet`, `foreach`, `ReadValue`. |
| `TableContext`, `TcColumn`, `TableRow` | TC (2.3.4) | Columns, a row index, rows copied into the caller's buffer, cells as references. |
| `LtpValueRef` | | Where a value's bytes are: `Empty`, `Inline` (up to eight bytes in the reference), `Heap` (an allocation) or `Subnode` (a subnode's data). Nothing is read until you ask. |
| `LtpValues` | | Fixed-size scalars (`TryGetInt16`, `Int32`, `Int64`, `Boolean`, `Single`, `Double`, `FileTime`, `Guid`) and multi-valued layouts (`TryGetMultiValueCount`, `TryGetMultiValueElement`). |
| `LtpPropertyTypes` | MS-OXCDATA 2.11.1 | The type codes, which sizes are fixed, the multi-value flag. |

Open a context from a node (`PropertyContext.TryOpen(file, nid, ...)`), a subnode entry, or a heap you already hold
(`HeapNode.TryOpenSubnode` gives the heap of a message's recipient table, for instance). Contexts opened from a file or
subnode own their heap and dispose it; a context made over your `HeapNode` leaves it to you. Opening a heap whose
`ClientSignature` is the wrong one for the context asks for is refused with `HeapClientSignatureMismatch`.

Value access is the same everywhere: `GetValueLength(ref)`, `ReadValue(ref, offset, buffer)` (a range, so large subnode values
stream), and `ReadValue(ref)` for a whole value as an array (null beyond `LtpOptions.MaxValueBytes`, default 256 MiB).

## How values are found

- **PC** (2.3.3.3): a BTH with 2-byte keys (property ID) and 6-byte entries (type, `dwValueHnid`). A fixed-size type of
  at most four bytes (Integer16, Integer32, Floating32, ErrorCode, Boolean) is in the entry itself and comes out as
  `Inline`. Everything else is an HNID: a zero HNID is `Empty`, an HID is `Heap`, a subnode NID is `Subnode`.
- **TC** (2.3.4.4): a cell of 1, 2, 4 or 8 bytes whose size equals the column type's fixed size is inline; a four-byte cell for
  any other type (strings, binary, objects, GUIDs, multi-valued) is an HNID, resolved as above. A cell that does not exist
  (its bit in the cell-existence bitmap is clear) gives no reference; the slot's bytes are not interpreted (2.3.4.4.3).
- **Rows** (2.3.4.4): the row matrix is one heap allocation (small tables) or the data of a subnode. In a subnode, rows never
  straddle a block, so row *N* is in block *N / rowsPerBlock* at *(N % rowsPerBlock) × rowSize*.
- **Multi-valued** (2.3.3.4): fixed-size elements are stored back to back; String8, String and Binary elements are a 4-byte
  count, one 4-byte offset per element from the start of the value, then the data.

## Reading never throws

`TryOpen` returns false and a typed list only when the structure is unusable. Everything else is an `LtpDefect` (kind, file
offset of the heap block, plain-language detail) in the heap's `Defects`, capped by `LtpOptions.MaxDefects` (1,000 per open heap;
the rest are counted in `DroppedDefects`). Problems in the blocks themselves (checksums, missing blocks, bad compression) are
`NdbDefect`s on the `NdbFile`; a block with a bad checksum is still returned by the node database layer, so LTP reads
through it and the two lists together say what was wrong.

| Group | Kinds |
|---|---|
| Nodes | `NodeNotFound`, `HeapUnreadable` (no data blocks) |
| Heap | `HeapBadHeader`, `HeapBadSignature` (bSig is not 0xEC: the node is not a heap), `HeapClientSignatureMismatch`, `HeapPageMapInvalid`, `HeapBlockMissing`, `HidInvalid` |
| Subnodes and values | `SubnodeMissing`, `SubnodesUnreadable`, `ValueUnreadable` |
| BTH | `BthBadHeader`, `BthTooDeep`, `BthNodeSize`, `BthKeyOrder`, `BthTooManyNodes` |
| PC | `PcBadShape` |
| TC | `TcBadInfo`, `TcBadLayout`, `TcRowIndexShape`, `TcRowOutOfRange`, `TcCellShape` |

`LtpValues` has no defect list: multi-valued data whose count or offsets do not fit returns false.

Lookups that fail return false, `-1` (a length) or null (a whole value) rather than a partial answer.

### Limits and hostile-input rules

| Rule | Value |
|---|---|
| BTH index levels | 8 (`LtpOptions.MaxBthLevels`); real trees have 1 to 3 |
| BTH nodes visited by one cursor | at most (heap length / 2) + 16: a node cannot be smaller than two bytes, so a well-formed tree cannot exceed this, and several parents pointing at one child cannot multiply the work |
| BTH key order | strictly increasing; a record that does not follow its predecessor is skipped with `BthKeyOrder` |
| `cbKey` / `cbEnt` | 2, 4, 8 or 16 / 1 to 32 (2.3.2.1); anything else is `BthBadHeader` |
| HID | type bits zero, index 1 to `cAlloc`, block inside the heap, allocation inside its block's data area (after the block header, before the page map) |
| Page map | `ibHnpm` leaves room for the map and the map fits the block; checked once per loaded block |
| Row access | row index and row size are checked against the matrix before any copy |
| Whole-value reads | `LtpOptions.MaxValueBytes` |

No allocation size is trusted before it is checked against the block it lives in; every read is a bounds-checked slice of the
decoded block. The layer has no `unsafe` code.

## Provisional resolutions

All are provisional defaults; none was contradicted by a real file.

1. **Rows per block of a subnode row matrix** (2.3.4.4 gives `floor((blockSize − trailer) / rowSize)` for 8 KiB blocks): taken
   from the first block's own size (`size / rowSize`), which equals the formula for 8 KiB blocks and adapts if 4 KiB files lay
   rows out in larger blocks. A single block holds all rows.
2. **BTH index keys** are not checked against the first key of the child they point to; lookups trust the key order and the
   level count.
3. **A BTH key that does not increase** is skipped, not returned, so a node referenced twice cannot produce duplicates.
4. **Allocation lower bound:** an allocation may not start before its block's header (12 bytes in block 0, 66 in blocks 8, 136,
   264... which carry a fill-level bitmap, 2 elsewhere). The bitmap-header blocks (2.3.1.4) were only exercised on files we
   wrote: no real file we have holds a heap of more than two blocks.
5. **Boolean and Integer16 in a PC** are inline (the entry's low one or two bytes); MS-PST states "up to four bytes".
6. **Unknown types** are read as HNIDs in a PC (a four-byte cell in a TC); a one-, two- or eight-byte cell of an unknown type is
   kept inline.
7. **`rgibAlloc` is zero-based while HID indexes are one-based:** HID index *i* is the allocation from `rgibAlloc[i − 1]` to
   `rgibAlloc[i]` (the page map's wording is ambiguous; every HID in the three real files resolves this way).
8. **Subnode lookup** compares the whole 32-bit NID, type bits included, as the HNID carries it.

## Real files

Checked on the NIST CFReDS Outlook 2013 OST (4 KiB pages, compressed blocks), Apache Tika's `testPST.pst` and
`testPST_variousBodyTypes.pst` (Unicode, 512-byte pages, permute-encoded), all fetched by `tools/corpus/fetch-pst.sh`.
`tools/oracles/libpff-ltp.sh <file>` reads every property and table context, then walks folders, items, recipients and
attachments the way a messaging layer would and compares with the libpff tools' export.

| File | PCs | TCs | Properties | Rows | Cells | LTP defects | NDB defects |
|---|---|---|---|---|---|---|---|
| CFReDS OST | 106 | 206 | 3,771 | 309 | 2,627 | 0 | 0 |
| `testPST.pst` | 21 | 38 | 998 | 25 | 360 | 0 | 0 |
| `testPST_variousBodyTypes.pst` | 14 | 34 | 733 | 15 | 252 | 0 | 0 |

(Nodes that are not heaps at all, such as search queues and internal lists, fail to open as `HeapBadSignature`,
`HeapBadHeader` or `HeapUnreadable`; the tool counts them separately and they are not defects of any LTP structure.)

Against `pffexport` (20180714) the folder hierarchy, items per folder, subjects, sizes, flags, three timestamps, sender,
conversation topic, recipients (name, address, type) and attachments (count and data size; an embedded message counts as one
with no size) agree for all three files: 48 / 6 / 6 folders, 14 / 7 / 4 items, 0 differences.

- The only systematic difference is a normalisation: a subject may begin with U+0001 and one more character (a prefix such as
  "RE: "); libpff drops both, the raw layer keeps them, the comparison strips them.
- `pffexport` reports no complaint in the live tree for any of the three files. All of its complaints (96, 40, 0) come from
  its final pass over *recovered* items, nodes found by scanning for old index entries. The twelve offsets it fails to read in
  the OST are the "unable to read data block" lines of that pass ("mismatch in data size"): five are not the start of any
  block in the file today, and seven hold blocks of a different size from the one it expected (three of them are live blocks
  that belong to other nodes now). The stale entries point at space that has been reused. Our reader never visits them,
  because nothing in the live node tree points there.

## Memory

On the CFReDS OST (release build, one thread): opening a property context allocates about 2.7 KB, opening a table context
2.6 KB, enumerating a context's properties 144 B; `TryGet`, `ReadValue` into a buffer, `TryReadRow`, reading every cell of a
row, and `TryFindRow` allocate nothing. Heap blocks are rented from the array pool and returned on dispose. Reading every
PC and TC in the file 200 times in a row keeps the managed heap at 339 to 355 KB after collection and the peak working set
at about 60 MB (of which the runtime is most), flat from the 75th pass on.


<!-- https://recursiveai.org/onemail/docs/pst-ndb/ -->

# PST/OST node database (NDB) layer

What `RecursiveAI.OneMail.Pst.Ndb` reads, how it treats damage, and what was settled by looking at real files. The structures
are MS-PST 2.2.2 (the NDB layer); where MS-PST is silent (4 KiB pages) the facts below come from a real Outlook 2013 OST and
from the libyal PFF format notes (GFDL 1.3: cited, nothing copied).

## Geometry

| | ANSI (wVer 14, 15) | Unicode (wVer 21, 23) | Unicode 4 KiB (wVer 36; 37 decided by the root page) |
|---|---|---|---|
| page | 512 | 512 | 4096 |
| page trailer | 12 | 16 | 24 |
| block trailer | 12 | 16 | 24 |
| block alignment | 64 | 64 | **512** |
| largest block data | 8192 − 12 | 8192 − 16 | 65536 − 24 |
| id size (NID/BID/IB) | 4 | 8 | 8 |
| B-tree footer at | 496 | 488 | 4056 (cEnt and cEntMax are 16 bits) |
| density list | 0x4200 | 0x4200 | 0x21000 |
| first AMap | 0x4400 | 0x4400 | 0x22000 |
| AMap bits | 496 bytes, one bit per 64 | 496 bytes, one bit per 64 | 4072 bytes, **one bit per 512** |
| first block | 0x4800 | 0x4800 | 0x23000 |

Observed on a real 4 KiB OST (a 16 MB Outlook 2013 offline store, wVer 36, `SO`, no encoding):

- Blocks are padded to 512 bytes and the 24-byte trailer sits at the end of the padded extent. A 56-byte block occupies 512.
- The trailer after `bid` carries two more 16-bit fields: a constant `2` (meaning not known) and the **uncompressed size**; 4 bytes follow.
- A block is compressed when its uncompressed size differs from the BBT's size. All 123 compressed blocks in the file are zlib streams (header `78 xx`), CRC computed over the stored (compressed) bytes. Internal blocks (XBLOCK, SLBLOCK) are never compressed. The reader accepts raw deflate too.
- The file has one AMap, the density list page (empty) and no PMap, FMap or FPMap pages. The AMap's free-byte total equals `cbAMapFree` in the header exactly when a bit stands for 512 bytes.
- The 4 KiB header root is laid out as the Unicode one (offsets 180..252).

Seen on real files since: Apache Tika's `testPST.pst` and `testPST_variousBodyTypes.pst` are Unicode files with 512-byte pages (wVer 23) and permute encoding (`bCryptMethod` 1). The first has nine AMap pages and two PMap pages; both open with no defect, scan mode finds every block the block B-tree lists, and the LTP layer reads every property and table context in them (`docs/pst-ltp.md`). Not yet seen on a real file: ANSI files (no public one was found), 4 KiB files with more than one AMap, and encoded 4 KiB files. Those paths are tested against files written from the specification by the test-side builder only.

## Sources

`NdbFile.TryOpen(IRandomAccessSource source, bool leaveOpen, ...)` opens a file over any source of the sources layer (a file, a stream, memory, or caller-supplied ranged reads); the path, stream and memory overloads are shortcuts for it. The node database reads with synchronous positioned reads, so a source that does not support them (`SupportsSyncRead` false) is refused with `ArgumentException`; short reads from a source are retried until the buffer is full or the source ends. Paths open with `FileShare.ReadWrite | FileShare.Delete`, so a file another program has open can still be read.

## Reading never throws

`NdbFile.TryOpen` returns false only when no usable header exists (too short, wrong magic, unknown version); everything else is a typed `NdbDefect` with the file offset. I/O errors (the file cannot be opened) throw `IOException` as usual.

Defects (`NdbDefectKind`), grouped:

- **Header:** FileTooShort, BadMagic, UnknownClientMagic, UnsupportedVersion, HeaderPartialCrcMismatch, HeaderFullCrcMismatch, BadSentinel, UnknownEncryption, ProtectedContent (bCryptMethod 0x10, contents not decodable), AMapFlagInvalid, FileSizeMismatch, RootOutOfRange, AmbiguousPageSize.
- **Pages:** PageOutOfRange, PageTruncated, PageTypeMismatch, PageTypeRepeatMismatch, PageSignatureMismatch, PageCrcMismatch, PageBidMismatch.
- **B-trees:** BTreeBadGeometry, BTreeLevelMismatch, BTreeTooDeep, BTreeKeyOrder (entry skipped), BTreeTooManyPages.
- **Blocks:** BlockOutOfRange, BlockTruncated, BlockTooLarge, BlockSizeMismatch, BlockSignatureMismatch, BlockCrcMismatch, BlockBidMismatch, BlockNotFound, BlockDecompressionFailed, AmbiguousCompressedEncodedBlock.
- **Data and subnode trees:** DataTreeBadHeader, DataTreeTooDeep, DataTreeTooLarge, DataTreeSizeMismatch, DataTreeChildMissing, SubnodeBadHeader, SubnodeTooLarge, SubnodeChildMissing.
- **Allocation:** AllocationPageInvalid, AMapFreeSpaceMismatch, DensityListInvalid.
- **DefectLimitReached:** the last slot of the list when `NdbOptions.MaxDefects` (100,000) is hit; later defects are only counted.

A block whose trailer checks fail is still returned (the defect says why), so a reader can show what it can; a block that cannot be read at all (outside the file, larger than a block can be, bad deflate data) yields nothing and a defect.

### Ambiguities (provisional resolutions)

- **Page size of wVer 37** (and any version above 36): decided by which trailer position repeats `ptype` on the NBT root page; if neither or both do, 512 with `AmbiguousPageSize`.
- **Compressed and encoded block together:** MS-PST does not say which comes first. The reader inflates, then decodes, and reports `AmbiguousCompressedEncodedBlock`. No real file has both.
- **ANSI allocation page CRC:** computed over the 496 map bytes (after 4 bytes of padding) by some writers and over the whole page body by others; both are accepted. Not yet observed on a real ANSI file.

## Resource limits (defaults, `NdbOptions`)

| Limit | Default | Why |
|---|---|---|
| defects kept | 100,000 | a damaged 50 GB file must not exhaust memory with its own complaints |
| leaf blocks per data tree | 4,194,304 | a data tree is at most 2 levels (XXBLOCK); this is 32 GiB of 8 KiB blocks |
| entries per subnode tree | 1,048,576 | |
| pages visited per B-tree walk | file length / page size + 8 | two parents pointing at one child cannot multiply the work |
| B-tree levels | 8 | MS-PST 2.2.2.7.7 |

Decompression is capped by the uncompressed size in the trailer (at most one block), so a compressed block cannot expand beyond 64 KiB.

## Scan mode

`NdbFile.Scan` finds pages and blocks without using either B-tree: a page is a page-sized unit whose last bytes are a valid page trailer (ptype 0x80..0x86 repeated, signature, CRC); a block is an extent ending, on a block-alignment boundary, in a trailer with a plausible size, a signature that matches its own offset and BID, and a matching CRC. One forward pass over a 1 MiB window; memory does not grow with the file. `IncludeDamaged` also reports structures that fail only one of signature or CRC. Results carry the offset, size, BID and which checks passed; this is the substrate for recovery.

On the real OST: all 381 BBT blocks and both root pages are found, plus 101 further clean blocks that no tree references (stale versions Outlook left behind).

## Comparison with the libpff tools (20180714, black box)

`tools/oracles/libpff-ndb.sh <file>` prints what `pffinfo` and `pffexport` say next to `PstOracle dump` for the same file. On the 4 KiB OST: both report `64-bit with 4k page`, `Offline Storage Tables (OST)`, no encryption, file size 16,818,176. pffexport builds the folder tree and exports messages, but logs "unable to read table" for 12 items in this file (its 2018 build and 4 KiB stores); our reader finds no defect anywhere in the file. Item-level agreement belongs to the LTP/Messaging layers.


<!-- https://recursiveai.org/onemail/docs/mapi/ -->

# The MAPI property model

What `RecursiveAI.OneMail.Mapi` is: the in-memory form of an Outlook item (message, recipient, attachment), independent of
the file it came from. MSG files fill it today (`docs/msg.md`); the PST/OST messaging layer fills the same types (`docs/pst-messaging.md`). Types and tags are
MS-OXCDATA 2.11.1 and MS-OXPROPS; nothing here is taken from another implementation.

## Objects

- `MapiMessage`, `MapiRecipient`, `MapiAttachment` derive from `MapiObject`, which owns a `MapiPropertyBag`, a list of
  `MapiDefect`s and a link to its parent.
- Recipients and attachments load on first use. An attachment that is an embedded message exposes it as `EmbeddedMessage`
  (a `MapiMessage` of its own); other object attachments expose `ObjectStorage`, a `MapiRawStorage` that keeps the storage's
  streams byte for byte so nothing is lost.
- `MapiMessage.Subject`, `Body`, `MessageClass`, recipient and attachment accessors are conveniences over `Properties`; the
  bag is the truth.

## The property bag

- A `MapiProperty` is `(tag, flags, value)`. The tag is a 16-bit ID plus a 16-bit `MapiType`; IDs 0x8000 and up are named
  properties and carry their `MapiNamedProperty` (property-set GUID plus a number or a string).
- **The raw value is always kept**: the bytes of the entry or value stream, read lazily through a `MapiValueSource`. Typed
  accessors (`AsInt32`, `AsDouble`, `AsBoolean`, `AsFileTime`, `AsGuid`, `AsString(encoding)`, `CurrencyUnits` ...) decode on
  demand and never change what is stored. Round-tripping a property never goes through a typed value.
- The bag keeps insertion order and duplicates. Lookups by ID (optionally restricted to types) return the **last** entry with
  that tag (`AmbiguousDuplicateProperty`). `Version` increments on change so callers can cache.
- A string property is `String8` (code page, see `docs/msg.md`) or `UnicodeString` (UTF-16LE). Asking for a string by ID
  accepts either type.

## Names

`tools/mapi/properties.tsv` lists the properties the readers use (271 tagged, 75 named), each with its canonical MS-OXPROPS
name. `tools/mapi/gen_properties.py` turns it into the generated `MapiPid.g.cs` constants and the `MapiPropertyNames` lookup.
The table is a static array: no reflection, so trimming and AOT keep it. Edit the TSV, never the generated file.

## Defects

`MapiDefect` has a `Kind`, a message and a location (object path plus stream name). Kinds named `Ambiguous*` are documented
resolutions of known ambiguities (`docs/ambiguities.md`, section MAPI/MSG); the rest are repairs. The reader never throws on
damaged content. Defect lists are capped by `MsgReaderOptions.MaxDefects`; later ones are only counted.


<!-- https://recursiveai.org/onemail/docs/tnef/ -->

# TNEF (winmail.dat)

Normative notes for `RecursiveAI.OneMail.Tnef`, written from MS-OXTNEF (v20250520) and checked against public files. Layers:

| Type | Layer |
|---|---|
| `TnefDocument`, `TnefAttributeRecord`, `TnefWriter` | Attribute stream: signature, legacy key, then attributes (level, id, length, data, checksum). Lazy data. |
| `TnefPropertyReader` | A property list (attMsgProps, attAttachment, recipient rows): types, named properties, multi-valued values. |
| `TnefMessage`, `TnefAttachment` | A `MapiMessage` (same shape as MSG and PST items), with embedded messages as lazy slices. |
| `TnefMessageWriter` | `MapiMessage` -> TNEF. Returns warnings naming each property the format cannot carry. |

`MimeEntity.EnumerateTnefParts()` finds `application/*` parts and `winmail.dat`/`*.tnef` names; `IsTnef`/`ReadTnef` open one.
`Mail.Open` opens a bare TNEF file as one item.

## Fidelity

- **Attribute layer is byte-identical.** `TnefDocument.WriteTo` reproduces any input: wrong checksums, unknown attributes, a truncated last
  attribute and trailing bytes are kept as they were.
- **MAPI layer is property-exact.** Tag, type, named-property GUID and name, and raw value survive write then read. Properties TNEF cannot
  carry (server IDs, named properties without a name) are reported by name in the writer's warnings rather than dropped silently.
- Legacy attributes (subject, dates, body, recipients, sender, priority, ...) become properties per MS-OXTNEF 2.1.3.3. The writer emits
  the legacy attributes old readers look for, but only for text the code page can hold; the rest stays in the Unicode property.

## Defects

Typed, in `TnefMessage.TnefDefects` (kind, byte offset in the stream concerned, detail) and mirrored as `LowerLayerDefect` in the MAPI
defect list. Parsers never throw on content. Kinds named `Ambiguous*` are the documented resolutions below; the others are repairs
(`AttributeTruncated`, `TrailingBytes`, `InvalidLevel`, `ChecksumMismatch`, `LimitReached`, `UnexpectedVersion`, `MissingCodePage`, ...).

## Limits (defaults, on)

| Limit | Default |
|---|---|
| attribute data | 1 GiB |
| attributes in a stream | 1,000,000 |
| attachments, recipients | 65,536 each |
| properties in a list, values in a multi-valued property | 1,000,000 |
| embedding depth | 16 |
| announced decompressed RTF size | 256 MiB (header check only; RTF stays compressed) |
| values read eagerly | 64 KiB; larger ones stay lazy |

Reaching a limit raises `LimitReached` and stops that part. The public file `oom.tnef` (a property count of 873,267,203) opens in
about 3 KB of allocations with `ChecksumMismatch`, `LimitReached`, `MissingCodePage` and `UnexpectedVersion`.

## Code pages

8-bit strings are read with attOemCodepage when present and nonzero (MS-OXTNEF 2.3.3.2, rule 2), reported as
`MapiCodePageSource.Declared`; otherwise the shared model's rules apply. Rule 1 (a MIME charset on the enclosing part) is not applied:
`MimeEntity.ReadTnef` does not pass it down. The writer uses the message's resolved code page, falling back to windows-1252, writes it as
attOemCodepage and reads back to the same message with no extra property.

Ambiguity entries are in `ambiguities.md` (T1-T8).

## TNEF inside MIME

`MimeEntity.EnumerateTnefParts()` and `ReadTnef` read the part on their own. `MimeToMapiConverter` merges a qualifying part into the
`MapiMessage` (MS-OXCMAIL 2.2.3.8; the rules and loss codes are in [`conversion.md`](https://recursiveai.org/onemail/docs/conversion/)) and `MimeMailItem` shows its attachments and
bodies in place of winmail.dat ([`shared-view.md`](https://recursiveai.org/onemail/docs/shared-view/), V9). A part qualifies when the stream has none of `NotTnef`,
`UnexpectedVersion`, `AttributeTruncated`, `ChecksumMismatch` or `LimitReached`, and its `PidTagTnefCorrelationKey` matches the message's `X-MS-TNEF-Correlator`. Of the six corpus
files, only `oom.tnef` (damaged) is not unpacked. `bug52400-winmail-simple.dat` and `bug52400-winmail-with-attachments.dat` end in CR LF outside the
attributes; MS-OXCMAIL 2.2.3.8 says to ignore such a stream but all four oracles unpack them, so they are unpacked and reported (`AmbiguousTnefTrailingCrLf`, OX20).
`MapiToMimeConverter` writes a TNEF message on request (`OxcMailExportOptions.Tnef`).


<!-- https://recursiveai.org/onemail/docs/rtf/ -->

# RTF bodies: compressed RTF and de-encapsulation

`RecursiveAI.OneMail.Rtf` reads and writes what Outlook stores in `PidTagRtfCompressed` and recovers the HTML or plain text
that Outlook wraps inside an RTF body. The namespace uses only the .NET base class library: it does not reference any other
part of OneMail, so it can be lifted out as the seed of a general RTF reader. Nothing in it throws on malformed input: damage
is a typed `RtfDefect` with a byte offset, and the ambiguity entries (RT1–RT33) are in [ambiguities.md](https://recursiveai.org/onemail/docs/ambiguities/).

| Piece | Type | Spec |
|---|---|---|
| Tokenizer | `RtfReader` (`ref struct`) | RTF 1.9.1, lexical rules |
| Decompress | `Compression.RtfDecompressor` | MS-OXRTFCP 2.2 |
| Compress | `Compression.RtfCompressor` | MS-OXRTFCP 2.3 |
| HTML/text out of RTF | `RtfDeencapsulator` | MS-OXRTFEX 2.2 |
| HTML/text into RTF | `RtfEncapsulator` | MS-OXRTFEX 2.1 |

## Compressed RTF (MS-OXRTFCP)

The stream is a 16-byte header (COMPSIZE, RAWSIZE, COMPTYPE, CRC, little-endian) and contents. `LZFu` contents are
LZ77 over a 4096-byte circular dictionary that starts with 207 bytes of common RTF text; a control byte tells which of the next
eight tokens are literals (1 byte) or references (2 bytes, big-endian: 12-bit dictionary offset, 4-bit length minus two). A
reference to the dictionary's write position ends the data. `MELA` contents are the RTF itself. The CRC is the reflected
CRC-32 polynomial (0xEDB88320) with a zero start value and no final inversion, over the contents (padding included).

**Decompression** (`RtfDecompressor.Decompress`) checks, in this order, and reports each failure without stopping unless it
cannot continue:

| Check | Defect | Output |
|---|---|---|
| fewer than 16 bytes | `HeaderTruncated` | empty |
| type neither `LZFu` nor `MELA` | `UnknownCompressionType` | empty |
| COMPSIZE + 4 differs from the bytes present | `CompressedSizeMismatch` | the declared range, or what is there |
| RAWSIZE above `MaxOutputBytes` (256 MiB) | `OutputLimitExceeded` | empty |
| RAWSIZE above 20 × the input length | `ExpansionRatioExceeded` | empty |
| data ends before the end marker | `MissingEndMarker` | everything decoded |
| reference to a dictionary position nothing was written to | `ReferenceBeyondDictionary` | zero bytes used |
| CRC differs | `CrcMismatch` | everything decoded |
| bytes produced differ from RAWSIZE | `RawSizeMismatch` | everything decoded (the buffer grows up to the limit) |

The size and ratio checks run before the output buffer is allocated. Honest data cannot expand more than 17 bytes per 2-byte
reference (8.5 : 1), so the ratio limit of 20 never refuses real data. `MELA` copies the contents and does not validate the CRC
(MS-OXRTFCP 2.2.3.1). `RtfDecompressionResult.CrcVerified` is true only for `LZFu` data whose CRC matched.

**Compression** (`RtfCompressor.Compress`) searches the 4095 bytes before the cursor through a hash chain of two-byte pairs
(at most 128 candidates per position), takes the longest match of 2 to 17 bytes (the older one on a tie, as the specification's
oldest-first scan does), and writes the end marker. Matches may overlap the bytes being written, which is what the
decoder's byte-at-a-time copy expects. Output decompresses back to the input exactly with `RtfDecompressor` and with the
oracle; it reproduces both worked examples of the specification byte for byte. It is not required to equal Outlook's own bytes
(see Measurements). An empty input becomes `01 0C F0` (an end marker only); Outlook's writer adds a NUL byte there, which
the reader treats as one byte of output (RT18).

## The tokenizer (`RtfReader`)

A forward-only reader over a `ReadOnlySpan<byte>`: `GroupStart`, `GroupEnd`, `ControlWord` (name, optional signed number,
one optional delimiting space that belongs to the word), `ControlSymbol` (`\*`, `\~`, `\{` …), `HexEscape` (`\'hh`), `Text` (a
run of bytes), and `Binary` (the `N` bytes after `\binN`, not parsed). CR and LF between tokens are skipped, as RTF requires.
A backslash before a line break is a control symbol that readers treat as `\par`. It never interprets code pages,
destinations or Unicode: those are the caller's.

Defects: `UnbalancedGroupEnd`, `UnclosedGroups`, `ParameterOverflow` (clamped to `int.MaxValue`), `InvalidHexEscape`,
`BinaryDataTruncated`, `TruncatedToken`, `NestingLimitExceeded` (default depth 1000, then reading stops).

## De-encapsulation (MS-OXRTFEX 2.2)

`Detect` follows 2.2.3.1: the document starts `{\rtf`, and among its first ten tokens (all group starts or control words)
`\fromhtml1` or `\fromtext` appears. `\fromhtml`, `\fromhtml0` and any other form do not count (RT12).
`Deencapsulate` returns the text as a .NET string, with the document's default code page. All bytes are decoded, so there is
no byte-oriented result to mis-label.

HTML (`\fromhtml1`):

- `{\*\htmltag…}` group content is copied, regardless of `\htmlrtf` (RT6); the numeric parameter is ignored. `\*\mhtmltag` and every
  other ignorable destination are skipped.
- Outside `\htmltag`, text and control words under `\htmlrtf` (or `\htmlrtf1`) are dropped until `\htmlrtf0`. The state is
  saved on `{` and restored on `}`. The current font is tracked even in suppressed text.
- `\par` and `\line` become CRLF, `\tab` a tab. Other control words are ignored.
- Destinations that hold no body text are skipped (`\colortbl`, `\stylesheet`, `\info`, `\pict`, `\object`, headers and footers,
  `\fldinst`, list and paragraph-number groups, bookmarks…); `\fonttbl` is read for code pages only (RT13).

Plain text (`\fromtext`): the same walk with no `\htmltag` groups; `\htmlrtf` is ignored (RT10).

**Characters.** `\uN` is a signed 16-bit UTF-16 unit; the next `\ucN` characters (default 1) are its fallback and are skipped.
A skipped character is a text byte or a `\'hh` escape; control words do not count, and `{` or `}` ends the skipping (RT8). `\uc`
is scoped to its group. An unpaired surrogate becomes U+FFFD (`LoneSurrogate`). `\'hh` and text bytes are decoded as runs in the code page of the
current font: `\cpgN` of the font, else the charset of `\fcharsetN` (0 is windows-1252 whatever `\ansicpg` says; 1 means the
document's code page), else `\ansicpgN` (or what `\ansi`, `\mac`, `\pc`, `\pca` imply, else 1252) (RT9). `\htmltag` content always
uses the document's default code page. Runs are decoded together so that two-byte characters split across `\'hh\'hh` come out
whole. Code pages come from `RtfDeencapsulationOptions.CodePageDecoder`, or by default from the base library's encodings and
`CodePagesEncodingProvider` (no global registration); an unknown code page uses the default one and reports
`AmbiguousUnknownCodePage` (RT11).

Control words and symbols with a character meaning: `\lquote \rquote \ldblquote \rdblquote \bullet \endash \emdash \enspace
\emspace \qmspace \zwj \zwnj \ltrmark \rtlmark`, `\~` (U+00A0), and `\_` (U+00AD inside `\htmltag`, U+2011 elsewhere) (RT5, RT7).
`\{ \} \\` are the literal characters.

## Encapsulation (MS-OXRTFEX 2.1)

`EncapsulateHtml` writes `{\rtf1\ansi\ansicpg1252\fromhtml1 \deff0{\fonttbl…}`, the empty `{\*\htmltag64}`, then walks the
HTML: markup (tags, comments, doctype, and `<script>`/`<style>` with their content) goes into `{\*\htmltagN …}` with the
TagType flag as parameter (plus 8 for closing tags); entities go into `\htmltag0` with their rendering after `\htmlrtf`; line
breaks in text become `\htmltag0 \par` plus a suppressed space; everything else is visible RTF text. The output is 7-bit ASCII
(`\uN?` for non-ASCII, `\'hh` for control characters). `EncapsulateText` writes `\fromtext` and the text with `\par` for every
line break. This is not an HTML-to-RTF converter: no formatting is produced.

Line breaks (CR, LF or CRLF) come back as CRLF, which MS-OXRTFEX 1.3.1 allows; everything else round-trips exactly.

## Measurements and method

`tools/rtf` extracts the RTF, HTML and text body streams from the public Tika and POI `.msg` files (OneArchive's compound-file
reader; `tools/corpus/fetch-msg.sh` fetches the files), runs OneMail over them, and compares with released packages run as
black boxes. See [tools/rtf/README.md](https://github.com/recursive-ai-oss/onemail-csharp/blob/bb4c9543917ab4ad466b5200d132e1899cb3ca14/tools/rtf/README.md) for the commands and the oracle versions.

On 55 `.msg` files (53 read; two fuzzer-minimised files have a damaged compound file) there are 45 compressed RTF bodies, 27
distinct streams:

| Check | Result |
|---|---|
| decompression vs compressed-rtf 1.0.7 | 45 / 45 byte-identical; header CRC verified on 45 / 45; no defects |
| our compression read back by OneMail and by compressed-rtf | 45 / 45 identical to the input |
| compressed-rtf's compression read back by OneMail | 45 / 45 identical, CRC verified |
| our compressed size vs Outlook's own stream | 97,438 vs 97,422 bytes over the 27 distinct streams (1.0002 ×); 3 of 45 streams byte-identical, the rest differ only in which of several equal-length matches is used |
| de-encapsulation vs RTFDE 0.1.2.2 | 31 HTML and 4 text results: identical apart from RT1 (all) and RT2 (4 files) |
| de-encapsulation vs the message's own `PidTagHtml` / `PidTagBody` | 4 text bodies identical; 6 HTML bodies: 5 differ only by an injected content-type `<meta>` (RT3), 1 is the same text with re-serialised markup (RT4) |
| encapsulate, then de-encapsulate, every recovered and own HTML and text body | 98 bodies: 97 exact, 1 equal once its LF line endings became CRLF, 0 different |
| 10 RTF bodies without `\fromhtml1`/`\fromtext` | `NotEncapsulated`; RTFDE refuses them too |
| 3,000 damaged streams (truncations, bit flips, header edits, insertions) | OneMail reports a defect on 2,998; the 2 without are valid; oracle refuses 2,616 of them; bytes equal wherever both decode, except 123 where OneMail's size or ratio limit refused what the oracle decoded (RT14) |

Corpus bodies are not committed; the specification's example blocks are in `testdata/rtf` and the tests.

## Native RTF to HTML (`RtfHtmlConverter`)

Outlook items whose RTF was authored as RTF (no `\fromhtml`/`\fromtext`) have no other body. `RtfHtmlConverter.Convert(bytes, options)`
turns them into HTML and plain text; the MAPI view uses it when an item has no HTML or text body (`MailBody.Origin` is
`RtfConverted`). The same independence rule holds: the namespace uses only the BCL (a test scans the sources).

- **Model.** A parse pass builds a small document model (paragraphs, runs, lists, tables, pictures, links) from fonts, colours,
  the stylesheet (names only, for headings), list tables and `\pn` lists, fields (HYPERLINK), shapes, `\object` results,
  `\objattph` positions, `\u`/`\uc` and code pages (RT19). Both writers read that model, so the text always matches the HTML.
- **HTML shape.** One `<div class="rtf-body">` fragment with the dominant font and size on the div; `p`, `h1`–`h6`, `ul`/`ol`/`li`,
  `table`, `a`, `img` and a few inline spans. Images are `data:` URIs by default, `cid:` with `RtfImageMode.ContentId`. EMF/WMF become
  an attachment-style placeholder. The output is well-formed and identical for identical input.
- **Text.** CRLF line breaks; list markers and tab-separated cells.
- **Limits** (`RtfHtmlOptions`): group depth 1000, 4 M nodes, 64 Mi output characters, 16 MiB per picture, 64 MiB of pictures; a
  hit is a typed defect and the output so far is kept. No input throws.
- **Defects added:** `InvalidPictureData`, `PictureLimitExceeded`, `UndefinedReference`, `NestedTableFlattened`,
  `AmbiguousPictureFormat`, `AmbiguousListLevel`, `AmbiguousSymbolFont`, `ContentLimitExceeded`, `AmbiguousBareCell`.
- **Resolutions:** RT19–RT33 in [ambiguities.md](https://recursiveai.org/onemail/docs/ambiguities/). RT19 differs from RT9 on purpose: the de-encapsulator reads RTF that
  Outlook wrote around HTML (the specification's own example has `\fcharset0` text under `\ansicpg1251`, decoded as 1252), while the
  converter reads RTF authored by Word and others, where `\fcharset0` means "ANSI" and the document's `\ansicpg` is the code page
  (LibreOffice 26.2.6 agrees; 24.2.7, pandoc and unrtf decode as 1252). No client was observed on either; both are provisional.

### Method and results

`tools/rtf/run-html-oracles.sh` converts every document with pandoc 3.13, unrtf 0.21.10 and LibreOffice 24.2.7 (`--convert-to
html`), all run as released programs, and `htmlcompare.py` reduces every output to a normalised model: words, which words are
bold/italic/underlined/struck/super-/subscript, list items, table cells, link targets, heading levels, image count. HTML bytes are
never compared. A metric an oracle never produces in any document (unrtf: lists, headings) is n/a for it. Every remaining
difference needs a rule in `tools/rtf/explained-html.tsv`; LibreOffice rows are itemised per document, pandoc and unrtf rows are
**class-level** (they decode every `\'hh` as 1252, drop heading styles and lists, and so on), restricted to the generated, Tika and
POI documents, so a difference on any other document would show up as unexplained.

123 documents: 17 native RTF bodies from the MSG/OST/PST/TNEF corpora (of 52 RTF bodies in 85 items; 35 are encapsulated), 43 Tika/POI
`.rtf` test documents, and 63 generated (23 feature documents, 40 seeded mixes).

| Oracle | Metrics agreeing | Explained | Unexplained | n/a |
|---|---|---|---|---|
| LibreOffice 24.2.7 | 1,446 | 18 | 0 | 0 |
| pandoc 3.13 | 1,327 | 113 | 0 | 0 |
| unrtf 0.21.10 | 1,105 | 125 | 0 | 246 |

On the 17 native bodies LibreOffice agrees on every metric; pandoc and unrtf differ only on two Chinese bodies (lists, tables, code
page). Text content of every native body agrees with LibreOffice.

Allocations (Release): over 83 documents (2,282 KB of RTF) 1,559 B per KB of RTF for HTML and 1,891 B/KB for HTML plus text; documents
without pictures (662 KB): 2,990 and 3,978 B/KB. Tests include truncation at every length and 3,000 seeded mutants; the
`rtf-html` fuzz target checks that the HTML is well-formed and deterministic.


<!-- https://recursiveai.org/onemail/docs/olm/ -->

# Outlook for Mac archives (OLM)

`RecursiveAI.OneMail.Stores.Olm` reads `.olm` files, the archives Outlook for Mac exports. Reading only: OneMail never writes
an OLM. This page is the normative description of how a record becomes an item, what is reported and what is left alone.

## What an OLM is

A zip file of XML records and attachment files, read through OneArchive's zip reader (OneMail has no zip code of its own).
The layout in both public sample archives (Outlook 16):

| Entry | Holds |
|---|---|
| `Local/com.microsoft.__Messages/<folder>/message_NNNNN.xml` | one `<emails><email>…</email></emails>` record per file; folders nest to any depth |
| `Local/com.microsoft.__Messages/<folder>/com.microsoft.__Attachments/<id>_NNNN` | the bytes of one attachment, named by the record's `OPFAttachmentURL` |
| `Local/Calendar/Calendar.xml`, `Local/Address Book/Contacts.xml`, `Groups.xml`, `Local/Tasks/Tasks.xml`, `Local/Notes/Notes.xml` | all records of one kind in one file |
| `Categories.xml` | category names and colours |
| `Accounts/…` | empty folders in the samples |

Attachments sit in the zip before or after their message; the reader does not depend on the order. Every XML value is an
attribute or the text of a leaf element; whole numbers and flags are written as `1E0`, `0E0`, `5.0614E4`.

What a file is comes from its content, never its name: a zip with an entry under `Local/` in one of the five trees above is an
OLM (`OlmArchive.IsOlm`, and `MailFormatSniffer.Sniff` returns `MailFormat.Olm`); an XML entry is a record file when its root
element is one of `emails`, `appointments`, `contacts`, `tasks`, `notes`, `groups` or `categories`. A different root is the
`UnrecognisedRoot` defect. Only the Outlook 16 layout has been seen; the reader does not depend on the path beyond the folder
names above, so a layout that keeps the record element names reads the same.

## API

| Call | Does |
|---|---|
| `OlmArchive.Open(path or source, options)` | opens the zip directory; throws `InvalidDataException` when no entry can be listed |
| `OlmArchive.Enumerate()` / `EnumerateAsync()` | the records in zip order, one record built at a time |
| `OlmArchive.GetFolders()` | the folder tree: message folders and the per-kind folders |
| `OlmArchive.Defects` | defects of the archive and of the entries read so far |
| `OlmItem` | a record: the shared view (`IMailItem`), the whole XML tree (`Element`), `Folder`, `MimeSource`, `Categories`, `UnmappedElements()`, `OlmDefects` |
| `OlmToCalendarConverter`, `OlmToVCardConverter` | appointment, task and note to iCalendar; contact and group to vCard; each returns a loss report |
| `Mail.Open("x.olm")` | the same records as `IMailItem`s through `MailContainer`; `onemail inspect x.olm` lists them |

An item and its attachment streams are valid until the archive is disposed. One archive is not thread-safe.

## The shared view

| View member | Source |
|---|---|
| `Kind` | email: message; appointment; contact; task; note; group: distribution list; category: other |
| `Subject` | `OPFMessageCopySubject`; appointment `OPFCalendarEventCopySummary`; contact `OPFContactCopyDisplayName` (else the name parts); task `OPFTaskCopyName`; note `OPFNoteCopyTitle`; group `OPFGroupCopyDisplayName`; category `OPFCategoryCopyName` |
| `From`, `ToRecipients`, `Cc`, `Bcc`, `ReplyTo`, `Sender` | `OPFMessageCopyFromAddresses`, `…ToAddresses`, `…CCAddresses`, `…BCCAddresses`, `…ReplyToAddresses`, `…SenderAddress`: `emailAddress` elements with `OPFContactEmailAddressAddress` and `…Name`. An appointment's attendees are its `ToRecipients`. |
| `Sent`, `TryGetDate` | `OPFMessageCopySentTime` is the client-submit time, `…ReceivedTime` the delivery time; a note's `OPFNoteCopyCreationDate` is its creation time. Dates are UTC with `ZoneKnown` false (O1). |
| `MessageId`, `InReplyTo`, `References` | `OPFMessageCopyMessageID`, `…InReplyTo`, `…References`, without angle brackets |
| `ConversationIndex`, `ConversationTopic` | `OPFMessageCopyThreadIndex` (base64), `…ThreadTopic` |
| `TextBody`, `HtmlBody` | email: the body element, as HTML when `OPFMessageGetHasHTML` is set, else as plain text (O2); appointment, contact and task: the `…Plain` element and the HTML element; note: its text is HTML. Bodies are the XML's Unicode text presented as UTF-8; the original character set is not in the file. |
| `Attachments` | the `messageAttachment` elements: `OPFAttachmentName`, `…ContentType`, `…ContentID`; the stream is the zip entry named by `OPFAttachmentURL`; size is the entry's size (O4); an attachment whose entry is missing is listed with no content (O9) |
| `Provenance` | format Olm, the file, the folder, the zip entry's local header offset and size, the item's index |
| `MimeSource` | `OPFMessageCopySource` when the record has one, as the XML text decodes. Neither sample archive has one; the element is in the published schema. |

## Calendar and vCard mappings

`OlmToCalendarConverter`: appointment to `VEVENT`, task to `VTODO`, note to `VJOURNAL`; `OlmToVCardConverter`: contact to a vCard 3.0,
group to a vCard 4.0 with `KIND:group`. Both return a loss report (`docs/conversion.md`) with one `Dropped` entry for every top-level
element they do not read (code `Unmapped`) and an `Approximated` entry wherever a value is written in a weaker form.

| OLM | iCalendar / vCard |
|---|---|
| appointment `UUID`, `Summary`, `Location`, `DescriptionPlain`, `Description` | `UID`, `SUMMARY`, `LOCATION`, `DESCRIPTION`, `X-ALT-DESC;FMTTYPE=text/html` |
| `StartTime`, `EndTime` (wall-clock) with `GetStartTimeZoneICSData` | `DTSTART`/`DTEND` with `TZID`, and the record's `VTIMEZONE` copied in; no zone data: floating times (`FloatingTime`); `GetIsAllDayEvent`: `VALUE=DATE` with the exclusive end (`AllDayEnd`); a different end zone is not kept (`EndZoneIgnored`) |
| `Recurrence` / `RecurrencePattern` Daily, Weekly (`DaysOfWeek`), AbsoluteMonthly, AbsoluteYearly | `RRULE` `FREQ`, `INTERVAL`, `BYDAY`, `BYMONTHDAY`, `BYMONTH`; `COUNT` when the record is numbered, else `UNTIL` as the UTC end date (O8); RelativeMonthly and RelativeYearly are mapped by name with 5 meaning last (`RelativeRecurrence`); another type is dropped (`RecurrenceKind`) |
| `HasReminder` with `ReminderDelta` (seconds) | a `VALARM` display alarm with a negative `TRIGGER` |
| `IsPrivate`, `Organizer`, attendees, category list | `CLASS:PRIVATE`, `ORGANIZER` (when it is an address), `ATTENDEE` with `CN` (status, type and response flags are not mapped: `AttendeeStatus`), `CATEGORIES` |
| task name, notes, start, due, completed, reminder, recurrence | `SUMMARY`, `DESCRIPTION`, `X-ALT-DESC`, `DTSTART`, `DUE`, `COMPLETED` with `STATUS:COMPLETED`, `VALARM`, `RRULE`; task dates are written as the UTC instants they hold |
| note title, creation date, text | `SUMMARY`, `DTSTART`, `X-ALT-DESC` (the text is HTML; there is no plain text) |
| contact names, nickname, company and department, business title | `FN`, `N`, `NICKNAME`, `ORG`, `TITLE` |
| contact phones (business 1 and 2, home 1 and 2, fax, cell, pager, other, assistant) | `TEL` with `TYPE` |
| contact e-mail list (the numbered repeats are not written twice) | `EMAIL` with `TYPE` work, home or other (O7) and `pref` for the default |
| contact home, business and other address | `ADR;TYPE=home`, `work`, `postal` |
| contact web pages, birthday, anniversary, spouse, notes, categories, modification time | `URL`, `BDAY` and `X-ANNIVERSARY` (the UTC date, `DateOnlyZone`), `X-SPOUSE`, `NOTE`, `CATEGORIES`, `REV` |
| group name, members | `FN`, `KIND:group`, one `MEMBER:mailto:` per member (the member's name is dropped: `MemberName`) |

### What is not mapped

Nothing is lost: every element stays in `OlmItem.Element`. Counts of top-level elements neither the shared view nor a conversion
reads, from `OlmTool stats` over the two samples:

| Kind | Items | Distinct unmapped element names | Unmapped elements (total) | Typical members |
|---|---|---|---|---|
| email | 7 + 483 | 17 and 14 | 101 and 6,762 | read flag, priority, flag status, modification date, inference class, read-receipt flags |
| appointment | 8 | 12 | 80 | free/busy, travel time, accept status, dismiss and original reminder times, time-zone defaults |
| contact | 5 + 1 | 34 and 6 | 60 and 6 | custom fields, astrology sign, blood type, children, IM addresses, radio phone, image flag, edit flags |
| group | 2 | 6 | 8 | flag status, suppress-names flag, primary category |
| task | 5 | 2 | 6 | priority, primary category |
| note, category | 1, 18 | 0 | 0 | |

## Limits

All are on by default (`OlmReaderOptions`); a limit that is hit is a typed defect and the rest of the archive is still read.

| Limit | Default | What happens |
|---|---|---|
| `MaxXmlBytes` (decompressed bytes of one XML entry) | 1 GiB | `XmlTooLarge`; the records before the cut are kept |
| `MaxXmlDepth` | 64 | `XmlTooDeep`; that record is dropped |
| `MaxElementsPerItem` | 1,000,000 | `XmlTooManyElements`; that record is dropped |
| `MaxTextLength` (one text run or attribute) | 256 Mi characters | `XmlTooLarge`; that record is dropped |
| `MaxExpansionRatio` (decompressed per compressed byte, from OneArchive) | 1000, after 32 MiB | `ArchiveLimitExceeded` or `EntryUnreadable` |
| `MaxEntryCount`, `MaxTotalSize`, `MaxCentralDirectorySize` | 50 million, 1 TiB, 4 GiB | `ArchiveLimitExceeded`; entries listed before the limit are kept |

XML is read with DTDs ignored and no resolver, so an entity bomb has nothing to expand; invalid characters are read, not refused.

## Defects

`XmlMalformed`, `XmlTooLarge`, `XmlTooDeep`, `XmlTooManyElements`, `UnrecognisedRoot`, `ElementCountMismatch` (the container's
`elementCount` is not what was found), `EntryUnreadable` (a damaged, encrypted or unsupported entry, or a failed checksum),
`ArchiveLimitExceeded`, `ArchiveUnreadable`, `AttachmentMissing`, `BadDate`, `BadNumber`, `AddressMissing`, `UnrecognisedEntry`
(a link or device entry), and the ambiguity defects `AmbiguousDateZone` and `AmbiguousBodyKind`. The ambiguity rows are O1 to O9
in [`ambiguities.md`](https://recursiveai.org/onemail/docs/ambiguities/). Through the shared view they surface as `MailDefect` with family `olm`.

## Memory

Records, attachments and bodies stream: one record is built at a time and attachments are lazy streams over zip entries, so
the working set does not depend on the size of the data. It does depend on the **number of zip entries**, because OneArchive's
zip reader keeps the whole central directory in memory (about 1 KB per entry) and OneMail adds a table of attachment entries
and a list of record entries on top (about 0.3 KB per entry). Measured on the worker (AMD Ryzen 9 7950X3D, .NET 10.0.402,
Release, workstation GC), generated archives of stored entries (`tools/olm/memory.sh`), every field of every item read and every
attachment streamed:

| Archive | Entries | Zip reader alone, peak working set | OneMail, peak working set | Allocated per item |
|---|---|---|---|---|
| 20,000 messages, 191 MiB | 26,678 | 62 MiB | 123 MiB | 84 KB |
| 80,000 messages, 762 MiB | about 107,000 | not measured | 197 MiB | 84 KB |
| 160,000 messages, 1,524 MiB | 213,345 | 242 MiB | 309 MiB | 84 KB |

So the working set is flat in data size and per-item state, and grows with entry count at about 1.3 KB per entry; it is **not
constant in the number of entries**. A lazy directory in the zip reader would remove that term; it is raised with OneArchive's
owners rather than worked around with a second zip reader here.

## How it was checked

Black-box, against the olmreader 1.9.0 jar (Maven Central, sha1 `da66d6ee52b2e1782eb338772a0cdd998d874393`, BSD-2-Clause) driven by
`tools/olm/OlmDump.java`, and the olm-reader 1.0.0 npm package (Apache-2.0) run as `olm-extract` with `TZ=UTC`. Neither's source was
read. `tools/olm/run.sh` runs both over the two sample archives (`tools/corpus/fetch-olm.sh`) and two archives written by
`tools/olm/make_olm.py`. A generated archive is also a schema check: the jar validates every record against the published schemas.

| Archive | Records | Jar: compared / unexplained | npm package: messages paired / unexplained |
|---|---|---|---|
| `outlook_example.olm` | 38 | all fields of 7 emails, 8 appointments, 5 contacts, 5 tasks, 1 note, 2 groups, 10 categories / **0** | 7 / **0** |
| `bill_rapp.olm` | 492 | 483 emails, 1 contact, 8 categories / **0** | 483 / **0** |
| generated, seed 1 | 54 | 40 emails (nested folders, non-ASCII, 14 attachments), every PIM kind / **0** | 40 / **0** |
| generated, seed 2 (stored, 70 KB attachments) | 74 | 60 emails and every PIM kind / **0** | 60 / **0** |

Compared per email: subject, From, To, Cc, Bcc, Sender, sent and received time, Message-ID, body, and every attachment's name,
size and SHA-256; per other kind the fields the jar exposes (summary, location, UUID, start, end, names, due date, members).

Explained differences, all in the oracles:

- The jar keys attachment bytes by name, so two attachments with one name collapse to one (1 message in the Enron sample).
- The jar gives no folder; the npm package's EML path does, and every message's folder matches.
- The npm package reads dates in the machine's zone, writes `unknown@unknown.com` for a missing sender and `No subject`,
  drops `<…>` runs and addresses it cannot parse (446 recipients in the Enron sample), writes non-SMTP address text as an address
  (118), writes HTML bodies as text, adds no attachments to its EML, and in one message inserts a bullet character (12 classes in
  `tools/olm/compare_npm.py`, each counted in the run's output).

Fuzzing: the `olm` target (zip built around mutated record XML, or a mutated whole zip) ran 600 s in the `Checked` build: 9.18 million
executions, no findings. An earlier run found two escapes, both fixed and kept as seeds in `testdata/fuzz/olm`: a line break in a
UID or address reaching the calendar writer, and an XML reader failure on a UCS-4 entry that was not caught. The full `Checked`
suite (2,130 tests) passes.

## Not done

Recovery of a damaged zip (it uses OneArchive's recovery mode, which is not in the pinned release), Outlook 2011 layouts (no sample
archive), archives with a raw MIME source, attendees or recurrence exceptions (none seen), and a typed view of the many flag
elements. No Outlook for Mac has been run, so the date zone, the e-mail slot numbering and the relative-recurrence numbering are
agent defaults.


<!-- https://recursiveai.org/onemail/docs/conversion/ -->

# Conversions and loss reports

A conversion turns an item of one model into an item of another and says what it could not carry. Nothing converts behind the
caller's back: a conversion is a call, and its result has two parts.

```csharp
public interface IItemConversion<in TSource, TResult>
{
    ConversionResult<TResult> Convert(TSource source);   // Value + Report; never throws for malformed content
}
```

Namespace `RecursiveAI.OneMail.Conversion`. Calendar, contact and store-to-store conversions implement the same interface and return
the same report.

## The loss report

`LossReport` is a list of `LossEntry(Family, Name, Kind, Code, Reason, Path)`.

| Field | Meaning |
|---|---|
| `Family` | `mapi` (a property of the source), `mime` (a header field or part of the source) or `ics` (a property, parameter or component of an iCalendar source) |
| `Name` | the property (`PidTagSubject`; a named property as `{set}/"name"`; `0x1234001F` when it has no canonical name) or the header field name |
| `Kind` | `Dropped` (nothing carried), `Approximated` (carried in a weaker form), `Overridden` (two sources disagreed, one set aside), `Internal` (store bookkeeping: entry IDs, search keys, access flags; listed so nothing is silent, not a loss of content) |
| `Code` | a stable identifier of the rule, catalogued below |
| `Path` | empty for the message, else `attachment[2]`, `recipient`, `attachment[0]/message/attachment[1]` |

An identical entry is added once (the same property on ten recipients is one fact about "recipient"). A report from a nested item is
merged into its parent's with the item's path in front. `HasContentLoss` is true when any entry is not `Internal`.

Every property of the source is either carried, listed in the report, or part of the documented derived set (a `PidTagSubject` that
comes back with a subject prefix and normalised subject added is not a loss). That statement is what the round-trip checks below test.

## MAPI to MIME: `MapiToMimeConverter` (MS-OXCMAIL 2.1)

`Convert(MapiMessage)` returns a new `MimeEntity` tree built from `MimePart`, `MimeMultipart` and `MimeMessagePart`; `ToArray()` writes it.
Options (`OxcMailExportOptions`): `UseTransportHeaders` (default true), `IncludeBcc` (false), `BoundaryFactory` (random boundaries
unless given), `MaxEmbeddedDepth` (16).

**Header block.** From `PidTagTransportMessageHeaders` when the message has it and the option is on, else built from the properties.

- The received fields are the base, in their order. `MIME-Version` and every `Content-*` field except `Content-Language` and
  `Content-Class`, and `Lines`, are dropped: the new message's structure is its own. Lines that are not header fields are not written
  (`TransportHeaderLineSkipped`). A `Bcc` field is not written unless `IncludeBcc`.
- For the fields the properties define (Subject, From, Sender, To, Cc, Reply-To, Date, Message-ID, In-Reply-To, References,
  Thread-Topic), the received field is kept as received when it **agrees** with the properties: same decoded subject (white space
  collapsed), same set of addresses, same instant (within a second), same identifiers. When it does not, the properties' value is
  written and `TransportHeaderOverridden` says so: the properties are the item's truth (OX2). When the addresses agree but a display
  name or the spelling of an address differs, the received spelling is kept and `AddressSpellingFromTransportHeader` says so.
- Fields the properties define that the received block lacks are added after it. `PidTagTransportMessageHeaders` itself is then an
  `Approximated` entry (`TransportHeadersRewritten`) whenever the written block differs from the stored one.
- Without a transport block the fields are written in this order: From, Sender, To, Cc, Bcc (option), Reply-To, Subject, Date,
  Message-ID, In-Reply-To, References, Importance, Sensitivity, Keywords, Thread-Topic, Thread-Index, Disposition-Notification-To,
  Return-Receipt-To, the `PS_INTERNET_HEADERS` properties, then `MIME-Version`.

**Properties to fields.**

| Property | Field |
|---|---|
| `PidTagSubject` without its prefix marker (U+0001 and one character), else `PidTagSubjectPrefix` + `PidTagNormalizedSubject` | `Subject` (an empty subject is an empty field) |
| `PidTagSentRepresenting*`, else `PidTagSender*` | `From`; `Sender` from `PidTagSender*` when it is another address |
| recipient rows by `PidTagRecipientType` 1, 2, 3 | `To`, `Cc`, `Bcc` |
| `PidTagReplyRecipientEntries` (one-off EntryIDs, MS-OXCDATA 2.2.5.1) | `Reply-To` |
| `PidTagClientSubmitTime` (else `PidTagMessageDeliveryTime`, `DateFromDeliveryTime`) | `Date`, in UTC (`+0000`), whole seconds |
| `PidTagInternetMessageId`, `PidTagInReplyToId`, `PidTagInternetReferences` | `Message-ID`, `In-Reply-To`, `References` (angle brackets added where missing) |
| `PidTagImportance` 0, 1, 2 | `Importance: Low`, `Normal`, `High` |
| `PidTagSensitivity` 1, 2, 3 | `Sensitivity: Personal`, `Private`, `Company-Confidential`; 0 has no field (`DefaultValueNotWritten`) |
| `PidNameKeywords` | `Keywords`, comma and space separated |
| `PidTagConversationTopic`, `PidTagConversationIndex` | `Thread-Topic`, `Thread-Index` (base64) |
| `PidTagReadReceiptRequested` true | `Disposition-Notification-To` (the sender's address); false has no field |
| `PidTagOriginatorDeliveryReportRequested` true | `Return-Receipt-To` (the sender's address); false has no field |
| string-named properties of `PS_INTERNET_HEADERS` | a field of that name |

**Addresses.** Fields carry SMTP addresses. An address stored as `EX` (an Exchange distinguished name) is resolved in this order
(OX1): its own `PidTagSmtpAddress`, `PidTagSenderSmtpAddress` or `PidTagSentRepresentingSmtpAddress`; the SMTP address of another
sender or recipient of the message with the same distinguished name; for the sender groups, the one mailbox of the matching
transport `From` (or `Sender`) field; the single address a transport `From`, `To`, `Cc`, `Sender` or `Reply-To` field gives for the
same display name. An address no step resolves is left out and reported (`AddressUnresolved`, or `ReplyToUnresolved`): nothing is
guessed from the distinguished name. An address that was resolved from `EX` is an `Approximated` entry (`ExchangeAddressWrittenAsSmtp`)
because reading the message back stores it as SMTP. A display name with non-ASCII characters is written as RFC 2047 encoded words
for the whole name (OX8).

**Bodies** (MS-OXCMAIL 2.1.3.3). `PidTagBody` becomes `text/plain; charset=utf-8` and `PidTagHtml` a `text/html` part whose bytes are
those of the property, with the charset of `PidTagInternetCodepage` (text and UTF-16 HTML are written as UTF-8). A message with
both is `multipart/alternative`; inline attachments (a content ID, not marked `attachment`) travel with the HTML in
`multipart/related`; other attachments make the whole `multipart/mixed`. A message with no body and no attachment is an empty
`text/plain`. The RTF body (`PidTagRtfCompressed`, MS-OXRTFCP) is decompressed and de-encapsulated (MS-OXRTFEX): HTML
encapsulation gives the HTML part when the message has none (`HtmlFromRtf`), plain-text encapsulation the text part (`TextFromRtf`).
RTF that encapsulates nothing is written as a `text/rtf` part when there is no HTML (`RtfCarriedAsTextRtf`), else not written
(`RtfRedundant`). A lone CR or LF in the text is written as CRLF (`BodyLineBreaksNormalized`).

**Attachments** (MS-OXCMAIL 2.1.3.4). By value: a part of type `PidTagAttachMimeTag` (`application/octet-stream` when absent or not a
media type), base64, `Content-Disposition` `inline` or `attachment` (from `PidTagAttachContentDisposition`, else a content ID or the
`ATT_MHTML_REF` flag means inline) with `filename` and `name` per RFC 2231, `Content-ID` in angle brackets and `Content-Location`. A
composite type on flat data (`multipart/*`, `message/*`) becomes `application/octet-stream` (`CompositeTypeOnFlatData`). An embedded
message becomes a `message/rfc822` part holding the converted message (limit `MaxEmbeddedDepth`). Attachments by reference, OLE
objects without flat data and attachments without data are not written and are reported.

**Everything else** a message, recipient or attachment holds is reported: `NoInternetMapping` for a property MS-OXCMAIL maps nowhere,
`StoreBookkeeping` for the store-internal ones. `PidTagMessageClass` other than `IPM.Note` is `MessageClassNotCarried`.

## MIME to MAPI: `MimeToMapiConverter` (MS-OXCMAIL 2.2)

`Convert(MimeEntity)` returns a new `MapiMessage` (property bag, recipient and attachment rows, embedded messages as items).
Options (`OxcMailImportOptions`): `Tnef` (`Merge`, the default, or `KeepAsAttachment`), `TnefReader` (limits for TNEF parts), `PartHandler`, `MaxEmbeddedDepth` (16), `MaxPartDepth` (64).

- **Message class** `IPM.Note`; `REPORT.IPM.Note.NDR` and `.IPNRN` for `multipart/report` with the matching `report-type`;
  `IPM.Note.SMIME.MultipartSigned` for `multipart/signed` and `IPM.Note.SMIME` for `application/pkcs7-mime`. Signed content and
  signature become a body and an attachment and the signature can no longer be verified against them (`SignatureNotVerifiable`, OX12).
- **`PidTagTransportMessageHeaders`** keeps the whole header block as received (UTF-8, else Latin-1), except the fields starting
  `X-MS-Exchange-Organization-` and `X-MS-Exchange-Forest-` (MS-OXCMAIL 2.2.3.2.28). What a property cannot hold is therefore not lost.
- **Fields to properties** are the inverse of the table above: `Subject` (also `PidTagSubjectPrefix`, possibly empty, and
  `PidTagNormalizedSubject`, split by the rule of 2.2.3.2.6.1), `From` (sent-representing group and, unless a `Sender` field says
  otherwise, the sender group), `Sender`, recipients (display name, `SMTP`, address and `PidTagSmtpAddress`, row IDs, object type),
  `PidTagDisplayTo/Cc/Bcc`, `Reply-To` (one-off entry IDs and names), `Date` (UTC; a zone other than UTC is `DateZoneNotStored`; a
  missing or invalid Date gives no property, never "now" so that the result is reproducible, OX7), the first `Received` time as
  `PidTagMessageDeliveryTime`, identifiers with angle brackets, `Importance` (else `Priority`, `X-Priority`, `X-MSMail-Priority`),
  `Sensitivity`, `Keywords`, `Thread-Topic`, `Thread-Index` (base64; anything else is ignored), the two receipt fields. Every other
  field not listed in 2.2.3.2.29 becomes a string-named property of `PS_INTERNET_HEADERS`, name as written, value decoded.
- **Groups** are flattened to their members (`GroupFlattened`). Only the first of a repeated single-valued field is mapped
  (`RepeatedHeaderNotMapped`, `InReplyToFirstOnly`, `ExtraAuthorsNotMapped`).
- **Bodies** are chosen as the shared view chooses them (`shared-view.md`, bodies for MIME): `text/plain` becomes `PidTagBody`;
  `text/html` the binary `PidTagHtml` in its own charset when that is a Windows code page (`PidTagInternetCodepage`), else UTF-8;
  an unnamed `text/rtf` part `PidTagRtfCompressed`. `PidTagNativeBody` says which. Any other leaf part, and any rendering of the body
  that is not chosen, is an attachment.
- **Attachments**: data, `PidTagAttachLongFilename` (and short), extension, `PidTagAttachMimeTag`, content ID without angle
  brackets, content location, the disposition as written (`PidTagAttachContentDisposition`), the inline flag, size. `message/rfc822`
  is an embedded message (method 5) converted by the same rules. Part headers with no property are `PartHeaderNotMapped`.
- **The hook.** `OxcMailImportOptions.PartHandler` sees every leaf part before it is classified; a caller that returns true has dealt with
  the part (it may add properties and attachments to the `MapiMessage` it is given). TNEF parts are handled by the converter itself
  (next section) after the handler has declined them.

### TNEF parts (MS-OXCMAIL 2.2.3.8, MS-OXTNEF)

A leaf part that holds a TNEF stream (found by content, as `MimeEntity.EnumerateTnefParts` does: `application/ms-tnef`, winmail.dat,
or any `application/*` part starting with the signature 0x223E9F78) is unpacked into the message. The rules, in the order the converter
applies them:

1. **The part qualifies** when the stream conforms to MS-OXTNEF (no wrong signature, version, truncation, checksum or limit defect), and its correlation key matches the message's `X-MS-TNEF-Correlator` field
   (step 4, OX21). A part that does not qualify is an ordinary attachment whose content type is `application/octet-stream`
   (step 4), reported as `TnefNotUsed` with the reason. The message is then pure MIME, as the spec says.
2. **Headers first.** The message is built from the header fields exactly as without a TNEF part. Then, for each TNEF property: not set
   yet, it is added as the TNEF reader gave it (named properties keep their name and are renumbered into the message's own named IDs);
   already set, it is ignored and, when its value differs, reported (`TnefPropertyIgnored`, `Overridden`). Properties that come from one
   header field are one group: subject (`PidTagSubject`, prefix, normalised subject), the sender, the sent-representing sender and the
   reply-to entries. When the headers set any of a group, the TNEF's other members of that group are ignored too (OX16).
3. **Bodies.** A body property is added when the MIME side has none of that kind, so a TNEF's compressed RTF or HTML stays next to the plain
   text of the MIME part (2.2: the plain text must not be preferred to a richer TNEF body). `PidTagNativeBody` and `PidTagInternetCodepage` are the
   TNEF's (the author's values) unless the HTML is the MIME part's; then `PidTagNativeBody` is the richest body the message has (HTML, then RTF,
   then text) (OX17).
4. **Message class.** `IPM.Note` is what the converter gives a message nothing says more about; a class the TNEF carries (`IPM.Appointment`,
   `IPM.Schedule.Meeting.Request`, ...) replaces it (`TnefMessageClassUsed`). A class the MIME structure gave (a report, S/MIME) stays (OX18).
5. **Recipients.** The headers' recipients win; the TNEF's recipient table is used only when the headers gave none, and ignored with a
   `TnefRecipientIgnored` entry for each address the headers lack (OX19).
6. **Attachments** of the TNEF (data, embedded messages, OLE storages) are the message's attachments in the TNEF part's place, with
   their properties as read; their 8-bit text keeps the code page of the TNEF. The 8-bit strings that were merged into the message are read
   with the TNEF's code page too (a second TNEF part with another code page has its single 8-bit strings stored as Unicode,
   `TnefCodePageDiffers`).
7. A stream that ends in CR LF outside any attribute is unpacked although 2.2.3.8 step 3 says to ignore it (OX20); the message gets an `Internal`
   `TnefTrailingCrLfIgnored` entry and the view an `AmbiguousTnefTrailingCrLf` defect.
8. The part itself is a `TnefPartUnpacked` entry (`Internal`: the content is carried, the carrier is not).

`OxcMailImportOptions.Tnef = KeepAsAttachment` leaves every TNEF part an attachment as for any binary part.

## MIME from a TNEF message (`OxcMailExportOptions.Tnef`)

`OxcMailTnefExport.TnefMessage` writes the TNEF form of MS-OXCMAIL 2.1.3.7 instead of a pure MIME message: `multipart/mixed` holding a
`text/plain` rendering of the body and one `application/ms-tnef; name="winmail.dat"` part, plus `X-MS-TNEF-Correlator`. The envelope fields
are still header fields (the reader takes them first). The TNEF holds every property, the attachments and the embedded messages; the
top-level recipient table is left to the headers (2.1.3.1.1), so a `Bcc` recipient is not written unless `IncludeBcc` puts it in the header.
Properties the TNEF format cannot carry are reported by name (`TnefWriterLimit`). A message with no plain-text body gets an empty
`text/plain` part (`TnefPlainTextNotRendered`; HTML to text is not done here). The correlation key is the message's own
`PidTagTnefCorrelationKey` when it has one (so the property is unchanged), else a value derived from the message so that output is
reproducible (OX22). Properties that travel in the TNEF are not listed as `NoInternetMapping`. The default is a pure MIME message.

## Outlook calendar and iCalendar: `MapiToIcsConverter`, `IcsToMapiConverter` (MS-OXCICAL)

Both live in `RecursiveAI.OneMail.Conversion.OxcIcal` and are described, with the item model they work on, in [`calendar-items.md`](https://recursiveai.org/onemail/docs/calendar-items/).
The `ics` family is the third `Family` value: its `Name` is the iCalendar property (`RRULE`, `ATTENDEE;ROLE`) or component.
Codes added by these two converters:

| Code | Kind | Direction | Meaning |
|---|---|---|---|
| `NoIcalMapping` | Dropped | item to ICS | the MAPI property has no iCalendar form in MS-OXCICAL |
| `NoMapiMapping` | Dropped | ICS to item | the iCalendar property has no MAPI form in MS-OXCICAL |
| `DerivedProperty` | Internal | both | a property the other side recomputes (end time from duration, clean global object ID) |
| `StoreBookkeeping` | Internal | item to ICS | entry IDs, search keys, flags |
| `MessageClassNotCarried` | Dropped | item to ICS | the class is not an appointment, meeting or task |
| `ForwardNotificationHasNoMethod` | Approximated | item to ICS | RFC 5546 has no method for a forward notification |
| `UidSynthesized` | Approximated | item to ICS | no usable global object ID: the UID was made from subject and times |
| `TaskStatusWeakened` | Approximated | item to ICS | waiting and deferred tasks are written IN-PROCESS |
| `RecurrenceNotCarried` | Dropped | item to ICS | a task recurrence with no iCalendar form |
| `RtfBodyNotCarried` | Dropped | item to ICS | the compressed RTF body; the text is the DESCRIPTION |
| `OrganizerWithoutSmtpAddress` | Dropped | item to ICS | an organizer with no SMTP address cannot be a mailto |
| `AmbiguousShortMonth`, `AmbiguousFirstInstance` | Approximated | item to ICS | see ambiguities OC1, OC2 |
| `NoEventOrTask` | Dropped | ICS to item | the calendar holds no VEVENT or VTODO |
| `UnknownTimeZone`, `FloatingTimeAssumedUtc` | Approximated | ICS to item | no zone, or a zone not defined: UTC |
| `MethodNotMapped` | Approximated | ICS to item | the METHOD has no message class: an appointment |
| `MultipleRRules`, `RecurrenceNotMapped` | Dropped | ICS to item | the rule does not fit an Outlook recurrence pattern |
| `DatesWithoutRecurrence`, `ExdateNotOnPattern`, `RdateWithoutException` | Dropped | ICS to item | EXDATE/RDATE with no series, a day the series does not occupy, or an added instance that is not a moved one |
| `OverrideWithoutSeries`, `OverrideWithoutTimes`, `OverrideNotOnPattern` | Dropped | ICS to item | a RECURRENCE-ID override that cannot become an exception |
| `ClassNotMapped`, `OrganizerNotMailto`, `ParameterNotMapped` | Dropped | ICS to item | CLASS value, ORGANIZER URI or ATTENDEE parameter with no MAPI field |
| `DtstampAsLastModified` | Approximated | ICS to item | DTSTAMP stands in for the last-modified time |
| `TaskCancelledAsDeferred` | Approximated | ICS to item | a cancelled VTODO is stored as deferred |

## Whole stores

`StoreConverter` (`RecursiveAI.OneMail.Conversion.Stores`) applies these converters to every item of a store and adds the store-level
codes (folders, flags, keywords, mbox line endings). The matrix of what each source and target carries, and those codes, are in
[`fidelity.md`](https://recursiveai.org/onemail/docs/fidelity/).

## Loss codes

`AddressSpellingFromTransportHeader`, `AddressUnresolved`, `AttachmentByReference`, `AttachmentWithoutData`, `BccNotWritten`,
`BodyLineBreaksNormalized`, `CompositeTypeOnFlatData`, `DateBeforeFileTimeEpoch`, `DateFromDeliveryTime`, `DateSubSecondDropped`, `DateZoneNotStored`,
`DefaultValueNotWritten`, `EmbeddedMessageTooDeep`, `EmptyValueNotWritten`, `ExchangeAddressWrittenAsSmtp`, `ExtraAuthorsNotMapped`,
`GroupFlattened`, `HtmlFromRtf`, `InReplyToFirstOnly`, `InvalidMediaType`, `MessageClassNotCarried`, `NamedPropertyValueSetAside`,
`NoInternetMapping`, `PartHeaderNotMapped`, `PartsTooDeep`, `RepeatedHeaderNotMapped`, `ReplyToUnresolved`, `RtfCarriedAsTextRtf`,
`RtfRedundant`, `RtfUndecodable`, `SignatureNotVerifiable`, `StoreBookkeeping`, `SubjectPrefixMarkerRemoved`,
`SubjectWhitespaceTrimmed`, `TextFromRtf`, `ThreadIndexNotBase64`, `TransportHeaderLineSkipped`, `TransportHeaderOverridden`,
`TransportHeadersRewritten`, `ValueOutOfRange`; TNEF: `TnefCodePageDiffers`, `TnefMessageClassUsed`, `TnefNotUsed`, `TnefPartUnpacked`, `TnefTrailingCrLfIgnored`,
`TnefPlainTextNotRendered`, `TnefPropertyIgnored`, `TnefPropertyWithoutName`, `TnefRecipientIgnored`, `TnefWriterLimit`.

## Limits

Embedded messages are converted to a depth of 16 and multiparts walked to a depth of 64; deeper content is left out and reported.
Property values are read as the MAPI layer limits them. The converters allocate in proportion to the message.

## How the conversions are checked

`tools/oxcmail/run.sh` (see its header) runs, over the MSG, PST and OST files of the corpus:

1. **Export.** Every item is converted to EML. OneMail parses it (no defect beyond those already in the received headers), and
   MimeKit 4.18.2 and Python 3.13 `email` read it through the oracle drivers of `tools/oracles`. `check_export.py` compares each
   oracle's subject, From, To, Cc, Date, Message-ID, plain and HTML body hashes and attachment names, sizes and SHA-256 with what
   the source item says. A difference that only one oracle shows and that `tools/oxcmail/explained.tsv` explains (an oracle not
   knowing a charset label of the received header) is listed, never silent.
2. **MAPI to MIME to MAPI** (`OxcMailTool rt-mapi`). Every property of the source is exact, the same text under another string type,
   or listed in the loss report; anything else is a silent difference and fails the run.
3. **MIME to MAPI to MIME** (`OxcMailTool rt-mime`) over message files: ten fields compared before and after (subject, From, To,
   Cc, Date, Message-ID, In-Reply-To, References, text and HTML bodies, attachments); each difference needs a loss entry.
4. **The view** (`OxcMailTool view`): how many senders and recipients resolve to SMTP.
5. **TNEF in MIME** (`TnefTool mime`, `tools/tnef/compare_mime.py`, `OxcMailTool rt-tnef`). Each TNEF file of the corpus is wrapped in four
   message shapes (Outlook's, a bare multipart, a single part, one with HTML from MIME). Every property of the TNEF is on the converted
   message exactly as the TNEF reader gave it or has a `TnefPropertyIgnored` entry; the attachments' properties are equal to a fresh read of the
   TNEF; a part that must not be unpacked is kept as an octet-stream attachment. The view's attachments (name, size, SHA-256) are compared with
   tnefparse 1.4.0, tnef 1.4.18, ytnef 2.1.2 and MimeKit 4.18.2 (`MimeKitTnef`: `TnefPart.ExtractAttachments`). `rt-tnef` is `rt-mapi` with the
   TNEF export.

## Contacts and vCards: `ContactToVCardConverter`, `VCardToContactConverter` (MS-OXVCARD)

Both implement `IItemConversion`; the reports use the families `mapi` (a contact property) and `vcard` (a card property, with its group). The mapping
tables, import rules and codes are in [contacts.md](https://recursiveai.org/onemail/docs/contacts/). The check (`ContactTool rt-contact`, `rt-vcard`) is the same statement as for MIME:
every property of the source is exact, in the documented derived set, or in the loss report. Codes added by this pair: `PhoneSlotChanged`,
`EmailSlotChanged` (Approximated), `EmptyValueNotWritten`, `DefaultValueNotWritten` (Internal), `PartialDate`, `TimeOfDayDropped`,
`OrgUnitsMerged`, `AddressExtendedMerged`, `PhotoNotInline`, `KeyNotX509`, `GenderIdentityDropped`.


<!-- https://recursiveai.org/onemail/docs/fidelity/ -->

# Fidelity: what carries over when an item changes format

`Mail.ConvertAsync` and `StoreConverter` read an item of any family and write it in another format. Every conversion returns a
loss report (`docs/conversion.md`); this page says, per source family and target, what to expect from it. The tables below are
generated from `FidelityMatrix` in the library, and a test converts items of every family into every target and fails when a report
holds a code its cell does not list, when a cell says "skipped" for something that converts (or the other way round), or when this
page differs from the matrix.

| Level | Meaning |
|---|---|
| Exact | the same bytes, or the same values |
| Derived | recomputed in the target's own form by a rule of the specification; reading it back gives the same meaning |
| Approximated | carried in a weaker form; every case is a loss-report entry |
| Lost | not carried; every case is a loss-report entry, or the item is skipped and counted with the reason shown |
| None | the source has nothing of this kind |

The columns: **Content** is headers, addresses, bodies and attachments (for calendar and contact items, their fields); **Properties**
is what the source holds that the target has no field for (MAPI properties without an Internet form, OLM elements nothing reads);
**Flags** is read, answered, flagged and draft state plus keywords or categories; **Folders** is the folder the item was in.

## Store to store

`StoreConverter` enumerates a source one item at a time and hands each converted item to an `IStoreSink`. Sinks exist for:

| Destination | Layout | Flags | Dates |
|---|---|---|---|
| `Eml` | `<folder>/<n>-<subject>.eml` | not kept (`FlagsNotCarried`) | file time = arrival date |
| `Mbox` | one `<folder>.mbox` per folder (root items in `_root.mbox`); a destination with an extension is one file | `Status`/`X-Status`/`X-Keywords` fields, added only to a message that has none | envelope date = arrival date, else sent date, else the Unix epoch (`EnvelopeDateUnknown`) |
| `Maildir` | Maildir++ (`.Parent.Child`); a dot inside a folder name becomes `_` | standard flags and up to 26 keywords per folder | file time = arrival date |
| `AppleMail` | `<folder>.mbox/Messages/<n>.emlx` | the `.emlx` flags word; keywords not kept | `date-received` and `date-sent` |
| `Ics`, `JCal`, `XCal`, `VCard`, `JCard`, `XCard`, `Tnef` | one file per item under the folder tree | none | file time |

Calendar and contact items that reach a mail store are written as `.ics` and `.vcf` files in an `_items` directory beside it
(`OtherItemsMode.Route`, the default) or skipped (`OtherItemsMode.Skip`). An item the target cannot hold (a contact for an
iCalendar target, a message for a card target) is skipped, counted under its reason, and the totals always satisfy
items read = items written + items skipped.

Source folders: PST/OST folder paths below the root, OLM folder paths, Maildir++ levels, Apple Mail bundle nesting, and, for a
directory that is not a store, the directory names (a file holding many items adds its own name). `FolderFilter` restricts a run to one subtree.

A cancelled run finishes its destination, so what was written reads back whole; the report holds the counts so far. A failure of the
destination stops the run; an item that cannot be read or converted is skipped as `Failed` and listed in `Errors`.

## The matrix

<!-- matrix:start -->
### MIME message (EML, mbox, Maildir, EMLX, DBX, MHTML)

| To | Content | Properties | Flags | Folders | Codes it can report | Note |
|---|---|---|---|---|---|---|
| Eml | Exact | None | Lost | Derived | `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried` | One .eml file per message; folders become directories. A file has no flags, so they are reported. |
| Mbox | Exact | None | Derived | Derived | `EnvelopeDateUnknown`, `FinalLineEndingAdded`, `FlagsNotCarried`, `FlagsWrittenAsHeaders`, `FolderFlattened`, `FolderNameChanged`, `KeywordsNormalized` | Flags that the message does not state itself are written as Status, X-Status and X-Keywords fields in front of it; the only other case in which bytes change is a message with no final line ending, which gains one. One file per folder; a single-file destination drops folders. |
| Maildir | Exact | None | Exact | Derived | `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried` | Flags and up to 26 keywords per folder go into the file name; Maildir++ folder names cannot hold a dot. |
| AppleMail | Exact | None | Derived | Derived | `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried` | Flags go into the .emlx property list; keywords have no place there. |
| Tnef | Approximated | None | None | Lost | `DateZoneNotStored`, `DefaultValueNotWritten`, `EmbeddedMessageTooDeep`, `EmptyValueNotWritten`, `ExtraAuthorsNotMapped`, `GroupFlattened`, `InReplyToFirstOnly`, `InvalidMediaType`, `NamedPropertyValueSetAside`, `PartHeaderNotMapped`, `PartsTooDeep`, `RepeatedHeaderNotMapped`, `SignatureNotVerifiable`, `StoreBookkeeping`, `ThreadIndexNotBase64`, `TnefCodePageDiffers`, `TnefMessageClassUsed`, `TnefNotUsed`, `TnefPartUnpacked`, `TnefPropertyIgnored`, `TnefPropertyWithoutName`, `TnefRecipientIgnored`, `TnefTrailingCrLfIgnored`, `TnefWriterLimit`, `ValueOutOfRange` | The message is converted to an Outlook item (MS-OXCMAIL) and written as TNEF. |
| Ics | Lost | None | None | Lost | skipped: `NotCalendarItem` | A message is not a calendar item; it is skipped and counted. |
| JCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | A message is not a calendar item; it is skipped and counted. |
| XCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | A message is not a calendar item; it is skipped and counted. |
| VCard | Lost | None | None | Lost | skipped: `NotContactItem` | A message is not a contact; it is skipped and counted. |
| JCard | Lost | None | None | Lost | skipped: `NotContactItem` | A message is not a contact; it is skipped and counted. |
| XCard | Lost | None | None | Lost | skipped: `NotContactItem` | A message is not a contact; it is skipped and counted. |

### Outlook mail (MSG, OFT, PST, OST, TNEF)

| To | Content | Properties | Flags | Folders | Codes it can report | Note |
|---|---|---|---|---|---|---|
| Eml | Derived | Lost | Lost | Derived | `AddressSpellingFromTransportHeader`, `AddressUnresolved`, `AttachmentByReference`, `AttachmentWithoutData`, `BccNotWritten`, `BodyLineBreaksNormalized`, `CompositeTypeOnFlatData`, `DateBeforeFileTimeEpoch`, `DateFromDeliveryTime`, `DateSubSecondDropped`, `DefaultValueNotWritten`, `EmbeddedMessageTooDeep`, `EmptyValueNotWritten`, `ExchangeAddressWrittenAsSmtp`, `FlagsNotCarried`, `FolderNameChanged`, `HtmlFromRtf`, `InvalidMediaType`, `KeywordsNormalized`, `KeywordsNotCarried`, `MessageClassNotCarried`, `NamedPropertyValueSetAside`, `NoInternetMapping`, `ReplyToUnresolved`, `RtfCarriedAsTextRtf`, `RtfRedundant`, `RtfUndecodable`, `StoreBookkeeping`, `SubjectPrefixMarkerRemoved`, `SubjectWhitespaceTrimmed`, `TextFromRtf`, `TransportHeaderLineSkipped`, `TransportHeaderOverridden`, `TransportHeadersRewritten`, `ValueOutOfRange` | MS-OXCMAIL 2.1 builds the message from the properties; properties with no Internet form are reported. Flags come from PidTagMessageFlags (read, unsent), PidTagFlagStatus and PidTagLastVerbExecuted. |
| Mbox | Derived | Lost | Derived | Derived | `AddressSpellingFromTransportHeader`, `AddressUnresolved`, `AttachmentByReference`, `AttachmentWithoutData`, `BccNotWritten`, `BodyLineBreaksNormalized`, `CompositeTypeOnFlatData`, `DateBeforeFileTimeEpoch`, `DateFromDeliveryTime`, `DateSubSecondDropped`, `DefaultValueNotWritten`, `EmbeddedMessageTooDeep`, `EmptyValueNotWritten`, `EnvelopeDateUnknown`, `ExchangeAddressWrittenAsSmtp`, `FinalLineEndingAdded`, `FlagsNotCarried`, `FlagsWrittenAsHeaders`, `FolderFlattened`, `FolderNameChanged`, `HtmlFromRtf`, `InvalidMediaType`, `KeywordsNormalized`, `KeywordsNotCarried`, `MessageClassNotCarried`, `NamedPropertyValueSetAside`, `NoInternetMapping`, `ReplyToUnresolved`, `RtfCarriedAsTextRtf`, `RtfRedundant`, `RtfUndecodable`, `StoreBookkeeping`, `SubjectPrefixMarkerRemoved`, `SubjectWhitespaceTrimmed`, `TextFromRtf`, `TransportHeaderLineSkipped`, `TransportHeaderOverridden`, `TransportHeadersRewritten`, `ValueOutOfRange` | MS-OXCMAIL 2.1 builds the message from the properties; properties with no Internet form are reported. Flags come from PidTagMessageFlags (read, unsent), PidTagFlagStatus and PidTagLastVerbExecuted. |
| Maildir | Derived | Lost | Derived | Derived | `AddressSpellingFromTransportHeader`, `AddressUnresolved`, `AttachmentByReference`, `AttachmentWithoutData`, `BccNotWritten`, `BodyLineBreaksNormalized`, `CompositeTypeOnFlatData`, `DateBeforeFileTimeEpoch`, `DateFromDeliveryTime`, `DateSubSecondDropped`, `DefaultValueNotWritten`, `EmbeddedMessageTooDeep`, `EmptyValueNotWritten`, `ExchangeAddressWrittenAsSmtp`, `FlagsNotCarried`, `FolderNameChanged`, `HtmlFromRtf`, `InvalidMediaType`, `KeywordsNormalized`, `KeywordsNotCarried`, `MessageClassNotCarried`, `NamedPropertyValueSetAside`, `NoInternetMapping`, `ReplyToUnresolved`, `RtfCarriedAsTextRtf`, `RtfRedundant`, `RtfUndecodable`, `StoreBookkeeping`, `SubjectPrefixMarkerRemoved`, `SubjectWhitespaceTrimmed`, `TextFromRtf`, `TransportHeaderLineSkipped`, `TransportHeaderOverridden`, `TransportHeadersRewritten`, `ValueOutOfRange` | MS-OXCMAIL 2.1 builds the message from the properties; properties with no Internet form are reported. Flags come from PidTagMessageFlags (read, unsent), PidTagFlagStatus and PidTagLastVerbExecuted. |
| AppleMail | Derived | Lost | Derived | Derived | `AddressSpellingFromTransportHeader`, `AddressUnresolved`, `AttachmentByReference`, `AttachmentWithoutData`, `BccNotWritten`, `BodyLineBreaksNormalized`, `CompositeTypeOnFlatData`, `DateBeforeFileTimeEpoch`, `DateFromDeliveryTime`, `DateSubSecondDropped`, `DefaultValueNotWritten`, `EmbeddedMessageTooDeep`, `EmptyValueNotWritten`, `ExchangeAddressWrittenAsSmtp`, `FlagsNotCarried`, `FolderNameChanged`, `HtmlFromRtf`, `InvalidMediaType`, `KeywordsNormalized`, `KeywordsNotCarried`, `MessageClassNotCarried`, `NamedPropertyValueSetAside`, `NoInternetMapping`, `ReplyToUnresolved`, `RtfCarriedAsTextRtf`, `RtfRedundant`, `RtfUndecodable`, `StoreBookkeeping`, `SubjectPrefixMarkerRemoved`, `SubjectWhitespaceTrimmed`, `TextFromRtf`, `TransportHeaderLineSkipped`, `TransportHeaderOverridden`, `TransportHeadersRewritten`, `ValueOutOfRange` | MS-OXCMAIL 2.1 builds the message from the properties; properties with no Internet form are reported. Flags come from PidTagMessageFlags (read, unsent), PidTagFlagStatus and PidTagLastVerbExecuted. |
| Tnef | Exact | Exact | Exact | Lost | `TnefWriterLimit` | TNEF carries every property; a type or an OLE object it cannot hold is reported by name. |
| Ics | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| JCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| XCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| VCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |
| JCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |
| XCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |

### Outlook appointment, meeting or task (MSG, PST, OST, TNEF)

| To | Content | Properties | Flags | Folders | Codes it can report | Note |
|---|---|---|---|---|---|---|
| Eml | Approximated | Lost | None | Derived | `AmbiguousFirstInstance`, `AmbiguousShortMonth`, `AttachmentNotCarried`, `AttachmentWithoutException`, `CountOutOfRange`, `DerivedProperty`, `EndMismatch`, `ExceptionWithoutAttachment`, `FlagsNotCarried`, `FolderNameChanged`, `ForwardNotificationHasNoMethod`, `InconsistentTimeZone`, `KeywordsNormalized`, `KeywordsNotCarried`, `MessageClassNotCarried`, `MissingTime`, `NoIcalMapping`, `OrganizerWithoutSmtpAddress`, `RecurrenceNotCarried`, `RtfBodyNotCarried`, `StoreBookkeeping`, `TaskStatusWeakened`, `TrailingBytes`, `Truncated`, `UidSynthesized`, `UnexpectedVersion`, `UnsupportedCalendar`, `ValueOutOfRange` | Routed to an .ics file in the side directory (or skipped with OtherItemsMode.Skip) by MS-OXCICAL. |
| Mbox | Approximated | Lost | None | Derived | `AmbiguousFirstInstance`, `AmbiguousShortMonth`, `AttachmentNotCarried`, `AttachmentWithoutException`, `CountOutOfRange`, `DerivedProperty`, `EndMismatch`, `ExceptionWithoutAttachment`, `FlagsNotCarried`, `FolderNameChanged`, `ForwardNotificationHasNoMethod`, `InconsistentTimeZone`, `KeywordsNormalized`, `KeywordsNotCarried`, `MessageClassNotCarried`, `MissingTime`, `NoIcalMapping`, `OrganizerWithoutSmtpAddress`, `RecurrenceNotCarried`, `RtfBodyNotCarried`, `StoreBookkeeping`, `TaskStatusWeakened`, `TrailingBytes`, `Truncated`, `UidSynthesized`, `UnexpectedVersion`, `UnsupportedCalendar`, `ValueOutOfRange` | Routed to an .ics file in the side directory (or skipped with OtherItemsMode.Skip) by MS-OXCICAL. |
| Maildir | Approximated | Lost | None | Derived | `AmbiguousFirstInstance`, `AmbiguousShortMonth`, `AttachmentNotCarried`, `AttachmentWithoutException`, `CountOutOfRange`, `DerivedProperty`, `EndMismatch`, `ExceptionWithoutAttachment`, `FlagsNotCarried`, `FolderNameChanged`, `ForwardNotificationHasNoMethod`, `InconsistentTimeZone`, `KeywordsNormalized`, `KeywordsNotCarried`, `MessageClassNotCarried`, `MissingTime`, `NoIcalMapping`, `OrganizerWithoutSmtpAddress`, `RecurrenceNotCarried`, `RtfBodyNotCarried`, `StoreBookkeeping`, `TaskStatusWeakened`, `TrailingBytes`, `Truncated`, `UidSynthesized`, `UnexpectedVersion`, `UnsupportedCalendar`, `ValueOutOfRange` | Routed to an .ics file in the side directory (or skipped with OtherItemsMode.Skip) by MS-OXCICAL. |
| AppleMail | Approximated | Lost | None | Derived | `AmbiguousFirstInstance`, `AmbiguousShortMonth`, `AttachmentNotCarried`, `AttachmentWithoutException`, `CountOutOfRange`, `DerivedProperty`, `EndMismatch`, `ExceptionWithoutAttachment`, `FlagsNotCarried`, `FolderNameChanged`, `ForwardNotificationHasNoMethod`, `InconsistentTimeZone`, `KeywordsNormalized`, `KeywordsNotCarried`, `MessageClassNotCarried`, `MissingTime`, `NoIcalMapping`, `OrganizerWithoutSmtpAddress`, `RecurrenceNotCarried`, `RtfBodyNotCarried`, `StoreBookkeeping`, `TaskStatusWeakened`, `TrailingBytes`, `Truncated`, `UidSynthesized`, `UnexpectedVersion`, `UnsupportedCalendar`, `ValueOutOfRange` | Routed to an .ics file in the side directory (or skipped with OtherItemsMode.Skip) by MS-OXCICAL. |
| Ics | Approximated | Lost | None | Lost | `AmbiguousFirstInstance`, `AmbiguousShortMonth`, `AttachmentNotCarried`, `AttachmentWithoutException`, `CountOutOfRange`, `DerivedProperty`, `EndMismatch`, `ExceptionWithoutAttachment`, `ForwardNotificationHasNoMethod`, `InconsistentTimeZone`, `MessageClassNotCarried`, `MissingTime`, `NoIcalMapping`, `OrganizerWithoutSmtpAddress`, `RecurrenceNotCarried`, `RtfBodyNotCarried`, `StoreBookkeeping`, `TaskStatusWeakened`, `TrailingBytes`, `Truncated`, `UidSynthesized`, `UnexpectedVersion`, `UnsupportedCalendar`, `ValueOutOfRange` | MS-OXCICAL: recurrence, exceptions, attendees and reminders carry; properties iCalendar has no field for are reported. |
| JCal | Approximated | Lost | None | Lost | `AmbiguousFirstInstance`, `AmbiguousShortMonth`, `AttachmentNotCarried`, `AttachmentWithoutException`, `CountOutOfRange`, `DerivedProperty`, `EndMismatch`, `ExceptionWithoutAttachment`, `FormAmbiguousValueType`, `FormBareParameterAsType`, `FormCharsetRemoved`, `FormComponentNotCarried`, `FormDuplicateParameterMerged`, `FormExtraComponentsDropped`, `FormFolding`, `FormGroupNotCarried`, `FormInlineBinaryAsDataUri`, `FormInvalidXmlCharacter`, `FormItemOrder`, `FormLineNotCarried`, `FormNameCase`, `FormParameterNotCarried`, `FormPropertyNotCarried`, `FormRedundantValueParameter`, `FormStructuredPadded`, `FormTransferEncodingRemoved`, `FormTypeNameNotCarried`, `FormValueNotRepresentable`, `FormVersionNotCarried`, `FormVersionSpecificForm`, `ForwardNotificationHasNoMethod`, `InconsistentTimeZone`, `MessageClassNotCarried`, `MissingTime`, `NoIcalMapping`, `OrganizerWithoutSmtpAddress`, `RecurrenceNotCarried`, `RtfBodyNotCarried`, `StoreBookkeeping`, `TaskStatusWeakened`, `TrailingBytes`, `Truncated`, `UidSynthesized`, `UnexpectedVersion`, `UnsupportedCalendar`, `ValueOutOfRange` | MS-OXCICAL: recurrence, exceptions, attendees and reminders carry; properties iCalendar has no field for are reported. |
| XCal | Approximated | Lost | None | Lost | `AmbiguousFirstInstance`, `AmbiguousShortMonth`, `AttachmentNotCarried`, `AttachmentWithoutException`, `CountOutOfRange`, `DerivedProperty`, `EndMismatch`, `ExceptionWithoutAttachment`, `FormAmbiguousValueType`, `FormBareParameterAsType`, `FormCharsetRemoved`, `FormComponentNotCarried`, `FormDuplicateParameterMerged`, `FormExtraComponentsDropped`, `FormFolding`, `FormGroupNotCarried`, `FormInlineBinaryAsDataUri`, `FormInvalidXmlCharacter`, `FormItemOrder`, `FormLineNotCarried`, `FormNameCase`, `FormParameterNotCarried`, `FormPropertyNotCarried`, `FormRedundantValueParameter`, `FormStructuredPadded`, `FormTransferEncodingRemoved`, `FormTypeNameNotCarried`, `FormValueNotRepresentable`, `FormVersionNotCarried`, `FormVersionSpecificForm`, `ForwardNotificationHasNoMethod`, `InconsistentTimeZone`, `MessageClassNotCarried`, `MissingTime`, `NoIcalMapping`, `OrganizerWithoutSmtpAddress`, `RecurrenceNotCarried`, `RtfBodyNotCarried`, `StoreBookkeeping`, `TaskStatusWeakened`, `TrailingBytes`, `Truncated`, `UidSynthesized`, `UnexpectedVersion`, `UnsupportedCalendar`, `ValueOutOfRange` | MS-OXCICAL: recurrence, exceptions, attendees and reminders carry; properties iCalendar has no field for are reported. |
| Tnef | Exact | Exact | Exact | Lost | `TnefWriterLimit` | Every property travels in the TNEF. |
| VCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |
| JCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |
| XCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |

### Outlook contact (MSG, PST, OST, TNEF)

| To | Content | Properties | Flags | Folders | Codes it can report | Note |
|---|---|---|---|---|---|---|
| Eml | Approximated | Lost | None | Derived | `AttachmentNotCarried`, `DefaultValueNotWritten`, `DerivedValue`, `EmailSlotChanged`, `EmptyValueNotWritten`, `ExchangeAddressNotExported`, `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `MemberNotCarried`, `NotAContact`, `PhoneSlotChanged`, `StoreBookkeeping`, `UnsupportedPhotoType` | Routed to a .vcf file in the side directory (or skipped) by MS-OXVCARD. |
| Mbox | Approximated | Lost | None | Derived | `AttachmentNotCarried`, `DefaultValueNotWritten`, `DerivedValue`, `EmailSlotChanged`, `EmptyValueNotWritten`, `ExchangeAddressNotExported`, `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `MemberNotCarried`, `NotAContact`, `PhoneSlotChanged`, `StoreBookkeeping`, `UnsupportedPhotoType` | Routed to a .vcf file in the side directory (or skipped) by MS-OXVCARD. |
| Maildir | Approximated | Lost | None | Derived | `AttachmentNotCarried`, `DefaultValueNotWritten`, `DerivedValue`, `EmailSlotChanged`, `EmptyValueNotWritten`, `ExchangeAddressNotExported`, `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `MemberNotCarried`, `NotAContact`, `PhoneSlotChanged`, `StoreBookkeeping`, `UnsupportedPhotoType` | Routed to a .vcf file in the side directory (or skipped) by MS-OXVCARD. |
| AppleMail | Approximated | Lost | None | Derived | `AttachmentNotCarried`, `DefaultValueNotWritten`, `DerivedValue`, `EmailSlotChanged`, `EmptyValueNotWritten`, `ExchangeAddressNotExported`, `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `MemberNotCarried`, `NotAContact`, `PhoneSlotChanged`, `StoreBookkeeping`, `UnsupportedPhotoType` | Routed to a .vcf file in the side directory (or skipped) by MS-OXVCARD. |
| VCard | Approximated | Lost | None | Lost | `AttachmentNotCarried`, `DefaultValueNotWritten`, `DerivedValue`, `EmailSlotChanged`, `EmptyValueNotWritten`, `ExchangeAddressNotExported`, `MemberNotCarried`, `NotAContact`, `PhoneSlotChanged`, `StoreBookkeeping`, `UnsupportedPhotoType` | MS-OXVCARD: names, phones, e-mail slots, addresses, dates and photo carry; properties vCard has no field for are reported. |
| JCard | Approximated | Lost | None | Lost | `AttachmentNotCarried`, `DefaultValueNotWritten`, `DerivedValue`, `EmailSlotChanged`, `EmptyValueNotWritten`, `ExchangeAddressNotExported`, `FormAmbiguousValueType`, `FormBareParameterAsType`, `FormCharsetRemoved`, `FormComponentNotCarried`, `FormDuplicateParameterMerged`, `FormExtraComponentsDropped`, `FormFolding`, `FormGroupNotCarried`, `FormInlineBinaryAsDataUri`, `FormInvalidXmlCharacter`, `FormItemOrder`, `FormLineNotCarried`, `FormNameCase`, `FormParameterNotCarried`, `FormPropertyNotCarried`, `FormRedundantValueParameter`, `FormStructuredPadded`, `FormTransferEncodingRemoved`, `FormTypeNameNotCarried`, `FormValueNotRepresentable`, `FormVersionNotCarried`, `FormVersionSpecificForm`, `MemberNotCarried`, `NotAContact`, `PhoneSlotChanged`, `StoreBookkeeping`, `UnsupportedPhotoType` | MS-OXVCARD: names, phones, e-mail slots, addresses, dates and photo carry; properties vCard has no field for are reported. |
| XCard | Approximated | Lost | None | Lost | `AttachmentNotCarried`, `DefaultValueNotWritten`, `DerivedValue`, `EmailSlotChanged`, `EmptyValueNotWritten`, `ExchangeAddressNotExported`, `FormAmbiguousValueType`, `FormBareParameterAsType`, `FormCharsetRemoved`, `FormComponentNotCarried`, `FormDuplicateParameterMerged`, `FormExtraComponentsDropped`, `FormFolding`, `FormGroupNotCarried`, `FormInlineBinaryAsDataUri`, `FormInvalidXmlCharacter`, `FormItemOrder`, `FormLineNotCarried`, `FormNameCase`, `FormParameterNotCarried`, `FormPropertyNotCarried`, `FormRedundantValueParameter`, `FormStructuredPadded`, `FormTransferEncodingRemoved`, `FormTypeNameNotCarried`, `FormValueNotRepresentable`, `FormVersionNotCarried`, `FormVersionSpecificForm`, `MemberNotCarried`, `NotAContact`, `PhoneSlotChanged`, `StoreBookkeeping`, `UnsupportedPhotoType` | MS-OXVCARD: names, phones, e-mail slots, addresses, dates and photo carry; properties vCard has no field for are reported. |
| Tnef | Exact | Exact | Exact | Lost | `TnefWriterLimit` | Every property travels in the TNEF. |
| Ics | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| JCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| XCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |

### Outlook for Mac mail (OLM)

| To | Content | Properties | Flags | Folders | Codes it can report | Note |
|---|---|---|---|---|---|---|
| Eml | Approximated | Lost | Derived | Derived | `AddressUnresolved`, `BccNotWritten`, `ConversationTopicWhitespaceTrimmed`, `DateZoneNotStored`, `EmbeddedMessageAsData`, `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `OlmFieldDiffersFromSource`, `Unmapped` | The message is rebuilt from the fields the view reads (addresses, subject, dates, identifiers, bodies, attachments); other OLM elements are listed as Unmapped. The read flag becomes Seen, categories become keywords. |
| Mbox | Approximated | Lost | Derived | Derived | `AddressUnresolved`, `BccNotWritten`, `ConversationTopicWhitespaceTrimmed`, `DateZoneNotStored`, `EmbeddedMessageAsData`, `EnvelopeDateUnknown`, `FinalLineEndingAdded`, `FlagsNotCarried`, `FlagsWrittenAsHeaders`, `FolderFlattened`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `OlmFieldDiffersFromSource`, `Unmapped` | The message is rebuilt from the fields the view reads (addresses, subject, dates, identifiers, bodies, attachments); other OLM elements are listed as Unmapped. The read flag becomes Seen, categories become keywords. |
| Maildir | Approximated | Lost | Derived | Derived | `AddressUnresolved`, `BccNotWritten`, `ConversationTopicWhitespaceTrimmed`, `DateZoneNotStored`, `EmbeddedMessageAsData`, `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `OlmFieldDiffersFromSource`, `Unmapped` | The message is rebuilt from the fields the view reads (addresses, subject, dates, identifiers, bodies, attachments); other OLM elements are listed as Unmapped. The read flag becomes Seen, categories become keywords. |
| AppleMail | Approximated | Lost | Derived | Derived | `AddressUnresolved`, `BccNotWritten`, `ConversationTopicWhitespaceTrimmed`, `DateZoneNotStored`, `EmbeddedMessageAsData`, `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `OlmFieldDiffersFromSource`, `Unmapped` | The message is rebuilt from the fields the view reads (addresses, subject, dates, identifiers, bodies, attachments); other OLM elements are listed as Unmapped. The read flag becomes Seen, categories become keywords. |
| Tnef | Lost | Lost | None | Lost | skipped: `NotMapiConvertible` | An OLM record has no property bag; it is skipped and counted. |
| Ics | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| JCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| XCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| VCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |
| JCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |
| XCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |

### Outlook for Mac appointment, task or note (OLM)

| To | Content | Properties | Flags | Folders | Codes it can report | Note |
|---|---|---|---|---|---|---|
| Eml | Approximated | Lost | None | Derived | `AllDayEnd`, `AttendeeStatus`, `CalendarAttachments`, `DateOnlyZone`, `EndZoneIgnored`, `FlagsNotCarried`, `FloatingTime`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `NotACalendarItem`, `OrganizerNotAnAddress`, `RecurrenceEnd`, `RecurrenceKind`, `RelativeRecurrence`, `Unmapped` | Routed to an .ics file in the side directory (or skipped). |
| Mbox | Approximated | Lost | None | Derived | `AllDayEnd`, `AttendeeStatus`, `CalendarAttachments`, `DateOnlyZone`, `EndZoneIgnored`, `FlagsNotCarried`, `FloatingTime`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `NotACalendarItem`, `OrganizerNotAnAddress`, `RecurrenceEnd`, `RecurrenceKind`, `RelativeRecurrence`, `Unmapped` | Routed to an .ics file in the side directory (or skipped). |
| Maildir | Approximated | Lost | None | Derived | `AllDayEnd`, `AttendeeStatus`, `CalendarAttachments`, `DateOnlyZone`, `EndZoneIgnored`, `FlagsNotCarried`, `FloatingTime`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `NotACalendarItem`, `OrganizerNotAnAddress`, `RecurrenceEnd`, `RecurrenceKind`, `RelativeRecurrence`, `Unmapped` | Routed to an .ics file in the side directory (or skipped). |
| AppleMail | Approximated | Lost | None | Derived | `AllDayEnd`, `AttendeeStatus`, `CalendarAttachments`, `DateOnlyZone`, `EndZoneIgnored`, `FlagsNotCarried`, `FloatingTime`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `NotACalendarItem`, `OrganizerNotAnAddress`, `RecurrenceEnd`, `RecurrenceKind`, `RelativeRecurrence`, `Unmapped` | Routed to an .ics file in the side directory (or skipped). |
| Ics | Approximated | Lost | None | Lost | `AllDayEnd`, `AttendeeStatus`, `CalendarAttachments`, `DateOnlyZone`, `EndZoneIgnored`, `FloatingTime`, `NotACalendarItem`, `OrganizerNotAnAddress`, `RecurrenceEnd`, `RecurrenceKind`, `RelativeRecurrence`, `Unmapped` | Appointments, tasks and notes (VJOURNAL) carry their fields; the rest is listed as Unmapped. |
| JCal | Approximated | Lost | None | Lost | `AllDayEnd`, `AttendeeStatus`, `CalendarAttachments`, `DateOnlyZone`, `EndZoneIgnored`, `FloatingTime`, `FormAmbiguousValueType`, `FormBareParameterAsType`, `FormCharsetRemoved`, `FormComponentNotCarried`, `FormDuplicateParameterMerged`, `FormExtraComponentsDropped`, `FormFolding`, `FormGroupNotCarried`, `FormInlineBinaryAsDataUri`, `FormInvalidXmlCharacter`, `FormItemOrder`, `FormLineNotCarried`, `FormNameCase`, `FormParameterNotCarried`, `FormPropertyNotCarried`, `FormRedundantValueParameter`, `FormStructuredPadded`, `FormTransferEncodingRemoved`, `FormTypeNameNotCarried`, `FormValueNotRepresentable`, `FormVersionNotCarried`, `FormVersionSpecificForm`, `NotACalendarItem`, `OrganizerNotAnAddress`, `RecurrenceEnd`, `RecurrenceKind`, `RelativeRecurrence`, `Unmapped` | Appointments, tasks and notes (VJOURNAL) carry their fields; the rest is listed as Unmapped. |
| XCal | Approximated | Lost | None | Lost | `AllDayEnd`, `AttendeeStatus`, `CalendarAttachments`, `DateOnlyZone`, `EndZoneIgnored`, `FloatingTime`, `FormAmbiguousValueType`, `FormBareParameterAsType`, `FormCharsetRemoved`, `FormComponentNotCarried`, `FormDuplicateParameterMerged`, `FormExtraComponentsDropped`, `FormFolding`, `FormGroupNotCarried`, `FormInlineBinaryAsDataUri`, `FormInvalidXmlCharacter`, `FormItemOrder`, `FormLineNotCarried`, `FormNameCase`, `FormParameterNotCarried`, `FormPropertyNotCarried`, `FormRedundantValueParameter`, `FormStructuredPadded`, `FormTransferEncodingRemoved`, `FormTypeNameNotCarried`, `FormValueNotRepresentable`, `FormVersionNotCarried`, `FormVersionSpecificForm`, `NotACalendarItem`, `OrganizerNotAnAddress`, `RecurrenceEnd`, `RecurrenceKind`, `RelativeRecurrence`, `Unmapped` | Appointments, tasks and notes (VJOURNAL) carry their fields; the rest is listed as Unmapped. |
| VCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |
| JCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |
| XCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |
| Tnef | Lost | Lost | None | Lost | skipped: `NotMapiConvertible` | An OLM record has no property bag; it is skipped and counted. |

### Outlook for Mac contact or group (OLM)

| To | Content | Properties | Flags | Folders | Codes it can report | Note |
|---|---|---|---|---|---|---|
| Eml | Approximated | Lost | None | Derived | `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `MemberName`, `NotAContact`, `Unmapped` | Routed to a .vcf file in the side directory (or skipped). |
| Mbox | Approximated | Lost | None | Derived | `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `MemberName`, `NotAContact`, `Unmapped` | Routed to a .vcf file in the side directory (or skipped). |
| Maildir | Approximated | Lost | None | Derived | `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `MemberName`, `NotAContact`, `Unmapped` | Routed to a .vcf file in the side directory (or skipped). |
| AppleMail | Approximated | Lost | None | Derived | `FlagsNotCarried`, `FolderNameChanged`, `KeywordsNormalized`, `KeywordsNotCarried`, `MemberName`, `NotAContact`, `Unmapped` | Routed to a .vcf file in the side directory (or skipped). |
| VCard | Approximated | Lost | None | Lost | `MemberName`, `NotAContact`, `Unmapped` | Contacts and groups (KIND:group) carry their fields; the rest is listed as Unmapped. |
| JCard | Approximated | Lost | None | Lost | `FormAmbiguousValueType`, `FormBareParameterAsType`, `FormCharsetRemoved`, `FormComponentNotCarried`, `FormDuplicateParameterMerged`, `FormExtraComponentsDropped`, `FormFolding`, `FormGroupNotCarried`, `FormInlineBinaryAsDataUri`, `FormInvalidXmlCharacter`, `FormItemOrder`, `FormLineNotCarried`, `FormNameCase`, `FormParameterNotCarried`, `FormPropertyNotCarried`, `FormRedundantValueParameter`, `FormStructuredPadded`, `FormTransferEncodingRemoved`, `FormTypeNameNotCarried`, `FormValueNotRepresentable`, `FormVersionNotCarried`, `FormVersionSpecificForm`, `MemberName`, `NotAContact`, `Unmapped` | Contacts and groups (KIND:group) carry their fields; the rest is listed as Unmapped. |
| XCard | Approximated | Lost | None | Lost | `FormAmbiguousValueType`, `FormBareParameterAsType`, `FormCharsetRemoved`, `FormComponentNotCarried`, `FormDuplicateParameterMerged`, `FormExtraComponentsDropped`, `FormFolding`, `FormGroupNotCarried`, `FormInlineBinaryAsDataUri`, `FormInvalidXmlCharacter`, `FormItemOrder`, `FormLineNotCarried`, `FormNameCase`, `FormParameterNotCarried`, `FormPropertyNotCarried`, `FormRedundantValueParameter`, `FormStructuredPadded`, `FormTransferEncodingRemoved`, `FormTypeNameNotCarried`, `FormValueNotRepresentable`, `FormVersionNotCarried`, `FormVersionSpecificForm`, `MemberName`, `NotAContact`, `Unmapped` | Contacts and groups (KIND:group) carry their fields; the rest is listed as Unmapped. |
| Ics | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| JCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| XCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| Tnef | Lost | Lost | None | Lost | skipped: `NotMapiConvertible` | An OLM record has no property bag; it is skipped and counted. |

### Outlook for Mac categories and other records (OLM)

| To | Content | Properties | Flags | Folders | Codes it can report | Note |
|---|---|---|---|---|---|---|
| Eml | Lost | None | None | Lost | skipped: `NotMail` | A category or settings record is not mail, a calendar item or a contact; it is skipped and counted. |
| Mbox | Lost | None | None | Lost | skipped: `NotMail` | A category or settings record is not mail, a calendar item or a contact; it is skipped and counted. |
| Maildir | Lost | None | None | Lost | skipped: `NotMail` | A category or settings record is not mail, a calendar item or a contact; it is skipped and counted. |
| AppleMail | Lost | None | None | Lost | skipped: `NotMail` | A category or settings record is not mail, a calendar item or a contact; it is skipped and counted. |
| Ics | Lost | None | None | Lost | skipped: `NotCalendarItem` | A category or settings record is not mail, a calendar item or a contact; it is skipped and counted. |
| VCard | Lost | None | None | Lost | skipped: `NotContactItem` | A category or settings record is not mail, a calendar item or a contact; it is skipped and counted. |
| JCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | A category or settings record is not mail, a calendar item or a contact; it is skipped and counted. |
| JCard | Lost | None | None | Lost | skipped: `NotContactItem` | A category or settings record is not mail, a calendar item or a contact; it is skipped and counted. |
| XCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | A category or settings record is not mail, a calendar item or a contact; it is skipped and counted. |
| XCard | Lost | None | None | Lost | skipped: `NotContactItem` | A category or settings record is not mail, a calendar item or a contact; it is skipped and counted. |
| Tnef | Lost | None | None | Lost | skipped: `NotMapiConvertible` | A category or settings record is not mail, a calendar item or a contact; it is skipped and counted. |

### Calendar file (ICS, jCal, xCal)

| To | Content | Properties | Flags | Folders | Codes it can report | Note |
|---|---|---|---|---|---|---|
| Eml | Exact | None | None | Lost | none | Kept as the file's own bytes in the side directory. |
| Mbox | Exact | None | None | Lost | none | Kept as the file's own bytes in the side directory. |
| Maildir | Exact | None | None | Lost | none | Kept as the file's own bytes in the side directory. |
| AppleMail | Exact | None | None | Lost | none | Kept as the file's own bytes in the side directory. |
| Ics | Exact | None | None | Lost | none | An iCalendar file converted to iCalendar is the same bytes; the forms of RFC 7265 and RFC 6321 carry the same components, and every difference is a Form code. |
| JCal | Derived | None | None | Lost | `FormAmbiguousValueType`, `FormBareParameterAsType`, `FormCharsetRemoved`, `FormComponentNotCarried`, `FormDuplicateParameterMerged`, `FormExtraComponentsDropped`, `FormFolding`, `FormGroupNotCarried`, `FormInlineBinaryAsDataUri`, `FormInvalidXmlCharacter`, `FormItemOrder`, `FormLineNotCarried`, `FormNameCase`, `FormParameterNotCarried`, `FormPropertyNotCarried`, `FormRedundantValueParameter`, `FormStructuredPadded`, `FormTransferEncodingRemoved`, `FormTypeNameNotCarried`, `FormValueNotRepresentable`, `FormVersionNotCarried`, `FormVersionSpecificForm` | Between forms the data carries; folding and name case change the text and are reported as bookkeeping. |
| XCal | Derived | None | None | Lost | `FormAmbiguousValueType`, `FormBareParameterAsType`, `FormCharsetRemoved`, `FormComponentNotCarried`, `FormDuplicateParameterMerged`, `FormExtraComponentsDropped`, `FormFolding`, `FormGroupNotCarried`, `FormInlineBinaryAsDataUri`, `FormInvalidXmlCharacter`, `FormItemOrder`, `FormLineNotCarried`, `FormNameCase`, `FormParameterNotCarried`, `FormPropertyNotCarried`, `FormRedundantValueParameter`, `FormStructuredPadded`, `FormTransferEncodingRemoved`, `FormTypeNameNotCarried`, `FormValueNotRepresentable`, `FormVersionNotCarried`, `FormVersionSpecificForm` | Between forms the data carries; folding and name case change the text and are reported as bookkeeping. |
| VCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |
| JCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |
| XCard | Lost | None | None | Lost | skipped: `NotContactItem` | Not an item of that family; it is skipped and counted. |
| Tnef | Lost | None | None | Lost | skipped: `NotMail` | A calendar file is not a message; it is skipped and counted. |

### Card file (vCard, jCard, xCard)

| To | Content | Properties | Flags | Folders | Codes it can report | Note |
|---|---|---|---|---|---|---|
| Eml | Exact | None | None | Lost | none | Kept as the file's own bytes in the side directory. |
| Mbox | Exact | None | None | Lost | none | Kept as the file's own bytes in the side directory. |
| Maildir | Exact | None | None | Lost | none | Kept as the file's own bytes in the side directory. |
| AppleMail | Exact | None | None | Lost | none | Kept as the file's own bytes in the side directory. |
| VCard | Exact | None | None | Lost | none | A vCard file converted to vCard is the same bytes. |
| JCard | Derived | None | None | Lost | `FormAmbiguousValueType`, `FormBareParameterAsType`, `FormCharsetRemoved`, `FormComponentNotCarried`, `FormDuplicateParameterMerged`, `FormExtraComponentsDropped`, `FormFolding`, `FormGroupNotCarried`, `FormInlineBinaryAsDataUri`, `FormInvalidXmlCharacter`, `FormItemOrder`, `FormLineNotCarried`, `FormNameCase`, `FormParameterNotCarried`, `FormPropertyNotCarried`, `FormRedundantValueParameter`, `FormStructuredPadded`, `FormTransferEncodingRemoved`, `FormTypeNameNotCarried`, `FormValueNotRepresentable`, `FormVersionNotCarried`, `FormVersionSpecificForm` | Between forms the data carries; every difference is a Form code. |
| XCard | Derived | None | None | Lost | `FormAmbiguousValueType`, `FormBareParameterAsType`, `FormCharsetRemoved`, `FormComponentNotCarried`, `FormDuplicateParameterMerged`, `FormExtraComponentsDropped`, `FormFolding`, `FormGroupNotCarried`, `FormInlineBinaryAsDataUri`, `FormInvalidXmlCharacter`, `FormItemOrder`, `FormLineNotCarried`, `FormNameCase`, `FormParameterNotCarried`, `FormPropertyNotCarried`, `FormRedundantValueParameter`, `FormStructuredPadded`, `FormTransferEncodingRemoved`, `FormTypeNameNotCarried`, `FormValueNotRepresentable`, `FormVersionNotCarried`, `FormVersionSpecificForm` | Between forms the data carries; every difference is a Form code. |
| Ics | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| JCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| XCal | Lost | None | None | Lost | skipped: `NotCalendarItem` | Not an item of that family; it is skipped and counted. |
| Tnef | Lost | None | None | Lost | skipped: `NotMail` | A card file is not a message; it is skipped and counted. |

<!-- matrix:end -->

## Codes added by the store layer

| Code | Kind | Meaning |
|---|---|---|
| `FolderNameChanged` | Approximated | a character a file system or Maildir++ cannot hold in a folder name was replaced |
| `FolderFlattened` | Dropped | a single mbox file has no folders |
| `FlagsNotCarried` | Dropped | the destination keeps no flags, or an mbox message's own Status fields were kept over the source's flags |
| `KeywordsNotCarried` | Dropped | the destination keeps no keywords (or a Maildir folder is past 26) |
| `KeywordsNormalized` | Approximated | white space in a keyword or category name became an underscore (Maildir and `X-Keywords` names cannot hold it) |
| `ConversationTopicWhitespaceTrimmed` | Approximated | OLM record rebuilt from the shared view: leading or trailing white space of the conversation topic cannot survive in a header value |
| `FlagsWrittenAsHeaders` | Internal | mbox: Status, X-Status and X-Keywords fields were added |
| `FinalLineEndingAdded` | Approximated | mbox: the message did not end in a line ending and gained one |
| `EnvelopeDateUnknown` | Approximated | mbox: the message has no date; the envelope line carries the Unix epoch so output is reproducible |
| `OlmFieldDiffersFromSource` | Overridden | an OLM record holds the original message as well; where a field of the record differs from it, the original is written and the field named |
| `BccNotWritten` | Dropped | a message rebuilt from the shared view has blind-copy recipients; they are not written |
| `DateZoneNotStored` | Approximated | a message rebuilt from the shared view is dated in UTC |
| `EmbeddedMessageAsData` | Approximated | a message rebuilt from the shared view writes an embedded message as an ordinary attachment |
| `FormFolding`, `FormNameCase`, `FormItemOrder`, `FormLineNotCarried`, `FormRedundantValueParameter`, `FormDuplicateParameterMerged`, `FormCharsetRemoved` | Internal or Approximated | what the JSON or XML form of a calendar or card could not carry exactly (the calendar library's form losses by kind); folding and name case are bookkeeping |

## Command line

```
onemail convert mail.olm out --to maildir          # a store to a Maildir++ tree, folders kept
onemail convert inbox.mbox out.mbox                # the format follows the extension (eml, maildir and emlx need --to)
onemail convert archive.pst out --to eml --folder "Top of Personal Folders/Inbox"
onemail export archive.pst calendar --format ics --json
```

`convert` and `export` print the totals (read, written, skipped by reason, per-folder counts, each loss code with its item count and an
example) and, with `--json`, JSON instead. `--no-other-items` skips calendar and contact items that would go to `_items`. The exit code
is 0 on success (skips and reported losses are not errors), 1 when an item errored or the run failed, 2 for a usage error, 130 when cancelled.

## What was checked

On the corpora (counts per run, all in `tools/convert/run.sh`):

- MIME to EML, mbox, Maildir and EMLX, read back byte-identical: probes 219/224 (5 have no header block and are skipped as `Unreadable`),
  Mime4j 78/80, dovecot mbox 3,237/3,237, dovecot messages 2,858/2,859, Enron sample 5,032/5,032, DBX store 2,053/2,053; none changed.
  An mbox message that lacks a final line ending gains one (`FinalLineEndingAdded`).
- MAPI to EML and back to MAPI: MSG 45 items, 8,743 properties and PST 25 items, 2,399 properties, 0 silent; OLM mail 530 items,
  10,070 fields, 0 silent (11 reported).
- Store to store, items read = written + skipped and folder trees equal on every store (CFReDS OST, Tika PSTs, OLM generated and samples,
  DBX store in 13 folders, dovecot mbox, a Maildir made from it, Enron sample); Python `mailbox` and `email` read every produced EML tree,
  mbox and Maildir with the same folders and counts (18 of 18 runs); MimeKit, Python and Go differential on the produced EMLs ran with
  no unexplained difference.
- Memory: the CMU Enron maildir (517,401 messages, 3,311 folders) to Maildir 48.7 s, peak working set 119 MB; to mbox 23.1 s, 129 MB.
  Every message is read back and counted in the destination.


<!-- https://recursiveai.org/onemail/docs/display-html/ -->

# Display HTML, the sanitiser and the remote-content policy

`RecursiveAI.OneMail.Display` turns a message body into HTML for a browser or web view. `HtmlSanitizer.Sanitize(html, options)` is the layer
that rewrites HTML; `item.ToDisplayHtml(options)` (in the root namespace, over any `IMailItem`) builds a whole message view on top of it.
Both return a structured result and never throw for the content they are given.

This document says what the code does. It is a description, not a promise of safety: the sanitiser has not been reviewed independently
yet, and anything that matters should still be shown in a frame with its own restrictions (see *What the output assumes*).

## How it works

1. The input is parsed with OneHtml's WHATWG parser, scripting enabled (so `noscript` is text, as in a browser).
2. The tree is walked and the output is **written** from it, element by element, from the allowlists below. Nothing is copied through as text:
   names are checked, values are escaped, CSS is parsed with OneHtml's CSS parser and every token is serialised again.
3. The output is parsed and written again with the same rules, up to three times, until it is identical to its input. A tree that does not
   survive being serialised and parsed again is what mutation attacks on sanitisers use; if the output never settles, it is replaced by the
   page's text alone and a `LimitReached` change says so.

## Elements

Kept: `a abbr acronym address article aside b bdi bdo big blockquote br caption center cite code col colgroup dd del details dfn div dl dt em
figcaption figure font footer h1-h6 header hr i img ins kbd li main mark nav ol p picture pre q rp rt ruby s samp section small source
(inside picture only) span strike strong sub summary sup table tbody td tfoot th thead time tr tt u ul var wbr`.

Removed with their content: `script noscript template iframe object embed applet frame frameset noframes noembed title xmp plaintext textarea
select datalist audio video track canvas dialog meta link base basefont param bgsound keygen map area input svg math listing isindex portal
fencedframe slot`, and every element outside the HTML namespace (SVG and MathML bring script vectors, their own URL attributes and namespace
switches). A `style` element is read, filtered and re-emitted (below); it is removed if its `type` is not `text/css` or its `media` is not
`all`/`screen`.

Everything else (`form`, `button`, `label`, unknown elements, `html`, `head`, `body`) is **unwrapped**: its text and children stay, the tag goes.
`body` becomes the wrapper (below). Comments, processing instructions and conditional comments are dropped.

## Attributes

Allowed by name: `class id style title alt lang dir align valign width height border cellpadding cellspacing colspan rowspan hspace vspace
size span start value bgcolor color bordercolor face nowrap noshade reversed open scope headers abbr summary rules frame clear char datetime
type` (on `ol ul li`), `href` (on `a`), `src srcset sizes` (on `img`, and `srcset sizes` on `source`), `background` (on `table tr td th thead
tbody tfoot`). Numeric, colour and word attributes are checked against their shape. All other attributes, including every `on*` handler,
`srcdoc`, `formaction`, `ping`, `data-*`, `usemap`, `contenteditable`, `autofocus`, `name`, and any attribute with a namespace, are removed.
`rel`, `target` and `referrerpolicy` are never copied: links get `rel="noopener noreferrer"` (and `target="_blank"` unless
`OpenLinksInNewWindow` is false), images get `referrerpolicy="no-referrer"`.

`class` and `id` values (and `#fragment` links and the CSS selectors that name them) are prefixed with `om-`, so the host page's styles and scripts
do not reach the message by accident, and the message's names cannot shadow the page's.

## URLs

A reference is cleaned as a browser's URL parser does (leading and trailing control characters and blanks removed, tab and line breaks removed
everywhere); any other control character makes it unusable. Then:

| Form | `a href` | `img src`, `srcset`, `background`, CSS `url()` |
|---|---|---|
| `http:`/`https:` with a host, or `//host` (taken as https) | the policy (kind `Link`), default allow | the resolver for a matching `Content-Location`, else the policy, default block |
| `mailto:`, `tel:` | kept | removed |
| `cid:` | removed | the resolver (`ResolveReference`), else removed |
| relative reference | removed | the resolver (`Content-Location`, RFC 2557), else removed |
| `#fragment` | kept, prefixed | removed |
| `data:` | removed | kept only for `image/png gif jpeg jpg webp bmp avif x-icon`, base64, nothing else in the header |
| `javascript:`, `vbscript:`, `file:`, `blob:`, any other scheme, malformed schemes (`java script:`, `http:foo`) | removed | removed |

SVG is never accepted as a data image.

## CSS

`style` elements and `style` attributes are parsed and rewritten.

- **Properties** are allowlisted (colour, background, border, margin, padding, size, font, text, list, table, display, visibility, float, overflow
  and the like). `position`, `z-index`, `transform`, `filter`, `mask`, `content`, `cursor`, `animation`, `transition`, `behavior`, `-moz-binding`,
  `clip-path`, custom properties (`--x`) and `var()` are not allowed. `!important` is kept.
- **Values** are rebuilt from tokens: identifiers, numbers, dimensions, percentages, colours, strings, commas, `/ + - *`, parentheses, and the functions
  `rgb rgba hsl hsla hwb lab lch oklab oklch color calc min max clamp` and the gradient functions. `url()` goes through the URL rules above.
  Any other function (`expression`, `image-set`, `attr`, `env`, ...) removes the declaration. Escapes are decoded before they are compared, so
  `u\72l(` and `expr\65ssion(` are seen for what they are.
- **At-rules**: `@media` (with a plain media query) is kept, `@font-face` is kept when it has `font-family` and a `src` whose URLs pass the policy
  (kind `Font`); `@import` is removed and reported as a stylesheet reference; everything else (`@supports @namespace @keyframes @page @charset
  @layer @container`, nested rules) is removed.
- **Selectors** are rebuilt and confined to the wrapper: each one is prefixed `.omx `; `html`, `body` and `:root` name the wrapper itself.
  Allowed: type, class, id, `*`, attribute selectors (not on `class`, `id`, `style`), the combinators, the pseudo-classes `hover focus active visited
  link first-child last-child only-child first-of-type last-of-type only-of-type empty checked disabled enabled focus-within focus-visible`,
  `nth-*()` with a plain argument, `:not()` of these, and the pseudo-elements `first-line first-letter before after selection marker`. A
  selector that uses anything else is removed.
- Every `<` in the output is written as the escape `\3C `, so no style content can close the element.
- The wrapper rule `.omx-box{display:block;position:relative;overflow:hidden;contain:paint;isolation:isolate}` clips the message to its box, so
  negative margins and large boxes cannot cover the host page.

## Output shape

```html
<style>…box rule and the message's filtered rules…</style><div class="omx-box"><div class="omx" style="…body attributes…">…body…</div></div>
```

`bgcolor`, `text`, `background` and `style` on `body` become the style of the inner `div`.

## The remote-content policy

Every way the markup names something to fetch is reported in `SanitizedHtml.RemoteReferences` (URL, kind, outcome, where it was found) and, for
kinds the output can carry, asked of the policy:

| Vector | Kind | In the output |
|---|---|---|
| `img src`, `img srcset`, `picture/source srcset`, CSS `list-style-image` | `Image` | kept / blocked / rewritten |
| `background` attribute (`body table tr td th ...`), CSS `background*` | `Background` | kept / blocked / rewritten |
| `@font-face src` | `Font` | kept / blocked / rewritten |
| `a href` and links in plain text | `Link` | kept / blocked / rewritten (default allow) |
| `link rel=stylesheet`, `@import` | `Stylesheet` | removed (`ElementDropped`) |
| `video poster` | `Image` | element removed (`ElementDropped`) |
| `video/audio/source/track src`, `iframe`, `embed`, `object`, `input type=image`, `bgsound`, SVG `image`/`use` | `Media` | element removed (`ElementDropped`) |

`SanitizerOptions.RemoteContent` (`Allow` or `Block`, default **Block**) sets the default for everything except links. `SanitizerOptions.Policy` is
called for each occurrence with `(Url, Kind)` and returns `Allow`, `Block` or `Rewrite(url)`; it overrides the default, a throwing policy blocks.
A rewrite target must be http(s), mailto, tel or a data image; relative and application-scheme URLs are accepted when `AllowRewriteToAnyUrl` is true
(the default), never `javascript:`, `vbscript:`, `file:`, `blob:` or a non-image `data:`.

**Tracking indications** (`TrackingIndications`) are information only. An `img` with a remote URL is flagged when its width or height (attribute or style)
is at most one pixel (`TinySize`), its style hides it (`display:none`, `visibility:hidden`, `opacity:0`: `Hidden`), or its URL contains a known
open-tracking pattern (`KnownPattern`; the list is in `SanitizerContext`). Flagged images are still blocked or allowed by the policy like any other.

## `cid:` and `Content-Location`

`ToDisplayHtml` supplies the resolver: `cid:x` (percent-decoded) is matched to an attachment's `ContentId` (exact, then ignoring case); a relative
reference, or an absolute URL, is matched to `IMailAttachment.ContentLocation` (equal, or one a `/`-tail of the other, RFC 2557 §5; exact, then ignoring
case). Both MIME (`Content-ID`, `Content-Location`) and MAPI items (`PidTagAttachContentId`, `PidTagAttachContentLocation`) work. The URL is a `data:` URI
(`ReferenceForm.DataUri`: the type comes from the bytes, PNG/GIF/JPEG/WebP/BMP only, at most `MaxInlineImageBytes` per image and `MaxInlineTotalBytes`
in all) or what `AttachmentUrl` returns (`ReferenceForm.Callback`). Resolved attachments leave the attachment list. Unresolved references are listed in
`UnresolvedReferences` and removed.

## Display HTML

`ToDisplayHtml` writes `omx-display`: a small stylesheet for its own chrome, an optional header block (subject, From, To, Cc, Bcc, date, all escaped), the
`CalendarCard` hook's HTML if the caller supplied one (written as given, not sanitised), the body, and the attachment list (name, type, size).

- The **HTML body** is used unless `PreferText` is set or there is none; otherwise the **text body**.
- **Plain text** is escaped, one block per line (`white-space:pre-wrap`). With `format=flowed` (RFC 3676) lines ending in a space are joined with the next line
  of the same quote depth (the space is deleted when `delsp=yes`), the signature separator `-- ` never flows, quote depth is the count of leading `>`
  and becomes nested `blockquote`s, and one leading space (space-stuffing) is removed.
- **Links in text** are only `http://` and `https://` addresses, with sentence punctuation and an unbalanced closing bracket left outside; each goes through the
  policy as a `Link`. `www.` and bare addresses are not linked.
- A body that cannot be decoded is shown as empty.

## Limits

`MaxInputLength` (8,000,000 chars; the rest is ignored), `MaxDepth` (256; deeper subtrees are removed), `MaxNodes` (500,000; output stops), `MaxCssLength` (1,000,000
chars), `MaxReported` (1,000 entries in each report list), URLs longer than 8,192 chars (remote) are removed. Reaching one sets `Truncated` and adds a
`LimitReached` change. OneHtml's own parse limits are set from these.

## Threat model

Trusted: the library, the host page that inserts the output, the caller's options and callbacks (policy, resolver, calendar card hook).
Untrusted: the HTML, its CSS, its URLs, attachment bytes and names.

Goals, in this order:

1. no script runs and no markup the browser treats as active reaches the output (script, handlers, `javascript:` and its kin, frames, plugins, forms, SVG/MathML);
2. nothing is fetched unless the policy allows it, so opening a message does not tell the sender;
3. the message cannot restyle, cover or read the host page (scoped CSS, clipped box, prefixed names, no `position`);
4. the markup survives a serialise/parse round trip, so a later `innerHTML` copy cannot change its meaning.

Not goals: sender authenticity, phishing detection (links keep their text; a link can say one thing and go elsewhere), spoofed display names, text
direction tricks, large or slow images, and content a policy allows (an allowed remote image is the sender's to serve).

## What the output assumes

The output is a fragment. The host should still:

- show it in an `iframe` with `sandbox` (no `allow-scripts`, no `allow-same-origin`) or a web view with script disabled, and a `Content-Security-Policy`
  (`default-src 'none'; img-src data: …; style-src 'unsafe-inline'`), as defence in depth;
- not insert it into a context other than an element's content (for example inside a `table` or an attribute);
- keep user interface that depends on `om-`-prefixed ids and classes out of the message's namespace.

## How it is checked

- `testdata/sanitizer/xss.txt`: 233 vectors we wrote from the categories in public evasion lists (script and handler variants, URL schemes and their encodings,
  CSS, SVG/MathML, parser-differential and mutation cases, every remote-fetch vector). `SanitizerTests` sanitises each, asserts the output holds only
  allowlisted elements, attributes and URL schemes, and that sanitising it again changes nothing dangerous.
- `tools/sanitizer/browser-check.sh` renders every vector in headless Chromium, raw (the control), sanitised with the default policy, and sanitised
  with remote content allowed, against a logging server: script calls and requests are counted.
- `tools/sanitizer/compare-dompurify.sh` compares with the released DOMPurify on the same vectors.
- `SanitizeTool corpus` sanitises real message bodies and compares the visible words before and after, apart from the elements removed above.
- Fuzz targets `sanitizer` (invariants on the output) and `sanitizer-reparse` (OneHtml's serialiser must be idempotent on the output).


<!-- https://recursiveai.org/onemail/docs/calendar/ -->

# ICS and vCard: content lines and component trees

Normative notes for `RecursiveAI.OneMail.Calendar` (BCL only). Behaviour on ambiguous input is in [`ambiguities.md`](https://recursiveai.org/onemail/docs/ambiguities/), calendar section.

## Layers

1. **Content lines** (`…Calendar.ContentLines`): `ContentLineReader` is a forward-only reader over a byte span. It yields `ContentLine` values (group, name, parameter text, value; offsets; the raw bytes). It unfolds (RFC 5545 §3.1: CRLF, LF or lone CR, then one space or tab), and for vCard 2.1 joins quoted-printable soft breaks and unindented base64 blocks. Parameters are read with `ContentParameterEnumerator`: quoted values, comma lists, bare parameters. `ContentLineWriter` folds at 75 octets without splitting a UTF-8 sequence, quotes parameter values that contain `, ; :` and encodes `^`, `"` and line breaks per RFC 6868. After the first folded line the reader allocates nothing per line (one reusable unfold buffer).
2. **Component trees** (`…Calendar`): `ContentDocument.Parse` builds `Component`/`ContentProperty` trees. Every line keeps its source bytes; an unchanged line is written as read, a changed or new line is written from its parts. Untouched documents therefore write back byte for byte, whatever was malformed. Blank lines, stray lines, an unmatched `END`, a byte order mark and anything cut off by a limit are kept as items. Editing a property, its parameters or a component name marks only that line modified.
3. **Typed views** (`…Calendar.Ics`, `…Calendar.VCard`): `IcsComponent` and `VCardView` read RFC 5545/7986/5546 and vCard 2.1/3.0/4.0 properties on demand. A value that does not parse yields null and stays in the tree. Values: `IcsTime` (DATE, floating, UTC, zoned with the `TZID` carried, never resolved), `IcsDuration` (units kept as written), `IcsPeriod`, `IcsUtcOffset`, `RecurrenceRule` (parsed parts, unknown parts kept), `VCardDate` (partial dates). Recurrence expansion and time-zone resolution are separate.

## Defects

`CalendarDefect(Kind, Offset, Length, Depth)`; `Ambiguous*` kinds mark a documented choice between valid readings, `IsLimit` kinds mark a resource limit. Parsing never throws on content. Offsets are absolute byte offsets in the input. `BareLineFeed`, `BareCarriageReturn` and `AmbiguousDoubleCarriageReturn` (CR CR LF, read as one line end) are reported once, at the first line. The kinds are listed with their meaning in `CalendarDefectKind`.

## Limits (`CalendarLimits`, on by default)

| Limit | Default | When exceeded |
|---|---|---|
| `MaxLineBytes` | 16 MiB | the value is cut there; the source bytes are kept |
| `MaxDepth` | 32 | deeper `BEGIN`/`END` lines become ordinary properties |
| `MaxComponents` | 1,000,000 | further `BEGIN`/`END` lines become ordinary properties |
| `MaxProperties` | 8,000,000 | the rest of the input is kept as one malformed block |
| `MaxTotalBytes` | 512 MiB | the rest of the input (from a line end) is kept as one malformed block |

Nothing is lost when a limit applies: untouched documents still write back identically.

## Charsets

UTF-8 is the format. A `CHARSET` parameter (vCard 2.1) selects the charset for that value from the encodings the .NET runtime ships (including the Windows and ISO code pages through the runtime's code-page provider); an unknown name is a defect and the bytes are read as UTF-8 or Windows-1252. Changed lines with a `CHARSET` are written in that charset.

## Recurrence and time zones

Expanding RRULE/RDATE/EXDATE/EXRULE and overrides into occurrences, and resolving `TZID`s (VTIMEZONE, IANA, Windows names): `docs/recurrence.md`.

## Checks

`tools/calendar/README.md` describes the corpus, the round-trip check, the oracle comparison and the fuzz targets (`contentlines`, `calendar`).


<!-- https://recursiveai.org/onemail/docs/recurrence/ -->

# Recurrence expansion and time zones

Normative notes for `RecursiveAI.OneMail.Calendar.Recurrence` and `.TimeZones`. The entry points are
`CalendarRecurrence.Expand(calendar, window, options, report)` (a VCALENDAR with its events, VTIMEZONEs and overrides) and
`RecurrenceSet` (one master component plus its overrides). Expansion reads the component trees and never changes them, so
untouched ICS still writes back byte-identical. It never throws on content: every repair, guess or cut-off is a typed
`RecurrenceDefect` collected in an `ExpansionReport`.

Sources: RFC 5545 §3.3.5, §3.3.10, §3.8.5.1–3.8.5.3, §3.6.5 (VTIMEZONE); RFC 7808 (`TZID-ALIAS-OF`); RFC 7529 (RSCALE, refused).
Windows zone names come from the Unicode CLDR `windowsZones.xml` (see THIRD-PARTY-NOTICES).

## What an occurrence is

An `Occurrence` has the start and end as the component writes them (DATE, floating, UTC, or local time in its zone), the UTC
instants where there is one (`StartUtc`/`EndUtc` are null for dates, floating times and unresolved zones), the origin (`Start`,
`Rule`, `Date`, `Period`, `Override`, `FutureOverride`) and the component it came from. Occurrences come in start order.

## The rule engine

The engine follows the expansion table of RFC 5545 §3.3.10 period by period (year, month, week, day, hour, minute, second, in
steps of `INTERVAL`): a BYxxx part expands where the table says *Expand* and filters where it says *Limit*. BYSETPOS selects
within the instants of one period after the other parts. `WKST` fixes the week for `WEEKLY` and `BYWEEKNO`. Negative BYxxx
values count from the end. `COUNT` counts rule instants before EXDATE/EXRULE removal, so `COUNT=4` with one EXDATE gives three
occurrences. `UNTIL` is inclusive: a UTC UNTIL is an instant; a DATE UNTIL covers its whole day; a local UNTIL is a wall-clock
value (so a UTC UNTIL on a floating series is compared as if the wall clock were UTC).

Defaults and choices (each is provisional until a client is observed; the ones where real implementations disagree
have an entry in `ambiguities.md`, section "Recurrence and time zones"):

- **DTSTART is always the first instance**, even when it does not match its rule (RFC 5545 §3.8.5.3 leaves that undefined).
  Defect `AmbiguousUnsynchronizedStart` (R3). It counts toward `COUNT`.
- **Parts that the table marks N/A** for the frequency (for example `BYWEEKNO` with `MONTHLY`) filter, not expand (R2).
- **A BYDAY ordinal where it is not allowed** (not MONTHLY/YEARLY, or with BYWEEKNO) is ignored: `OrdinalIgnored`.
- **`BYWEEKNO` with no day part** takes the weekday of DTSTART (R1).
- **Weekly `BYSETPOS`** counts in the whole week, including the part before DTSTART in the first week (R7).
- **`EXDATE` as a DATE on a date-time series** removes every occurrence on that day (R6, `AmbiguousExdateType`).
- **`UNTIL` of another value type than DTSTART** is compared as described above (R4, `AmbiguousUntilType`).
- **Sub-day frequencies** (`HOURLY`, `MINUTELY`, `SECONDLY`) step in wall-clock time of the series' zone. Two wall times that
  land on one instant (the repeated hour) stay two occurrences.
- **A rule with a time-of-day frequency, or BYHOUR/BYMINUTE/BYSECOND, on a DATE start** keeps dates and collapses repeats:
  `TimeRuleOnDate`.
- **`RSCALE` other than GREGORIAN** is not expanded: `UnsupportedRScale`. An unparseable rule or an out-of-range part
  (`BYMONTHDAY=0`) ignores that rule: `InvalidRule`.
- **EXRULE** (obsolete, RFC 2445) is honoured, with RDATE and EXDATE, in the order RFC 5545 §3.8.5.3 gives.
- **Overrides** (`RECURRENCE-ID`) replace the occurrence they name; `RANGE=THISANDFUTURE` moves the later ones by the same
  shift (`FutureOverride`). An override whose master is missing is yielded alone (`OrphanOverride`); two overrides for one
  instance: the later in the file wins (`DuplicateOverride`).
- **Durations** follow §3.8.5.3: an exact length from DTEND/DUE; nominal days plus an exact time part from DURATION, so a
  one-day event across a clock change ends at the same wall time.

## Daylight-saving transitions

For a zoned series (`DTSTART;TZID=…`) the local time of each instance is converted to an instant through the zone:

| Local time | Policy |
|---|---|
| Exists once | converted as is |
| Skipped (the hour clocks jump over) | **Shift** (default): read with the offset before the gap, so 02:30 becomes 03:30 (RFC 5545 §3.3.5). `SkippedTimePolicy.Skip` drops it without counting it against COUNT (the strict reading of §3.3.10). Defect `AmbiguousSkippedLocalTime` per occurrence (R5) |
| Happens twice | the first one is used (RFC 5545 §3.3.5). Defect `AmbiguousRepeatedLocalTime` per occurrence (R5) |

Inside a skipped hour, occurrences generated from different wall times may map out of instant order; the output is ordered
by the instant it computes, ties by generation order.

## Windows

`OccurrenceWindow` is half-open `[Start, End)` in UTC. `WindowMode.Overlapping` (default) returns occurrences that start before
the end and finish after the start (a zero-length one must start inside); `StartingIn` returns those that start inside.
Floating times and DATE values have no instant, so they are compared by their wall-clock value as if it were UTC.
Rules without `COUNT` jump to the window instead of walking from DTSTART (the result is the same); rules with `COUNT` must
generate the instances before the window.

## Limits (on by default, in `ExpansionOptions`)

| Limit | Default | When exceeded |
|---|---|---|
| `MaxOccurrences` | 100,000 | enumeration ends, `OccurrenceLimit` |
| `MaxIterationsWithoutHit` | 1,000,000 | that rule ends, `IterationLimit` (rules that never match, like `BYMONTH=2;BYMONTHDAY=30`, end here or at year 9999) |
| `MaxInstancesScanned` | 5,000,000 | that rule ends, `ScanLimit` (instances generated before the window, for rules with COUNT) |

A rule that can never match still yields DTSTART (R3) and then ends at a limit.

## Time zones

`TimeZoneResolver.Resolve(tzid)` takes the first hit of this list, and records `UnknownTimeZone` once per TZID if there is none:

1. **A VTIMEZONE in the file** with that `TZID` (`ZoneSource.VTimeZone`). Its STANDARD/DAYLIGHT observances (with their own
   RRULE/RDATE) are expanded, as are fixed ones. A VTIMEZONE without usable observances is `InvalidTimeZone`; then
   `TZID-ALIAS-OF` (RFC 7808) is tried.
2. **UTC names** (`UTC`, `Etc/UTC`, `GMT`, `Z`, …).
3. **The runtime's IANA database** (`ZoneSource.Iana`), when `UseRuntimeDatabase` is true (default).
4. **A vendor-prefixed id** (`/softwarestudio.org/Olson_20011030_5/America/New_York`): the longest tail of up to three segments
   found in the runtime database. Defect `AmbiguousZoneName`.
5. **A Windows zone name** (`Eastern Standard Time`), through the committed CLDR table to the default-territory IANA zone
   (`ZoneSource.WindowsName`). Defect `AmbiguousWindowsZone`: the same name can mean different IANA zones by region, and the
   table keeps one. Off with `UseWindowsNames = false`. This step runs *before* step 3 for any name in the table, so the result
   is the same whether the runtime knows Windows ids (Windows) or not (Linux, macOS).

Every runtime lookup (steps 3 to 5) asks `TimeZoneInfo` for the name first. If the runtime does not know an IANA name, as on
Windows without ICU (and `InvariantGlobalization` removes ICU), it asks again under the Windows zone id CLDR assigns that IANA name
(the default-territory row first, then a territory row; 444 IANA names). The zone keeps its IANA `Id` and `ZoneSource.Iana`, with no
defect. A name known to neither side is `UnknownTimeZone` (Z4), never an exception. Names that are IANA aliases CLDR does not list
(`US/Eastern`) resolve on Windows only if the runtime itself knows them.

**Windows zone data is thinner than IANA's.** The Windows registry carries the current rules and a few years of history, not IANA's
full past: US dates before 2007 use today's rules (1997's changes fall on the wrong days), and zones that changed rules long ago
differ similarly. Expansion of a recurring event over such dates on Windows can therefore differ in UTC instants (never in wall-clock
values) from the same file on Linux or macOS. A VTIMEZONE in the file always wins and gives the file's own history; use it, or run
without the runtime database (`UseRuntimeDatabase = false`), where results must not depend on the host.

An unresolvable TZID leaves the series floating (no instants are claimed). Names that would reach the machine's own settings
(`localtime`, `posixrules`, `posix/…`, `right/…`), paths and names with other characters than real zone names use are refused
before any lookup.

A VTIMEZONE in the file beats the database: the file says what its author meant, and a Windows-style custom zone may differ
from IANA. If the file has none, expansion falls back to the database and says so with the defects above.

**Browser (WASM).** The runtime's database may be absent in the browser. Lookups are guarded by try/catch and return "unknown",
never an exception. Set `TimeZones = new TimeZoneResolverOptions { UseRuntimeDatabase = false }` there and rely on the file's
VTIMEZONEs; the AOT smoke app exercises that configuration. The browser smoke app also expands a rule through the runtime's database (`America/New_York` across the spring-forward day)
and through a VTIMEZONE; both give the same instants in headless Chromium, so the database is present in the browser runtime as built.

## Method of the differential check

`tools/calendar/run_expand_diff.sh` expands every recurring component of the calendar corpus (see `tools/calendar/README.md`) and of
the committed generated fixtures in a fixed window, with ours (`RecurrenceDump expand`) and three oracles run as black-box
processes, and compares the sorted start lists (`compare_expand.py`):

- Python `recurring-ical-events` (on `icalendar`; the one that applies RECURRENCE-ID and EXDATE/RDATE like a calendar client),
- Python `python-dateutil` `rrule` driven by our glue (`py_expand.py du`: a rule engine with no overrides or EXRULE),
- Ical.Net, from its NuGet package (`tools/calendar/oracles/OracleExpand`), public API only.

Versions and licences are in THIRD-PARTY-NOTICES. Components an oracle cannot expand (errors, timeouts) are counted and never
compared. Every remaining difference must get an id from `compare_expand.py`, defined in the section "Recurrence and time zones"
of `ambiguities.md` (R1–R7 resolve a documented ambiguity; `L-*` are limits of one oracle). Anything left unexplained is a bug.
The generated grid (`tools/calendar/make_recurrence_fixtures.py`, fixed seed) holds every subset of BYxxx parts per frequency,
DST edges, window and override cases; RFC 5545 §3.8.5.3's examples run as unit tests (`RecurrenceRfcTests`).

`RecurrenceDump rate` and `RecurrenceDump ratepure` measure throughput and allocations per occurrence; `fuzz/run.sh recurrence`
fuzzes rule parsing and expansion with small limits.


<!-- https://recursiveai.org/onemail/docs/jcal-xcal/ -->

# jCal, jCard, xCal and xCard

Normative notes for `RecursiveAI.OneMail.Calendar.Forms` (BCL only). The standards are RFC 7265 (jCal), RFC 7095 (jCard), RFC 6321 (xCal) and RFC 6351 (xCard). Choices the standards leave open are the `J` rows in [`ambiguities.md`](https://recursiveai.org/onemail/docs/ambiguities/), calendar section.

## What the layer does

`JsonForms.ToJson` and `XmlForms.ToXCal`/`ToXCard` write a `ContentDocument` (the ICS or vCard tree) as jCal/jCard or xCal/xCard. `JsonForms.FromJson` and `XmlForms.FromXml` read the forms into a `ContentDocument` again. Both directions go through one model (components, properties with a type and values, parameters), so the JSON and XML forms behave the same.

Reading never throws on content. Every repair is a `FormDefect` with a byte offset (JSON), line and column (XML) and a component path. Writing never fails: what a form cannot carry is a `FormLoss` with a kind, the property path and a count, returned beside the bytes. The text document is not changed.

## Mapping

- Names are lower case in JSON and XML and upper case again in text. A vCard group prefix becomes the `group` parameter (RFC 7095 §3.3.1.2); the same is done for the rare group on iCalendar data.
- The value type is `VALUE` when given, otherwise the property's default type from RFC 5545/7986/6350. Values take the JSON shapes of the RFCs: structured text (`N`, `ADR`, `REQUEST-STATUS`, `GEO`) as arrays, recurrence rules as objects with lower-case part names and a day string for `WKST`, periods as two-element arrays, `TYPE`/`PID`/`SORT-AS` as arrays, numbers as JSON numbers.
- Dates, times and offsets use the extended spelling in jCal, jCard and xCal (`2026-03-28T02:30:00Z`, `-05:00`). xCard uses the basic spelling its schema shows for vCard dates; `date-and-or-time` is written as the `date`, `date-time` or `time` element its value fits.
- xCard is vCard 4.0: `VERSION` is not carried, and a 3.0 card is written with the 4.0 forms of the properties that differ (`GEO` as a geo URI, `TYPE=pref` as `PREF=1`).
- Inline binary (vCard 2.1/3.0 `PHOTO`, `LOGO`, `SOUND`, `KEY`, and extension properties with base64) becomes a `data:` URI, because jCard and xCard have no binary type; it goes back to `ENCODING=b`/`BASE64` for 2.1/3.0. iCalendar `ATTACH` with `VALUE=BINARY` keeps the `binary` type and its `ENCODING` parameter.
- XML elements that are not in the xCal/xCard namespaces, under a property, become an `XML` property (RFC 6351 §6, RFC 6321 §4.1) and come back as elements.
- A value that does not parse as its type is carried as `unknown` with the raw text, and the declared `VALUE` stays as a parameter so the text writes back as read. `ValueNotRepresentable` reports it.
- A vCard 2.1 line break needs quoted-printable; the writer adds `ENCODING=QUOTED-PRINTABLE;CHARSET=UTF-8` for it and never ends a folded physical line in `=` (which a 2.1 reader takes for a soft break).

## Loss kinds (`FormLossKind`)

`Folding`, `NameCase`, `ItemOrder`, `LineNotCarried` (blank, malformed or stray lines, which have no place in the forms), `RedundantValueParameter`, `DuplicateParameterMerged`, `CharsetRemoved`, `TransferEncodingRemoved`, `BareParameterAsType`, `InlineBinaryAsDataUri`, `ValueNotRepresentable`, `StructuredPadded`, `ExtraComponentsDropped`, `VersionNotCarried`, `GroupNotCarried`, `ComponentNotCarried`, `PropertyNotCarried`, `ParameterNotCarried`, `TypeNameNotCarried` (a name that is not an XML name cannot be an element), `InvalidXmlCharacter` (replaced by U+FFFD), `VersionSpecificForm`, `AmbiguousValueType`.

## Limits (`FormLimits`, on by default)

| Limit | Default | Defect |
|---|---|---|
| `MaxInputBytes` | 256 MiB | `InputTooLarge`, checked before parsing |
| `MaxDepth` | 128 nesting levels (JSON arrays or XML elements; the 32-level component limit needs about 100) | `DepthLimitExceeded` |
| `MaxNodes` | 16,000,000 | `NodeLimitExceeded` |
| `MaxStringChars` | 16 Mi characters in one string | `StringTooLong` |

XML is read with the document type declaration prohibited and no resolver, so there is no entity expansion and no external access (`DtdProhibited`). JSON accepts comments and trailing commas as a repair.

## Checks

- RFC examples: RFC 7265 Appendix B, RFC 7095 Appendix B, RFC 6321 Appendix B, RFC 6351 §4 and the section examples, in both directions (`testdata/calendar/forms/rfc`; the corrections made to the RFC text are in `ERRATA.txt`).
- `FormsDump check <root>…` converts every text file of a corpus to JSON and XML, reads the forms back, compares property by property, and checks that the second form is the same as the first. Over the corpora (biweekly, ez-vcard, ical.js, libical, sabre/vobject, Thunderbird, our generated and recurrence fixtures: 2,000+ files, about 130,000 properties) every file is stable in JSON; the only comparison differences are listed below. Differences in XML that follow a reported loss (`InvalidXmlCharacter`, `PropertyNotCarried`, `ParameterNotCarried`, `ComponentNotCarried`, `TypeNameNotCarried`) are counted apart: the file lost something XML cannot hold, and a second pass starts from a different file.
- Oracle comparison, `tools/calendar/oracles/forms/run_forms_diff.sh`: the same inputs through released ical.js (jCal, jCard), Python icalendar (jCal), biweekly (jCal, xCal) and ez-vcard (jCard, xCard), in three directions: the oracle writes the form from the text, the oracle reads our form and writes it again, and we read the oracle's form and write it again. Every difference is a rule in `tools/calendar/oracles/forms/explained.tsv`; the run reports unexplained classes, and the count is zero. The rules are the `X`, `Y`, `V`, `W` and `Z` rows: the oracle rejects the input (X1), we keep what the oracle repairs or guesses (X2), the oracle departs from the RFC (ical.js numeric `WKST`, biweekly period strings and basic spellings), the oracle normalises (URI scheme case, UTC conversion, durations), or the oracle writes vCard 4.0 from a 3.0 card (ez-vcard).
- There are two independent oracles for jCal (ical.js, icalendar) and for jCard (ical.js, ez-vcard) on the corpus. xCal and xCard have one oracle each: biweekly and ez-vcard. No other released reader of the XML forms is known to us; xCal and xCard are otherwise checked against the RFC examples and our own round trips.
- Fuzz targets `jcal-jcard` and `xcal-xcard` (`fuzz/`): reading never throws, the result writes in all forms, and the JSON form of it reads back to itself.


<!-- https://recursiveai.org/onemail/docs/calendar-items/ -->

# Outlook calendar and task items, and their iCalendar conversion

Normative notes for `RecursiveAI.OneMail.Calendaring` (the typed views over a `MapiMessage`) and
`RecursiveAI.OneMail.Conversion.OxcIcal` (MS-OXCICAL). Specs: MS-OXOCAL, MS-OXOTASK, MS-OXCICAL, MS-OXPROPS. Losses are reported as in
[`conversion.md`](https://recursiveai.org/onemail/docs/conversion/); choices on ambiguous input are rows OC1-OC6 in [`ambiguities.md`](https://recursiveai.org/onemail/docs/ambiguities/).

## Typed views (read)

- `OutlookAppointment.TryCreate(message)` returns a view for the calendar message classes, or null: `IPM.Appointment`, the meeting classes
  (`IPM.Schedule.Meeting.Request`, `.Resp.Pos`, `.Resp.Tent`, `.Resp.Neg`, `.Canceled`, `.Notification.Forward`) and the exception-object class.
  `Kind` says which. The view reads the named properties on demand; nothing is copied, and the property bag stays the truth.
- Fields: subject, location, start and end (UTC instants) with `StartLocal`/`EndLocal` in the item's own zone, all-day, busy status, sensitivity,
  importance, response status, sequence, meeting type, reminder, global object ID and clean global object ID, organizer, attendees (recipient
  table rows with role and response), the recurrence (`AppointmentRecurrencePattern`), the time zone (`OutlookTimeZone`: TZDEFINITION when
  present, else the TIME_ZONE_STRUCT, and a `VTIMEZONE` component built from it), modified instances (the exception attachments joined
  with the `ExceptionInfo` records) and deleted instance dates.
- `OutlookTask.TryCreate(message)` reads `IPM.Task`: status, percent complete, start and due date, owner, assigner state, completion, task
  recurrence (the same `RecurrencePattern`).
- `MapiMailItem.Appointment` and `MapiMailItem.TaskItem` open the views from the shared view; `Kind` stays `Appointment`, `Task` or `Message`.
- Blob readers (`AppointmentRecurrencePattern.TryParse`, `TimeZoneDefinition.TryParse`, `TimeZoneStruct.TryParse`) are lenient: they never
  throw, return `CalendaringDefect`s with the byte offset, and are bounded by `CalendaringLimits` (exceptions, deleted/modified instances,
  rules, string lengths). Writing a parsed blob back gives the same bytes for well-formed input.
- A text/calendar part inside MIME (RFC 6047 iMIP) is a `MimePart`: `MimeMailItem.CalendarParts` lists every one, attachments and
  alternatives of the body alike (a text alternative is not in `Attachments`, RFC 2046 5.1.4). `IcsToMapiConverter` turns its content into the item.

## Recurrence

An Outlook recurrence (`RecurrencePattern`: frequency, pattern type, period, first date/day-of-week, end) maps to one RRULE by the templates of
MS-OXCICAL 2.1.3.2.2; `OutlookRecurrence.ToRRule` writes it and `OutlookRecurrence.Days` walks the valid days exactly as MS-OXOCAL 2.2.1.44.1
describes (used as the independent check in the tests). Expansion goes through `RecursiveAI.OneMail.Calendar.Recurrence`; there is no
second engine. Moved/changed instances become `RECURRENCE-ID` overrides, deleted ones `EXDATE`. The pattern's start offset and the series zone
give `DTSTART;TZID=`. `WKST` is always written (OC4) and `INTERVAL` is omitted when 1.

## Item to iCalendar (`MapiToIcsConverter`)

`Convert(MapiMessage)` returns the VCALENDAR bytes as a `ContentDocument` and a `LossReport`. Appointments and meeting objects give a VEVENT
(`METHOD` from the meeting type: REQUEST, REPLY, CANCEL; PUBLISH for a plain appointment; none for a forward notification), tasks a VTODO.
Mapped: UID (from the global object ID, OC5), DTSTAMP, DTSTART/DTEND with TZID or UTC or DATE (all-day), SUMMARY, LOCATION, DESCRIPTION,
CLASS, PRIORITY, TRANSP and `X-MICROSOFT-CDO-BUSYSTATUS`, SEQUENCE, CREATED, LAST-MODIFIED, ORGANIZER, ATTENDEE (CN, ROLE, PARTSTAT, RSVP),
VALARM from the reminder, RRULE/EXDATE/RDATE, exceptions, VTIMEZONE from the item's definition. Everything else is in the report
(`NoIcalMapping`, store bookkeeping as `Internal`).

## iCalendar to item (`IcsToMapiConverter`)

`Convert(IcsComponent)` converts the first component; `ConvertAll` gives one result per VEVENT series (overrides with the same UID become exception
attachments of their series) and VTODO. Zones: a TZID with a `VTIMEZONE` gives a TZDEFINITION by sampling the zone through the item's year (two
changes = a DST rule, OC6); a known Windows or IANA name is resolved through the `.Calendar` zone layer; otherwise UTC with `UnknownTimeZone`.
A floating time or a date takes UTC (`FloatingTimeAssumedUtc`, OC3). A task's UID must fit the 16-byte `PidLidTaskGlobalId`: a UID of ours
(32 hex digits, `@onemail`) comes back unchanged, any other is replaced by a derived GUID and reported (`UidReplacedByGuid`).

## How it is checked

- Spec blobs: the weekly and daily examples of MS-OXOCAL 4.1.1 read and write back byte for byte; TIME_ZONE_STRUCT offsets for Pacific, Sydney
  (southern hemisphere) and a zone without DST.
- 18 generated recurrences (daily, weekly, monthly, monthly-nth, yearly, yearly-nth; ending never, by count, by date) in three zones (UTC,
  Pacific, Sydney): the expansion of the RRULE we write equals the spec's day walk, 54 of 54.
- The same 54 ICS files (`testdata/calendar/oxocal/generated`, written by `ONEMAIL_OXOCAL_OUT=<dir>` on the tests) through
  `tools/oxocal/oracle-checks.sh`: dateutil 2.9.0.post0, recurring-ical-events 3.8.2 and Ical.Net 5.2.3 expand them; Python icalendar 7.3.0 and
  Ical.Net parse every file we produce.
- Item to ICS to item and ICS to item to ICS (`tools/oxocal/OxocalTool roundtrip|ics`, and the `OxcIcalRoundTripTests`): mapped properties equal,
  the rest in the report, nothing silent.
- `tools/oxocal/compare_extract_msg.py`: subject, location, attendees, task status and dates against extract-msg 0.55.0.
- Fuzz target `oxocal`: recurrence blobs, TZ structures, and conversion over patched iCalendar and damaged items.


<!-- https://recursiveai.org/onemail/docs/calendar-card/ -->

# The calendar card

`RecursiveAI.OneMail.Display.CalendarCard` turns a calendar object in a message into the compact card mail clients show above the body, as an
HTML fragment (`ToHtml()`) and as plain text (`ToText()`). It is format core: it runs in the browser runtime and under NativeAOT.

## Inputs

All inputs end up as an iCalendar object (`IcsComponent`), so there is one card builder:

| Source | Entry point | Path |
|---|---|---|
| iCalendar bytes, an `.ics` file | `CalendarCard.FromIcs(bytes)` | parsed by `OneMail.Calendar` |
| a `text/calendar` part (RFC 6047) | `FromMime(part)` | decoded content, then as above |
| an Outlook appointment or meeting item from MSG, PST, OST, TNEF | `FromMapi(message)` | MS-OXCICAL export (`MapiToIcsConverter`), plus `PidLidNetShowUrl` as `X-MS-OLK-NETSHOWURL` |
| an Outlook for Mac appointment | `FromItem(olmItem)` | `OlmToCalendarConverter` |
| any shared-view item | `FromItem(item)` | the item's `text/calendar` parts, or the Outlook item |
| a parsed component | `Create`, `CreateAll` | one card per event series (same UID; overrides join their master) |

A message that is not a calendar item gives no card. To-dos get a card too (their due time stands in for the end). Nothing is wired into the
display HTML of a whole message; a caller composes the card above it.

## What the card says

- **Action** from METHOD (RFC 5546): `REQUEST` "Invitation" ("Updated invitation" when SEQUENCE is above 0), `CANCEL` "Cancelled", `REPLY` "*name* accepted /
  declined / tentatively accepted" (the first attendee with a PARTSTAT), `COUNTER` "Counter-proposal", `DECLINECOUNTER` "Counter-proposal declined",
  `ADD`, `REFRESH`, `PUBLISH` and no method "Event"; a method the enum does not name "Calendar item". STATUS `CANCELLED` or `TENTATIVE` is shown as a badge
  (a `CANCEL` method is cancelled whatever STATUS says).
- **When**: "Mon, 30 Nov 2026 14:00–15:00 (Europe/Paris, UTC+01:00)", "All day, Mon, 30 Nov 2026" (DTEND of a date is exclusive), a range across days as
  "… to …". `CalendarCardOptions.DisplayTimeZone` converts through the `.Calendar` zone resolution (the calendar's own VTIMEZONE, then Windows names,
  then the runtime's IANA database); the event's own time is kept as `OriginalTime` ("Organizer's time") when the zones differ. Floating times and dates
  are shown as written. An unresolvable TZID is shown as written with `UnresolvedZone`; an unknown display zone falls back to the event's zone with
  `DisplayZoneUnresolved`.
- **Recurrence** in plain English (below) plus the first `NextOccurrences` (default 5, at most 50) from the library's expansion (`RecurrenceSet`), starting
  at the series start or at `After` (a UTC instant the caller supplies; the card never reads the clock). EXDATEs are noted (first five), changed
  occurrences counted.
- **Where**: LOCATION; URLs from `CONFERENCE` (RFC 7986), `URL`, `X-MS-OLK-NETSHOWURL` (`PidLidNetShowUrl`, MS-OXOCAL 2.2.1.56.8),
  `X-MICROSOFT-ONLINEMEETINGCONFLINK`, `X-MICROSOFT-SKYPETEAMSMEETINGURL` and `X-GOOGLE-CONFERENCE`. Only absolute `http`, `https` and `mailto` URLs
  up to the field limit are kept (at most five); any other is dropped with `UrlRejected`. The Teams and Zoom values Outlook keeps in named properties
  that MS-OXPROPS does not define are not read; a meeting link Outlook exports as one of the properties above is.
- **People**: organizer, attendees with role (required, optional, chair, for information), participation status (accepted, declined, tentative,
  no reply yet, delegated) and a non-person kind (room, resource, group). The first 50 are listed, the rest counted (`AttendeesTruncated`).
- **Description**: truncated to 1,000 characters and 30 lines (`DescriptionTruncated`); attachments: at most ten file names or URLs, as text.
- Every one-line field is cut at 200 characters (`FieldTruncated`); the cards of one calendar object stop at 20 (`EventsTruncated` on the first).

All words are in `CalendarCardStrings`, one internal class, in English; dates use a fixed English layout and the invariant culture, so output does not
depend on the machine.

## Recurrence sentences

`CalendarCardRecurrence.Describe(rule, start)` returns a sentence only when it says exactly what the rule does:

- `Hourly`, `Every 2 hours`, `Every minute`, `Every 15 minutes`; `Daily`, `Every 3 days`, `Every weekday`;
- `Weekly on Monday and Wednesday`, `Every 2 weeks on Monday, Wednesday and Friday` (no BYDAY: DTSTART's weekday);
- `Monthly on day 15`, `Monthly on day 1 and day 15`, `Monthly on the last day`, `Monthly on the first Monday and the third Wednesday`,
  `Monthly on the last Friday` (also `BYDAY=FR;BYSETPOS=-1`), `Monthly on the last weekday` (Monday to Friday with one BYSETPOS), `the second to last weekend day`;
- `Yearly on 15 March`, `Yearly on 1 and 15 March`, `Yearly on the last day of March`, `Yearly on the first Monday of March`;
- each followed by `, until 30 Nov 2026` or `, 10 times`.

Everything else (BYHOUR, BYMINUTE, BYSECOND, BYYEARDAY, BYWEEKNO, RSCALE, SKIP, several BYSETPOS values, BYMONTHDAY with BYDAY, several months, SECONDLY,
COUNT with UNTIL, other parts) gives "Repeats (custom rule)" and `RecurrenceNotDescribed`. An interval above 1 with several weekdays depends on the week
start (WKST, RFC 5545 §3.3.10): a rule with `WKST=SU` (what Outlook writes on every rule) is described when neither DTSTART's day nor a listed day is a
Sunday, because then Monday-start and Sunday-start weeks group the days alike; any other week start is left to the custom text. A series with only RDATEs
is "Repeats on selected dates".

The sentences are checked by parsing them back (`tests/OneMail.Tests/Display/SentenceParser.cs`, written from the sentence forms and independent of the
generator): the parsed frequency, interval, weekday set with ordinals, days of the month, month, count and until date must equal the rule's, with what
the rule leaves out taken from DTSTART.

## Markup

The HTML is built from escaped text with fixed markup: `div`, `p`, `ul`, `li`, `span`, `strong` and, only when `ToHtml(linkUrls: true)` is asked, `a`.
Attributes are `class` and `style` with constant values, plus `href` and `rel="noopener noreferrer"` on links, where `href` is the normalised absolute
URL of an `http`, `https` or `mailto` link. Event text enters only through HTML encoding after control characters, bidirectional overrides and isolates,
zero-width characters and the line and paragraph separators are replaced by spaces. By default URLs are plain text. The card passes
`HtmlSanitizer.Sanitize` with no change and no reference other than its own links (a test over every fixture, with and without links).

## How it is checked

- `tests/OneMail.Tests/Display/CalendarCardTests.cs`: sentences, fallbacks, parse-back over a grid of rules, every method, zones, hostile and huge input
  (a 3 MB description, 10,000 attendees, 100 events, a rule that never matches), inertness, the sanitiser.
- `tools/calcard`: `run_sweep.sh` (cards for every calendar object in the fetched corpora and the fixtures, per source format and method, OneHtml parse
  errors, sentence parse-back), `run_occ_diff.sh` (occurrence lists against the library's expansion and `recurring-ical-events` 3.8.2 over the recurring
  components of the calendar corpus), `browser-check.sh` (the sanitiser's XSS vectors in every field, in headless Chromium with no sanitiser in between),
  `make_fixtures.py` (`testdata/calendar/card`: one file per method and variant, 2,700 rule shapes).
- Fuzz target `calcard`; AOT and browser smoke "calcard".


<!-- https://recursiveai.org/onemail/docs/contacts/ -->

# Contacts: the vCard typed view, Outlook contacts, and MS-OXVCARD

Three pieces, each usable alone:

- `VCardView` (`RecursiveAI.OneMail.Calendar.VCard`, package `.Calendar`): typed, read-only accessors over a `VCARD` component. Nothing is
  copied out of the tree; the bytes stay in the component, so a card written back untouched is byte-identical.
- `MapiContact` and `MapiDistributionList` (`RecursiveAI.OneMail.Contacts`): typed views over the property bag of an `IPM.Contact` or
  `IPM.DistList` message. They also hang off the shared view as `MapiMailItem.Contact` / `.DistributionList`.
- `ContactToVCardConverter` and `VCardToContactConverter` (`RecursiveAI.OneMail.Conversion.OxVCard`): MS-OXVCARD in both directions, as
  `IItemConversion` with a loss report (`docs/conversion.md`). Family names in the report: `mapi` for contact properties, `vcard` for card
  properties.

## The vCard typed view

`VCardView` reads vCard 2.1, 3.0 and 4.0 alike. Accessors for a property that may repeat return every occurrence in file order
(`Names`, `Titles`, `Roles`, `Notes`, `Organizations`, `Birthdays`, `Anniversaries`); the singular ones (`Name`, `Note`, `Birthday`) are the first.

- **Typed values** (`VCardTyped`): `Value`, `Types` (bare 2.1 parameters and `TYPE=` merged), `Pref`, `AltId`, `Pid`, `Language`,
  `ValueType`, `Label`, `HasType(word)`. Used by `Emails`, `Phones`, `Addresses`, `Urls`, `Impps`, `Related`, `Languages`, `TimeZones`.
- **Media** (`VCardMedia`): `PHOTO`, `LOGO`, `SOUND`, `KEY`. `IsInline`, `Uri`, `MediaType`; `Data` is decoded the first time it is read, not
  at parse time. Base64 is read leniently: surplus or missing `=` padding is accepted, a character outside the alphabet makes `Data` null (the
  raw value is still on the property). A `data:` URI is inline.
- **Dates**: `Birthday`, `Anniversary` are `VCardDate` (year, month, day and optional time, each possibly absent: `--0412`, `1980-03`). A
  `VALUE=text` date, or free text, is returned by `BirthdayText` / `AnniversaryText` and `Birthday` is null. `RevisionTime` is the `REV` instant.
- **Others**: `Gender` (sex and identity text), `Geos` (3.0 `lat;lon`, 4.0 `geo:` URI), `Kind`, `Members`, `Extensions` (`X-` properties, name
  and value), `GetText(property)` and `Unescape(text)` for a property the view has no accessor for.
- **Text unescaping.** In 3.0 and 4.0 a backslash before any character other than `\\ \; \, n N` is dropped and the character kept; 2.1 uses
  `\\` and `\;` only and keeps everything else (ambiguity CV1). The content-line layer is unchanged: the raw value always keeps every backslash.

## The Outlook contact view

`MapiContact(MapiMessage)` reads the properties of MS-OXOCNTC. Strings that are absent and strings that are empty both read as null.

| Area | Members | Properties |
|---|---|---|
| Names | `DisplayName`, `FileUnder`, `GivenName`, `MiddleName`, `Surname`, `Initials`, `NamePrefix`, `NameSuffix`, `Nickname`, `Gender` | `PidTagDisplayName`, `PidLidFileUnder`, `PidTagGivenName` ... `PidTagGeneration`, `PidTagNickname`, `PidTagGender` |
| Work | `CompanyName`, `Department`, `JobTitle`, `Profession`, `OfficeLocation`, `ManagerName`, `AssistantName` | `PidTagCompanyName` ... `PidTagAssistant` |
| Personal | `SpouseName`, `ChildrensNames`, `Hobbies`, `Notes`, `Categories`, `BirthDate`, `AnniversaryDate`, `Birthday`, `Anniversary` | `PidTagSpouseName`, `PidTagChildrensNames`, `PidTagHobbies`, `PidTagBody`, `PidNameKeywords`, `PidTagBirthday`, `PidLidBirthdayLocal`, ... |
| Addresses | `Address(kind)`, `Addresses`, `MailingAddress` | home `PidTagHomeAddress*`; business `PidLidWorkAddress*` (0x8045-0x804A); other `PidTagOtherAddress*`; `PidLidPostalAddressId` selects the mailing address. 0x3A26-0x3A2B are the mailing-address copy and are not read as an address of their own |
| Phones | `Phones`, `Phone(kind)` | the 19 telephone and fax properties in `ContactPhoneKind` |
| Email and fax | `EmailAddresses`, `FaxAddresses` | `PidLidEmail1..3*`, `PidLidFax1..3*`; `SmtpAddress` is the address when the type is `SMTP` (or untyped and address-shaped), null for `EX` and other types |
| Other | `InstantMessagingAddress`, `PersonalHomePage`, `BusinessHomePage`, `FreeBusyUrl`, `UserFields` (4), `PhotoAttachment`, `HasPicture`, `X509Certificates` | |

**Birthday and anniversary** are dates. Outlook writes two properties: `PidTagBirthday` at 11:59 UTC of the date, and `PidLidBirthdayLocal` at
0:00 of the date. The local one is preferred; with only the first, the UTC date is taken. (A reader that shows the instant in a time zone west of
UTC shows the evening before; that is the reader's choice, not a different date.)

`MapiDistributionList(MapiMessage)` reads `PidLidDistributionListName` and the members. When `PidLidDistributionListStream` (0x8064) is present
it is the list of members (MS-OXOCNTC 2.2.2.1); otherwise the multi-valued `PidLidDistributionListMembers` (0x8055) with the one-off entries
(0x8054) are used. A member is a `DistributionListMember`: `Kind` (`OneOff`, `Contact` for a wrapped entry ID of a contact slot, or a list),
`DisplayName`, `AddressType`, `Address`, `SmtpAddress`, `ContactSlot`. A malformed entry is skipped, never thrown.

## Contact to vCard (`ContactToVCardConverter`)

Options: `Version` (3.0 default, as MS-OXVCARD specifies; 4.0), `IncludePhoto` (true).

| Contact | vCard |
|---|---|
| display name, else normalised subject, else the name parts joined | `FN` |
| surname; given; middle; prefix; suffix | `N` |
| nickname | `NICKNAME` |
| photo attachment (bmp, gif, jpeg, png only) | `PHOTO` (3.0 `ENCODING=b;TYPE=`; 4.0 `data:` URI) |
| birthday, anniversary | `BDAY`; `X-MS-ANNIVERSARY` (3.0) or `ANNIVERSARY` (4.0) |
| three addresses | `ADR` with `TYPE=home|work|postal`; 3.0 also `LABEL`; the mailing address has `pref` (3.0) or `PREF=1` (4.0) |
| phones | `TEL` with `TYPE=work|home|cell|car|pager|isdn|fax...`; company, assistant, callback, radio, TTY/TDD as `X-MS-TEL;TYPE=`; telex as `EMAIL;TYPE=tlx` |
| SMTP email addresses (first is preferred) | `EMAIL;TYPE=internet` |
| job title, profession, company and department, categories, body, web pages, free/busy URL | `TITLE`, `ROLE`, `ORG`, `CATEGORIES`, `NOTE`, `URL;TYPE=home|work`, `FBURL` |
| last modification time | `REV` |
| sensitivity | `CLASS` (3.0 only) |
| X.509 certificates | `KEY` |
| children, user fields, IM address, spouse, manager, assistant, hobbies, default postal address | `X-MS-CHILD`, `X-MS-TEXT`, `X-MS-IMADDRESS`, `X-MS-SPOUSE`/`MANAGER`/`ASSISTANT` (`TYPE=N`), `X-MS-INTERESTS`, `X-MS-OL-DEFAULT-POSTAL-ADDRESS` |
| gender (4.0) | `GENDER` |
| distribution list (4.0) | `KIND:group`, one `MEMBER:mailto:` per member with an SMTP address |

Text is escaped for 3.0/4.0 and a CR LF becomes `\n`. Every property of the source is either written or in the report. Codes:

| Code | Kind | Meaning |
|---|---|---|
| `NoVCardMapping` | Dropped | MS-OXVCARD maps nothing from this property |
| `NotInThisVersion` | Dropped | 3.0 has no `GENDER` or group; 4.0 has no `CLASS` |
| `ExchangeAddressNotExported` | Dropped | an `EX` address with no SMTP form |
| `AdditionalValuesNotCarried`, `UnsupportedPhotoType`, `AttachmentNotCarried`, `MemberNotCarried` | Dropped | a second value; a photo that is not bmp/gif/jpeg/png; any other attachment; a list member without SMTP address |
| `PhoneSlotChanged`, `EmailSlotChanged` | Approximated | a second number or an address after a gap moves up when imported: a vCard has no slot order |
| `EmptyValueNotWritten` | Internal | the property is the empty string; a vCard property is not written for it (an empty and an absent text mean the same), and it is absent after import |
| `DefaultValueNotWritten` | Internal | `PidLidPostalAddressId` is 0 (no mailing address) |
| `DerivedValue`, `StoreBookkeeping` | Internal | copies of the display name, formatted addresses, entry IDs, flags |

## vCard to contact (`VCardToContactConverter`)

The result is an `IPM.Contact` message; `PidTagMessageClass`, `PidTagSensitivity` (`CLASS`, default 0) and the derived properties of the
next section are always set. The rules follow the examples of MS-OXVCARD 2.1.3 literally:

- `TEL`: every rule that matches fires. `TYPE=VOICE,MSG,WORK` fills Business and Other; a fax alone fills Primary Fax; `pref` fills Primary; no type fills Other. The second work (home) number goes to Business2 (Home2).
- `ADR`: with no recognised type, Business and Other; `postal`, `dom`, `intl`, `parcel` fill Other. `pref` sets `PidLidPostalAddressId`. A fourth address is `AdditionalValueNotMapped`; the extended-address part is merged into the street (`AddressExtendedMerged`).
- `EMAIL`: the first three fill the email slots (`PidLidEmail1..3`); more are `AdditionalValueNotMapped`.
- `BDAY`: `PidTagBirthday` is the date at 11:59 UTC, `PidLidBirthdayLocal` the date at 0:00 (`TimeOfDayDropped` when the card had a time). A partial date is `PartialDate` and not stored; `X-MS-ANNIVERSARY` and 4.0 `ANNIVERSARY` likewise.
- `N` components written as lists are stored as the comma-joined text; several organisation units are joined with ", " (`OrgUnitsMerged`).
- `PHOTO`: inline data of a supported type becomes the contact photo attachment; a URI is `PhotoNotInline`. `KEY` that is not X.509 is `KeyNotX509`.
- Properties with no home (`UID`, `GEO`, `TZ`, `LANG`, `IMPP` beyond the first, ...) are `NoContactMapping`; `VERSION`, `PRODID` and `MAILER` are `VCardBookkeeping`.

**Derived on import** (never a loss): `PidTagSubject`, `PidTagNormalizedSubject`, `PidTagConversationTopic` (copies of the display name),
`PidTagDisplayName` when the card has no `FN` (composed from `N`), `PidLidEmailNDisplayName` and the formatted addresses, the mailing-address copy
(`PidTagPostalAddress`, 0x3A26-0x3A2B), `PidLidFileUnder` / `PidLidFileUnderId`, `PidTagMessageClass`, and flags.

## Fidelity

- Contact → vCard → contact: each mapped property comes back equal, or the report names it. "Equal" is the exact property value; what the
  check does not count as a loss is the derived set above.
- vCard → contact → vCard: the contact is stable on a second round; the first differs from the source only in the derived additions (an `FN`
  composed from `N`) and in what the report lists.

## Checks

`tools/contacts` holds them; nothing there ships in a package.

| Tool | What it checks |
|---|---|
| `ContactTool dump / export / rt-contact / rt-vcard / vfacts / gen` | reads every contact and list item of MSG/PST/OST; converts both ways; counts exact, reported, derived, mismatch, silent |
| `ContactTool gen DIR [seed [count]]` | seeded contact and list MSG files covering the MS-OXOCNTC properties (empty strings, odd characters, slot gaps, photos) |
| `make_vcards.py` | seeded vCards 2.1/3.0/4.0 (QP, groups, partial dates, photos, `KIND:group`) |
| `compare_extract_msg.py` | contact fields against extract-msg 0.55.0 |
| `compare_vcard.py` + `calendar/oracles` | typed view against vobject 0.9.9 and FolkerKinzel.VCards 8.2.0 (rule classes OL-*, V21COMMA, ESC, K1) |
| `run.sh`, `vcard_checks.sh` | the worker runs of all of the above |

Results are in [compatibility.md](https://recursiveai.org/onemail/docs/compatibility/). The fuzz target is `vcard-view` (a vCard, or a damaged contact or list, through the view and
both conversions; the vCard written must parse again).


<!-- https://recursiveai.org/onemail/docs/compatibility/ -->

# Fidelity and compatibility

What OneMail reads and writes today, how each format is checked, and the gaps we know about. This table is updated with every
preview; a gap is listed here, never hidden.

**Oracles** (run as released programs, black-box): MimeKit 4.18.2 (NuGet), Python 3.13.17 `email` and `mailbox`, Go 1.26.8
`net/mail` and `mime/multipart`, Python `icalendar` 7.3.0 and `vobject` 0.9.9, libpff tools 20180714. Differences between
OneMail and an oracle are each mapped to an entry in [`ambiguities.md`](https://recursiveai.org/onemail/docs/ambiguities/), a documented oracle limit or a
normalisation of the comparison; "unexplained" counts the rest.

**Corpora:** the CMU Enron maildir (May 2015 version, 517,401 messages), Dovecot's `dovecot-crlf` mbox (2,859 messages by
the oracles' split), Apache James Mime4j test messages (80), our generated probes, and the calendar data files listed in
[`THIRD-PARTY-NOTICES.md`](https://github.com/recursive-ai-oss/onemail-csharp/blob/bb4c9543917ab4ad466b5200d132e1899cb3ca14/THIRD-PARTY-NOTICES.md). None of these is committed; `tools/corpus/` fetches them by pinned URL and hash.

## Calendar card (0.3.0-preview.2)

`CalendarCard` ([`calendar-card.md`](https://recursiveai.org/onemail/docs/calendar-card/)), checked on the calendar corpora (libical, ical.js, biweekly, sabre/vobject, Thunderbird holidays, the
RFC examples) and the generated fixtures `testdata/calendar/card`, the MSG, OLM, TNEF and PST/OST corpora, and Outlook items generated from the
iCalendar files by the MS-OXCICAL import. Cards: 3,524 from the fetched corpora (ics 3,502, msg 2, olm 8 appointments; Add 96, Cancel 161, Counter 65,
DeclineCounter 32, Publish 171, Refresh 64, Reply 309, Request 790, none 1,816, other 10), 0 exceptions, 0 OneHtml parse errors over 7,048 documents.
Recurrence sentences: 336 of 337 real rules described, the other "Repeats (custom rule)"; every described rule of the corpora (1,070) and of 2,700
generated shapes parses back to the same frequency, interval, weekdays, days, month, count and until date. Occurrence lists (50 per series) against the
expansion they are built on and against `recurring-ical-events` 3.8.2 over 2,384 recurring components: 2,357 equal, 14 stopped early by the card's own scan
limit with a typed defect, 13 outside the naive case window, 0 unexplained; against the oracle 2,248 equal, 98 where the expansion itself differs
from the oracle (explained in `recurrence.md`), 0 unexplained. XSS: the 233 sanitiser vectors in every text field (summary, location, description,
URL, conference, attachment and its name, organizer and attendee names and addresses, role, status), rendered in headless Chromium without a sanitiser:
0 scripts run, 0 requests (the raw vectors as control: 42 scripts, 58 requests). The Outlook meeting items in the corpora are few (two appointments in
MSG); the Outlook-item path is checked on generated items only, and Outlook-produced meeting items wait for Outlook.

## 0.3.0-preview.1

Calendar and contacts, the other stores and the Outlook item semantics on read are complete for this stage; the sections below
hold each format's numbers. Fuzzing, 600 s per target with 0 findings: `dbx` 8.1M executions, `olm` 17.5M, `oxocal` 0.3M,
`rtf-html` 5.0M, `jcal-jcard` 12.8M, `xcal-xcard` 19.7M, `contentlines` 20.4M, `calendar` 11.2M, `vcard-view` 11.7M,
`recurrence` 2.1M, plus the display, bodies and composition targets in their sections. Known gaps: no Outlook-produced
calendar, contact or distribution-list fixtures yet (needs Outlook), and the display sanitiser has not had its independent
review.

## 0.2.0-preview.1

Added since 0.1. The results of the 0.1 rows below still hold; where a 0.1 gap names something a row above now covers, the row above replaces it. Further oracles for these rows, all run as released programs: Python `extract-msg` 0.55.0,
`compressed-rtf` 1.0.7, `RTFDE` 0.1.2.2 and `tnefparse` 1.4.0; `tnef` 1.4.18 and `ytnef` 2.1.2 (distribution packages); `recurring-ical-events`
3.8.2 and `python-dateutil` 2.9.0; Ical.Net 5.2.3 (NuGet, public API). Further corpora, fetched and never committed: the NIST CFReDS Data
Leakage case OST, and MSG, PST and TNEF test files from Apache Tika and Apache POI ([`THIRD-PARTY-NOTICES.md`](https://github.com/recursive-ai-oss/onemail-csharp/blob/bb4c9543917ab4ad466b5200d132e1899cb3ca14/THIRD-PARTY-NOTICES.md)).
No Outlook has been run yet.

| Format | Read | Write | Differential (unexplained / compared) | Round-trip | Fuzzed | Known gaps |
|---|---|---|---|---|---|---|
| MSG / OFT (MS-OXMSG) | yes | no | 0 against extract-msg on 58 files, 15,609 properties; view fields 0 on 53 files, 690 fields | property streams re-encode identical 60 / 60 | yes | two fuzz-minimised files from the corpus are refused by the compound file reader (typed defect); writing comes later |
| PST Unicode, OST (MS-PST: NDB, LTP, messaging) | yes | no | libpff `pffexport` agrees on folders, items, recipients and attachments: 0 differences on the CFReDS OST, Tika `testPST` and `testPST_variousBodyTypes`, and generated ANSI and Unicode stores (128 items); property contexts and tables read with 0 defects (3,771 + 998 + 733 properties) | n/a | node database, tables, messaging | no public ANSI or 512-byte-page Unicode PST from Outlook exists; those layouts and 4 KiB multi-map stores are checked only on files we generate. libpff cannot read 4 KiB tables whose rows sit in a data tree, so those have no differential yet. Associated (hidden) items are read but libpff does not export them |
| PST ANSI | yes | no | generated stores only (see above) | n/a | yes | no real-world sample |
| TNEF / winmail.dat (MS-OXTNEF) | yes | yes | 0 against tnefparse, tnef and ytnef: 563 checks, 125 explained | attributes identical 17 / 17; MAPI property-exact 17 / 17 | yes | no TNEF-in-MIME message in the public corpora; checked on generated wrappers (64 messages) |
| Compressed RTF (MS-OXRTFCP) | yes | yes | decompression identical to compressed-rtf on 45 / 45 bodies, CRC 45 / 45; our output read back by both readers 45 / 45 | 97 exact + 1 line-ending | yes | `\objattph` positions become placeholders |
| HTML and text from RTF (MS-OXRTFEX); native RTF to HTML | yes | encapsulation only | against RTFDE on 35 bodies: only line-ending and no-break-space differences (documented); native RTF to HTML on 123 documents: 0 unexplained against LibreOffice 24.2.7, pandoc 3.13 and unrtf 0.21.10 (pandoc/unrtf rules are class-level) | n/a | yes | |
| MAPI and MIME conversion (MS-OXCMAIL) | yes | yes | see the conversion table below | 0 silent losses both ways | yes | |
| ICS recurrence and time zones (RFC 5545 3.8.5, VTIMEZONE, IANA, Windows names) | yes | n/a | 2,384 recurring components: 0 unexplained against recurring-ical-events (2,372 compared), python-dateutil (2,378) and Ical.Net (2,188); every RFC 5545 example passes; DST transitions 108 / 108 in four zones | n/a | yes | |


Speed: PST/OST against XstReader.Api, MSG against MsgReader and TNEF against MimeKit all meet the bar on x64 and arm64 (geometric means
0.197 / 0.320, 0.096 / 0.101, 0.312 / 0.287; worst row 0.569). Working set stays near 100 MB (x64) from 1,000 to 50,000 messages (826 MB
store). Details in [`benchmarks.md`](https://recursiveai.org/onemail/docs/benchmarks/).

## Windows

The suite also runs on win-x64. Two things behave differently there, both documented: IANA time zone names resolve through the Windows
zone each maps to (CLDR), so instants before the last rule change (US dates before 2007) follow today's rules rather than IANA's
([`recurrence.md`](https://recursiveai.org/onemail/docs/recurrence/), Z5); and Maildir files are written with `;` before the info section instead of `:`, which
Windows file names cannot hold, while both are read everywhere ([`mailboxes.md`](https://recursiveai.org/onemail/docs/mailboxes/), S19).

## 0.1.0-preview.1

| Format | Read | Write | Differential (unexplained / compared) | Round-trip of untouched input | Fuzzed | Known gaps |
|---|---|---|---|---|---|---|
| MIME / EML (RFC 5322, 2045-2049, 2047, 2231, 6532) | yes | yes | 0 against each of MimeKit, Python, Go on 517,401 + 2,859 + 80 + 224 messages | byte-identical: 517,401 / 517,401; 2,859 / 2,859; 80 / 80 | reader, tree | ambiguity resolutions are provisional until mail clients are observed |
| Charsets and encoded words (WHATWG encodings, UTF-7, IMAP UTF-7, ISO-2022-JP, RFC 2047/2231) | yes | yes | 0 against MimeKit (22,000 cases), Python (23,973), Go (21,412) | n/a | yes | |
| Structured headers (addresses, dates, message ids, parameters) | yes | yes | included in the MIME row | n/a | via tree | |
| multipart/report: DSN (RFC 3464), MDN (RFC 8098), ARF (RFC 5965) | yes | yes | 0 against MimeKit and Python on 125 messages | 125 / 125 | yes | no real-world reports in the corpora; checked on generated and RFC-derived fixtures |
| MHTML (RFC 2557) | yes | yes | 0 (same run); URL resolution 0 against Python `urljoin` on 218 pairs | yes | yes | |
| `Authentication-Results` (RFC 8601), `Received-SPF` (RFC 7208) | yes | no | 0 against MimeKit (34 fields); Python has no parser | n/a | yes | `Received-SPF` has no oracle |
| mbox: mboxo, mboxrd, mboxcl, mboxcl2, Gmail Takeout labels (RFC 4155) | yes | yes | message boundaries: 0 against Python and MimeKit on 17 files, 3,798 messages | record copy 16 / 16 | yes | MMDF not supported |
| Maildir (Courier and Dovecot flags and keywords) | yes | yes | 0 against Python `mailbox` | 48 / 48 | via store tests | Courier keyword file line format unverified against a real store; one directory listing held in memory |
| Outlook Express DBX (5/6) | yes | no | 0 against undbx 0.21: 2,053 real records (34 with text) in 13 folders, 3,512 generated messages in 12 folders | n/a | dbx | one real store (a Windows XP laptop image, 28 files); OE4 files and deleted messages are not read |
| EMLX and Apple Mail `.mbox` bundles | yes | yes | ground truth 14 / 14 | 14 / 14 | yes | flag bits unverified against a real Apple Mail store |
| ICS (RFC 5545, 5546, 7986), vCard 2.1 / 3.0 / 4.0 | yes (content lines, component trees, typed views; contacts below) | yes | see [`calendar.md`](https://recursiveai.org/onemail/docs/calendar/) | 1,372 / 1,372 | yes | recurrence expansion, time zones and j/x forms come later |
| The shared read view, `Mail.Open`, `Mail.Parse`, format sniffing | yes | n/a | agrees with the tree on 8,566 messages; attachment lists agree with Python except documented choices (V1-V8) | n/a | yes | DBX items not in the view yet |
| PST / OST node database (MS-PST NDB) | yes (lower layer only) | no | NIST CFReDS OST: every node and block enumerates, header agrees with libpff | n/a | yes | no messages from PST yet; ANSI and 512-byte-page Unicode PSTs untested on real files (no public sample) |


## Contacts (vCard typed view, Outlook contacts, MS-OXVCARD)

Method and rules: [`contacts.md`](https://recursiveai.org/onemail/docs/contacts/). No Outlook has been run. The corpus has two contact items (Tika and POI MSG, the same contact) and no
Outlook contact PST, so the MS-OXOCNTC properties are covered by 56 generated contacts and 8 lists (`ContactTool gen`, seed 20261031). Oracles run black-box:
vobject 0.9.9, FolkerKinzel.VCards 8.2.0 (public API), extract-msg 0.55.0.

| Check | Against | Result |
|---|---|---|
| vCard typed view, fetched corpus (28 .vcf) | vobject / FolkerKinzel.VCards | vobject: 25 compared (3 it cannot parse), 365 agree, 17 explained, **0 unexplained**; FolkerKinzel: 28 compared, 417 agree, 37 explained, **0 unexplained** |
| vCard typed view, generated 2.1/3.0/4.0 (60) + RFC examples + hand fixtures (86 files) | vobject / FolkerKinzel.VCards | vobject: 83 compared, 998 agree, 31 explained, **0**; FolkerKinzel: 86, 909 agree, 81 explained, **0** |
| vCards we produce (132 files from the 2 real and 64 generated items, 3.0 and 4.0) | vobject / FolkerKinzel.VCards | both parse **all 132**; vobject 2,393 agree, 17 explained, **0 unexplained**; FolkerKinzel 2,204 agree, 254 explained, **0** |
| Contact fields (2 real + 56 generated MSG) | extract-msg | 58 contacts, 4,408 field checks, **0 differences** |
| Contact to vCard to contact (3.0 / 4.0) | every property exact, derived or in the loss report | real (2): 44 / 42 exact, 168 / 170 reported, 16 derived, **0 mismatch, 0 silent**; generated (56): 2,016 / 2,002 exact, 713 / 717 reported, 103 / 113 derived, **0 mismatch, 0 silent** |
| vCard to contact to vCard (138 cards) | every card property carried or reported; contact stable on a second round | 1,311 carried, **0 not carried and not reported**; 137 / 138 stable (the one: a card without `FN` gets one composed from `N`) |
| Byte-identical round trip of untouched vCards | the content-line tree | 28 + 86 corpus and generated files and 132 produced cards: all byte-identical |
| Fuzz `vcard-view` | 600 s, `Checked` build | 5,049,889 executions, 0 findings |

Gaps: no Outlook-produced contact PST or MSG with the full property set, and no Outlook import/export of vCards to compare against (the Outlook oracle, before 0.5).
The slot order of telephone numbers and email addresses is approximated by a vCard and reported (`PhoneSlotChanged`, `EmailSlotChanged`).

## MS-OXCMAIL conversions (MAPI to MIME and back)

Checked on the 45 Tika and POI MSG files and the 25 Tika and CFReDS PST/OST items that are mail, and on message files for the other
direction. Method and rules: [`conversion.md`](https://recursiveai.org/onemail/docs/conversion/). No Outlook has been run; the oracles are MimeKit 4.18.2 and Python 3.13.17
`email`, black-box.

| Direction | Checked against | Result | Known gaps |
|---|---|---|---|
| MAPI to MIME (EML) | OneMail, MimeKit and Python read each exported EML; subject, From, To, Cc, Date, Message-ID, text and HTML body hashes, attachment names, sizes and SHA-256 compared with the source item's | 70 / 70 items parse with no defect the received headers did not already have; 0 unexplained differences in 630 fields per oracle (MSG 405 + PST 225); one field in MimeKit differs for a received Subject that uses the charset label `Cp1251` (listed in `tools/oxcmail/explained.tsv`; Python and OneMail agree with the source) | EX addresses with no SMTP form in the item are left out and reported (30 `AddressUnresolved` entries over the 70 items); Bcc is not written by default |
| MAPI to MIME to MAPI | every property of the source: exact, same text under another string type, or in the loss report | 11,142 properties: 6,406 exact, 446 same text, 4,290 reported, **0 silent** | `PidTagTransportMessageHeaders` is rewritten (reported) |
| MIME to MAPI to MIME | ten fields before and after (subject, From, To, Cc, Date, ids, bodies, attachments) on Mime4j 80, Dovecot 1,430, Enron sample 5,032 messages | 65,420 fields: 65,392 equal, 28 differ and each has a loss entry, **0 silent** | Bcc dropped on the way out (reported); display-name spelling of a kept received header |
| The view | `MapiMailItem` senders and recipients | all 50 EX recipients and 16 of 28 EX senders resolve to SMTP with the X.500 address kept; the other 12 senders (all in the MSG set) have no SMTP form; a further 9 PST senders have no address at all | no directory lookup |

## Outlook calendar and task items (MS-OXOCAL, MS-OXOTASK, MS-OXCICAL)

| Area | Checked against | Result | Known gaps |
|---|---|---|---|
| Typed views: appointments, meetings, tasks | extract-msg 0.55.0 on the corpus items (subject, location, attendees, task status and dates) | 14 of 14 comparable fields agree, 4 not exposed by the oracle (start, end), 0 unexplained | the corpus holds 2 distinct appointment and 2 task items and no meeting or recurring item: those are covered by generated items following the MS-OXOCAL examples |
| Recurrence blob and time zone structures | MS-OXOCAL 4.1.1 weekly and daily examples | byte for byte; TZ offsets for Pacific, Sydney, no DST | no Outlook-written recurring item yet |
| Item to ICS, expansion | 18 recurrences in 3 zones (54 series) against dateutil 2.9.0.post0, recurring-ical-events 3.8.2, Ical.Net 5.2.3 | dateutil 54/54, recurring-ical-events 54/54, Ical.Net 38/54 (16 explained: window wall clock 5, outvoted by the other two 11), 0 unexplained | |
| ICS we write | Python icalendar 7.3.0, Ical.Net 5.2.3 parse | 58 of 58 files parse in both; icalendar facts identical 58/58; Ical.Net facts identical 54/58 (the rest are typed properties its public API does not list) | |
| Item to ICS to item, ICS to item to ICS | `OxcIcalRoundTripTests`, `OxocalTool` | 13 generated items property-exact, corpus items 4 of 4 stable, generated ICS 54 of 54 stable on the second pass, 0 silent | Outlook-written meetings and exceptions unseen |

## OLM (Outlook for Mac) read

Added after 0.2.0-preview.1. Oracles, run as released programs and never linked: the olmreader 1.9.0 jar (Maven Central, BSD-2-Clause) and
the olm-reader 1.0.0 npm package (Apache-2.0). Corpus, fetched and never committed: the two sample archives of the OLMReader project
(BSD-2-Clause; [`THIRD-PARTY-NOTICES.md`](https://github.com/recursive-ai-oss/onemail-csharp/blob/bb4c9543917ab4ad466b5200d132e1899cb3ca14/THIRD-PARTY-NOTICES.md)). No Outlook for Mac has been run.

| Format | Read | Write | Differential (unexplained / compared) | Round-trip | Fuzzed | Known gaps |
|---|---|---|---|---|---|---|
| OLM (Outlook for Mac archive) | yes: mail, appointments, contacts, groups, tasks, notes, categories; attachments as lazy streams | no, by design | jar: 0 on both samples and on two generated archives (outlook_example 38 records, bill_rapp 492, generated 54); npm package: 0 on 485 messages, 12 documented tool differences | records keep every element (`OlmElement` tree); raw MIME source kept as text | yes | no Outlook 2011 layout sample; no archive with a raw MIME source, attendees or exceptions seen; a damaged zip is refused (recovery comes later); the working set is not constant in the entry count: OneArchive holds the zip directory (about 1 KB per entry; 123 MiB at 26,678 entries, 309 MiB at 213,345; see [`olm.md`](https://recursiveai.org/onemail/docs/olm/)) |

## Conversions and store to store

See [`fidelity.md`](https://recursiveai.org/onemail/docs/fidelity/) for the matrix (generated from the code; a test fails when a reported loss is not in its cell) and the
command line. Checked on the worker with `tools/convert/run.sh`:

| Check | Result |
|---|---|
| MIME to EML / mbox / Maildir / EMLX, read back byte-identical | probes 219/224, Mime4j 78/80, dovecot mbox 3,237/3,237, dovecot messages 2,858/2,859, Enron sample 5,032/5,032, DBX store 2,053/2,053; 0 changed (the skipped have no header block) |
| MAPI to EML to MAPI | MSG 45 items / 8,743 properties and PST 25 items / 2,399 properties: 0 silent; OLM mail 530 items / 10,070 fields: 0 silent |
| Store to store (OST, Tika PSTs, OLM, DBX, dovecot mbox, Maildir, Enron sample) | read = written + skipped everywhere; folder trees equal; Python `mailbox`/`email` agree on folders and counts in 18 of 18 runs |
| Memory | CMU Enron (517,401 messages, 3,311 folders) to Maildir: 119 MB peak; to mbox: 129 MB peak |

Gaps: flags and keywords from Outlook items (`PidTagFlagStatus`, last verb) are mapped by rule and not yet observed in Outlook; MSG and PST
are not destinations until 0.5 (the `IStoreSink` extension point is public).

## Replies and forwards ([`composition.md`](https://recursiveai.org/onemail/docs/composition/))

`CreateReply`, `CreateReplyAll` and `CreateForward` (inline and as an attachment) on any item. Each message is made, written, and parsed back by OneMail, MimeKit 4.18.2,
Python 3.13.17 `email` and Go 1.26.8 `net/mail`; subject, From, To, Cc, In-Reply-To and References are compared with what the composer says it wrote
(`tools/composition`). A reply, a reply-all and the two forwards are made of every item.

| Source | Items | Messages made | In-process checks failing | Oracle fields compared (MimeKit / Python / Go), differing | Notes |
|---|---|---|---|---|---|
| Enron maildir sample | 5,032 | 20,128 | 0 | 120,537 / 120,537 / 120,251, 0 / 0 / 0 | 1,839 parents already had a prefix; none stacked; 5,032 embedded originals byte-identical |
| Dovecot mbox | 2,858 | 11,432 | 0 | 68,592 each, 0 | 1,524 prefixed parents; 2,858 embedded originals byte-identical |
| Mime4j test messages (80 files, 4 mbox items) | 78 | 312 | 0 | 1,872 each, 0 | 78 embedded originals byte-identical |
| MSG (Tika, POI) | 45 | 180 | 0 | 1,080 each, 0 | MAPI threading properties unchanged through MAPI to MIME to MAPI on 180 / 180 |
| PST, OST | 25 | 100 | 0 | 600 each, 0 | same, 100 / 100 |
| TNEF | 7 | 28 | 0 | 168 each, 0 | same, 28 / 28 |
| OLM | 490 | 1,960 | 0 | 11,760 each, 0 | records have no MIME source: the embedded original is rebuilt from the view (`ParentSynthesized`, 490) |
| DBX | 2,053 | 8,212 | 0 | 49,272 each, 0 | most records of the corpus have no text (no sender, no ID: `NoReplyTarget` 4,038); 34 non-empty originals byte-identical |

Known gaps: addresses with a doubled or edge dot in the local part (`k..allen@x`) are written as they were; the three oracles drop or cut them (ambiguity A15, A16),
so they are left out of that comparison. Display text that the user would see in other languages (attribution, forward banner) is English unless replaced. The
subject prefix list is an agent default. No mail client has been compared yet (no Thunderbird run); the Outlook oracle will settle the forward-threading and
conversation-index choices. Real Outlook conversation indexes carry the FILETIME from their first byte, which differs from the layout MS-OXOMSG draws (see `composition.md`).

## Display HTML (sanitiser, remote-content policy)

Added after 0.2.0-preview.2. Vectors are our own, written from the categories of public evasion lists (no text copied). DOMPurify 3.4.17 (npm) is run as a released program
only. What the code does is in [`display-html.md`](https://recursiveai.org/onemail/docs/display-html/); the sanitiser has not had its independent review yet.

| Area | Checked against | Result | Known gaps |
|---|---|---|---|
| Vectors in headless Chrome (per-vector iframes) | 233 vectors: raw page, sanitised with the block policy, sanitised with the allow policy | raw: 42 script executions and 58 requests; block policy: 0 executions, 0 requests; allow policy: 0 executions, 29 paths fetched, none from a removed element | one browser engine |
| DOMPurify 3.4.17, default configuration, same scorer in the same page | the same 233 vectors | ours 0 findings, DOMPurify 89, raw 194; ours looser in 0 categories | |
| Real mail HTML kept | HTML bodies of 45 MSG, 18 PST, 6 TNEF, 5 Mime4j messages, plus 123 RTF-to-HTML outputs | text preserved in every body (0 differ), output unchanged by a second pass (0 unstable); 22 remote references and 4 tracking indications reported in the mail bodies | the CMU Enron maildir has no HTML parts |
| Fuzzing | `sanitizer` (checks the output for banned elements, attributes and URL schemes) and `sanitizer-reparse` (output is a fixed point of parse and sanitise), 600 s each | `sanitizer` 6,533,781 executions, `sanitizer-reparse` 6,917,798, 0 findings (libFuzzer, Checked build). Earlier runs found a `pre` newline that did not survive a re-parse, a deeply nested misnested anchor that never settled (now unwrapped), and two checker false positives | |

## Speed

MIME rows against MimeKit 4.18.2: see [`benchmarks.md`](https://recursiveai.org/onemail/docs/benchmarks/) (geometric mean of the OneMail/MimeKit time ratio
0.091 on x64 and 0.093 on arm64; worst row 0.584).


<!-- https://recursiveai.org/onemail/docs/benchmarks/ -->

# Benchmarks: MIME rows

`bench/OneMail.Benchmarks` compares OneMail with MimeKit 4.18.2 (a released NuGet package, referenced only from `bench/`) on
the same bytes, in memory, with BenchmarkDotNet 0.15.8. `bench/run-mime-rows.sh OUTDIR` builds, runs every row
(3 launches, 3 warm-up and 10 measured iterations each), measures peak memory in separate processes and prints the table.
Run it alone on the machine. The ratio is OneMail median time / MimeKit median time; below 1 is faster. The bar is a geometric
mean ≤ 1.0 across rows and no row above 1.25.

## Inputs

Picked deterministically from the corpora that `tools/corpus/fetch.sh` fetches (never committed):

| Name | What |
|---|---|
| `enron-2k`, `enron-20k` | the first file, in path order, of the 1-in-100 Enron sample of 1.5–2.5 KB and of 15–25 KB |
| `dovecot-multipart` | the first multipart message of 40–300 KB in the Dovecot test mbox |
| `generated-attachment-6m` | a text part and a 4.5 MB random attachment as base64 (about 6 MB), fixed seed |
| mbox | the whole Dovecot test mbox, 2,859 messages |

## What each side does

| Row | OneMail | MimeKit |
|---|---|---|
| parse | `MimeEntity.Parse` over the bytes | `MimeMessage.Load` over a `MemoryStream` of the bytes |
| headers | parse, then Subject, From and To (address lists), Date, Message-ID, decoded | parse, then the same five properties |
| bodies | parse, then `GetDecodedContent()` of every leaf | parse, then `Content.DecodeTo` a fresh `MemoryStream` for every `MimePart` in `BodyParts` |
| write | `WriteTo(Stream.Null)` of a parsed, unmodified message (parse not timed) | `WriteTo(Stream.Null)` of a parsed message (parse not timed) |
| build | a multipart with a text part and a 1 MB attachment (base64), written to `Stream.Null` | the same with `Multipart`, `TextPart`, `MimePart` |
| mbox | one shared splitter (a `From ` line after a blank line opens a message), then each message parsed whole | the same splitter, then `MimeMessage.Load` of each range |

Things that make the rows differ in kind, not only speed:

- OneMail's parse reads headers and structure and leaves bodies in the input as ranges; MimeKit's default parse copies bodies
  into content streams. The parse and write rows therefore reward the lazy design; the bodies row is the like-for-like one.
- MimeKit re-serialises its model on write; OneMail copies untouched ranges, which is what keeps output byte-identical.
- The mbox row is parse per message over a split, not a mailbox reader. MimeKit's own mbox-mode parser is not used: it
  starts a message at every `From ` line, including ones inside bodies, and throws on the fragments of this file.
- Peak memory is measured by running one row's work in a loop for four seconds in its own process. The working-set figure is
  dominated by the runtime's allocation budget on a machine with a large cache, so read the allocation columns for memory behaviour.

## Results (first run, no tuning needed)

Medians in ns per operation (per message for mbox); B/op is bytes allocated per operation.

| Row | Input | OneMail x64 | MimeKit x64 | ratio x64 | ratio arm64 | OneMail B/op | MimeKit B/op |
|---|---|---:|---:|---:|---:|---:|---:|
| parse | enron-2k | 1,020 | 13,451 | 0.076 | 0.106 | 5,280 | 19,448 |
| parse | enron-20k | 865 | 21,389 | 0.040 | 0.052 | 4,360 | 39,360 |
| parse | dovecot-multipart | 4,830 | 68,927 | 0.070 | 0.110 | 9,920 | 171,802 |
| parse | generated 6 MB | 96,646 | 1,924,495 | 0.050 | 0.072 | 5,344 | 6,318,608 |
| headers | enron-2k | 2,797 | 16,699 | 0.167 | 0.205 | 10,328 | 22,344 |
| headers | enron-20k | 2,267 | 22,356 | 0.101 | 0.110 | 8,328 | 41,400 |
| headers | dovecot-multipart | 6,588 | 76,763 | 0.086 | 0.128 | 15,088 | 174,570 |
| headers | generated 6 MB | 156,043 | 1,877,793 | 0.083 | 0.073 | 10,688 | 6,321,655 |
| bodies | enron-2k | 1,393 | 13,371 | 0.104 | 0.125 | 8,160 | 20,904 |
| bodies | enron-20k | 3,723 | 21,579 | 0.173 | 0.128 | 49,496 | 101,208 |
| bodies | dovecot-multipart | 154,182 | 321,836 | 0.479 | 0.351 | 503,617 | 556,690 |
| bodies | generated 6 MB | 6,057,218 | 13,025,692 | 0.465 | 0.331 | 16,778,003 | 18,583,957 |
| write | enron-2k | 63 | 1,598 | 0.039 | 0.033 | 184 | 1,752 |
| write | enron-20k | 352 | 14,862 | 0.024 | 0.018 | 184 | 4,696 |
| write | dovecot-multipart | 1,965 | 88,760 | 0.022 | 0.020 | 184 | 6,912 |
| write | generated 6 MB | 94,852 | 3,611,187 | 0.026 | 0.022 | 184 | 5,912 |
| build | text + 1 MB attachment | 530,959 | 908,433 | 0.584 | 0.331 | 4,112,652 | 36,782 |
| mbox | dovecot, per message | 2,792 | 23,410 | 0.119 | 0.184 | 8,475 | 28,393 |

Geometric mean 0.091 (x64, AMD Ryzen 9 7950X3D, 32 logical cores) and 0.093 (arm64, Apple M2, 8 cores); worst row 0.584 and 0.351.
The one row where OneMail allocates more is `build`: it base64-encodes the attachment when the content is set, where MimeKit
encodes while writing.

# Benchmarks: Outlook-format rows (PST/OST, MSG, TNEF)

`bench/run-outlook-rows.sh OUTDIR` runs these rows. They use the same harness as the MIME rows (BenchmarkDotNet 0.15.8, 3
launches, 3 warm-up and 10 measured iterations; the 10k and 50k stores use 3 launches of 1 warm-up and 3 iterations because one
walk takes seconds), the same ratio (OneMail median / comparator median, below 1 is faster) and the same bar: geometric mean ≤ 1.0
per area, no row above 1.25. Arguments after `OUTDIR` replace the default BenchmarkDotNet filter and skip the extra steps.

## Comparators

Each is the released NuGet package, referenced only from `bench/` and used through its public API. Versions are pinned in
`Directory.Packages.props`.

| Area | Comparator | Version | Licence | Benchmarking permitted |
|---|---|---|---|---|
| PST/OST | `XstReader.Api` | 1.0.6 (2022-12) | Microsoft Public License (MS-PL): "Copyright (c) 2016, Dijji; 2021, iluvadev, and released under Ms-PL License" | Yes: MS-PL grants the right to use the software; nothing is copied, bundled or redistributed |
| MSG | `MsgReader` | 6.2.0 | MIT | Yes |
| TNEF | `MimeKit` (`TnefPart`) | 4.18.2 | MIT | Yes |
| PST/OST, native reference | libpff `pffexport` | 20180714 (Ubuntu noble `pff-tools`) | LGPL-3.0-or-later | Run as a binary only; reported, not gating |

The comparator packages pull in a `System.Security.Cryptography.Pkcs` build that carries an advisory (NU1902/NU1903). The bench
project is never shipped, so it suppresses those two audit warnings and nothing else.

## Inputs

| Name | What |
|---|---|
| `cfreds-ost` | the NIST CFReDS informant's Outlook 2013 OST (14 messages, 49 folders); x64 only (the fetch needs a 7-Zip and 20 GB) |
| `tika-pst`, `tika-bodytypes` | Apache Tika's two Unicode PSTs (7 and 4 messages) |
| `gen-u1k`, `gen-u10k`, `gen-u50k` | generated Unicode stores of 1,000, 10,000 and 50,000 messages (17 MB, 165 MB, 826 MB) over 20 folders; per message a plain body, an HTML body for every second one, one to four recipients, and an attachment on 70 % (2 to 22 KB, one in 500 is 1 MB); 50,000 messages carry 490 MB of attachments |
| `gen-k4-1k`, `gen-k4-10k` | the same mail in the 4 KiB-page layout (OneMail only, see below) |
| `msg-smallest`, `msg-median`, `msg-largest`, `msg-corpus` | the 53 Tika and POI MSG files that both libraries open (of 58): the smallest, the median and the largest by size (one has 1,321 recipients), and all 53 per operation |
| `poi-*`, `tika-testwinmail` | the six TNEF streams of the POI and Tika corpora with content (the 53-byte `oom.tnef` has none) |

The generated stores come from the test-side PST writer (`tests/OneMail.Tests/Pst`), which now leaves a free slot for each AMap
page a larger file needs (MS-PST 2.2.2.7.2) and sets `MSGFLAG_HASATTACH`. libpff 20180714 opens the stores (`pffinfo`, and
`pffexport -m items` exports them), OneMail and XstReader read them to the same counts, and a unit test builds a 20 MB store in every
layout and reads it back with no defects. libpff's recovery scan (`pffexport -m all`) fails on them, so the reference below uses
`-m items`. The writer holds the file in one array, so the largest store it can make is about 1.5 GB; the 826 MB store is the
largest measured.

## What each side does

Every message: subject, sender name and address, submit time, every recipient (name and address), the body, and every attachment's
bytes read to the end into a discarded buffer. A `verify` step runs both sides over every input first and prints what each saw
(`verify.tsv`); messages, recipients, attachments and attachment bytes agree everywhere except as listed below.

| Row | OneMail | Comparator |
|---|---|---|
| PST | `PstStore.TryOpen(path)`, every folder, `EnumerateMessages(Normal)`, best body (stored HTML, else RTF decompressed and de-encapsulated, else text), `OpenData()` per attachment | `new XstFile(path)`, `RootFolder`, `Folders`/`Messages` recursively, `Body.Text`, `SaveToStream` into a counting stream per file attachment; `message.ClearContents()` off |
| MSG | `MsgFile.TryOpen(path)`, then the same fields and all three bodies (text, HTML from the property or de-encapsulated from RTF, RTF decompressed) | `new Storage.Message(path)`, `Subject`, `Sender`, `Recipients`, `SentOn`, `BodyText`, `BodyHtml`, `BodyRtf`, `Attachment.Data` |
| TNEF | `TnefMessage.Read(bytes)`, then the PST fields | `TnefPart.ConvertToMessage()` over the bytes, then the message's fields, body and `Attachments` decoded to a counting stream |

Things that make the rows differ in kind or in work:

- **`ClearContents`.** XstReader documents `ClearContents()` as the way to release what a message cached. On the 50,000-message store
  it made the walk 14 times slower (139 s against 9.9 s) and its peak working set no lower (157 MiB against 150), so the timed rows
  leave it off: the comparator's fastest correct use. The memory section shows both.
- **`XstFile.Dispose`.** It starts a background release that reopens the file; in a loop over a short walk that thread threw after
  the file was closed and ended the process, on Linux and on macOS. The rows leave the file to the garbage collector instead.
- **4 KiB layout.** XstReader.Api 1.0.6 throws `XstException: Unexpected header in compressed data stream` on every 4 KiB-page store
  this writer makes, so those rows have no comparator and no ratio (OneMail alone: 1,000 messages in 92 ms on x64 and 100 ms on
  arm64, 10,000 in 1.19 s and 1.35 s). No real 4 KiB store was available to tell a writer fault from a reader gap
  (libpff 20180714 opens the layout but cannot read tables stored in data trees, which these stores use).
- **MSG corpus.** MsgReader read 40 attachments (268,647 bytes) where OneMail read 41 (274,743 bytes) across the 53 files: one
  6,096-byte attachment is not handed out by MsgReader. The ratio is on slightly unequal work, in the comparator's favour.
- **Folders.** The folder counts differ (OneMail also counts search folders, 49 against 41 in the OST); messages agree.
- **PST bodies.** XstReader hands out one body (`Body.Text`); OneMail's PST row reads the best one, so the work is matched. MSG
  reads all three on both sides because MsgReader computes all three on request.
- **Memory.** Peak working set is measured in a separate process that repeats one row's work for at least four seconds, with
  `DOTNET_GCgen0MaxBudget` set to 32 MiB so the first-generation budget of a large-cache machine does not hide the live set. A
  background thread samples the working set every 10 ms (the runtime's own peak counter reads 0 on macOS). Single-message
  rows (MSG, TNEF) sit on the runtime's baseline and say nothing; the PST rows and the curves are the memory evidence.

## Results

Medians; B/msg is bytes allocated per message (for `msg-corpus`, per file). Ratios are OneMail / comparator.

| Area | Input | Msgs | OneMail x64 | Comparator x64 | ratio x64 | ratio arm64 | OneMail B/msg | Comparator B/msg |
|---|---|---:|---:|---:|---:|---:|---:|---:|
| PST | cfreds-ost | 14 | 3.71 ms | 194.15 ms | 0.019 | n/a | 95,386 | 605,609 |
| PST | tika-pst | 7 | 596.4 µs | 2.29 ms | 0.261 | 0.252 | 84,463 | 437,083 |
| PST | tika-bodytypes | 4 | 490.6 µs | 3.00 ms | 0.164 | 0.169 | 84,298 | 767,458 |
| PST | gen-u1k | 1,000 | 71.63 ms | 176.79 ms | 0.405 | 0.409 | 19,485 | 220,845 |
| PST | gen-u10k | 10,000 | 714.7 ms | 1,800.5 ms | 0.397 | 0.408 | 19,374 | 219,661 |
| PST | gen-u50k | 50,000 | 4,117 ms | 9,205 ms | 0.447 | 0.475 | 19,371 | 220,179 |
| MSG | msg-smallest | 1 | 21.4 µs | 46.5 µs | 0.460 | 0.569 | 107,362 | 80,801 |
| MSG | msg-median | 1 | 137.9 µs | 876.0 µs | 0.157 | 0.180 | 293,905 | 1,272,393 |
| MSG | msg-largest | 1 | 11.71 ms | 440.99 ms | 0.027 | 0.023 | 17,491,388 | 53,247,584 |
| MSG | msg-corpus | 53 | 21.75 ms | 484.67 ms | 0.045 | 0.043 | 656,360 | 1,925,639 |
| TNEF | poi-simple | 1 | 9.1 µs | 38.7 µs | 0.234 | 0.243 | 110,069 | 62,446 |
| TNEF | poi-sample1 | 1 | 35.7 µs | 140.2 µs | 0.255 | 0.281 | 119,754 | 427,807 |
| TNEF | tika-testwinmail | 1 | 72.0 µs | 224.7 µs | 0.320 | 0.370 | 181,977 | 276,927 |
| TNEF | poi-quick | 1 | 84.1 µs | 230.6 µs | 0.365 | 0.372 | 181,982 | 276,930 |
| TNEF | poi-attachments | 1 | 93.7 µs | 260.7 µs | 0.360 | 0.239 | 124,163 | 1,204,040 |
| TNEF | poi-bug63955 | 1 | 146.1 µs | 397.1 µs | 0.368 | 0.248 | 173,098 | 1,880,228 |

| Area | Geomean x64 | Worst x64 | Geomean arm64 | Worst arm64 | Bar |
|---|---:|---:|---:|---:|---|
| PST/OST vs XstReader.Api | 0.197 (6 rows; 0.314 without the OST) | 0.447 | 0.320 (5 rows, no OST) | 0.475 | met |
| MSG vs MsgReader | 0.096 | 0.460 | 0.101 | 0.569 | met |
| TNEF vs MimeKit | 0.312 | 0.368 | 0.287 | 0.372 | met |

Both machines pass every area and every row with room to spare, so no tuning was done. Two rows allocate more than the comparator: the
smallest MSG (107 KB against 81 KB) and the smallest TNEF stream (110 KB against 62 KB). Both are fixed per-message costs that
vanish into the time ratio (0.46 and 0.23).

### Memory over the large stores

Peak working set, repeating the walk (KiB, OneMail / XstReader.Api, `ClearContents` off):

| Store | x64 | arm64 |
|---|---|---|
| gen-u1k (17 MB) | 102,136 / 116,900 | 66,672 / 90,416 |
| gen-u10k (165 MB) | 101,304 / 123,296 | 66,240 / 105,632 |
| gen-u50k (826 MB) | 101,080 / 152,792 | 66,320 / 119,488 |

OneMail's working set does not move between 1,000 and 50,000 messages (a file 50 times larger, 826 MB, in about 100 MB or 66 MB,
most of it the runtime and its allocation budget). XstReader.Api grows with the store. The `curve` samples agree (working set in
MiB at 10 %, 25 %, 50 %, 75 % and 95 % of the 50,000-message walk): OneMail x64 88, 95, 98, 98, 99 and arm64 58, 63, 63, 64, 64;
XstReader.Api x64 102, 118, 129, 142, 146 and arm64 70, 85, 95, 108, 114. With `ClearContents` after every message its working set
is flat but noisy (x64 75, 86, 135, 129, 119) and the walk takes 139 s on x64 and 144 s on arm64. The generated stores top out at
826 MB, so the claim is for that size; a 50 GB store needs a streaming test writer.

### Native reference

`pffexport -q -m items -f text` (libpff 20180714, x64, writes every item to disk, so it is not like-for-like), wall seconds, median
of 3: gen-u1k 0.24 s (OneMail 0.072 s), gen-u10k 2.73 s (0.715 s), gen-u50k 20.09 s (4.12 s), peak RSS 207 MB for 50,000 messages.
The real files take under 0.03 s. Reported, not gating.

## Machines and runs

| | x64 | arm64 |
|---|---|---|
| CPU | AMD Ryzen 9 7950X3D, 32 logical cores, AVX-512 | Apple M2, 8 cores |
| Runtime, SDK | .NET 10.0.13, SDK 10.0.402 | .NET 10.0.11, SDK 10.0.400 |
| Job | `wq` bench 000847, 14:32 to 15:01 UTC, 28.5 min of running | `wq` bench on the Mac, 14:01 to 14:26 UTC |
| Load, 1/5/15 min, before | 6.01 / 8.49 / 10.41 (the previous job's tail) | 2.08 / 3.19 / 3.11 |
| Load, 1 min, during (every 2 min) | 1.56 to 2.93, almost all of it this job | 2.05 to 2.42 |
| Load after | 1.65 / 1.89 / 3.45 | 1.77 / 2.00 / 2.26 |

BenchmarkDotNet 0.15.8, MimeKit 4.18.2, MsgReader 6.2.0, XstReader.Api 1.0.6. The raw tables are `report.tsv`, `peak.tsv`,
`curve.tsv`, `verify.tsv` and `pffexport.tsv` in each run's output directory. Two script faults found after the runs and fixed in
the script: the TNEF peak row for `tika-testwinmail` was routed to the PST path and has no line (it is the same 66,276 bytes as
`poi-quick`), and on macOS the `theirs-clear` tag in `curve.tsv` was not applied (the second `theirs` block per store is the
`ClearContents` run).

# Benchmarks: calendar and contact rows

`bench/run-calendar-rows.sh OUTDIR` runs these rows with the same harness, ratio and bar as above (OneMail median / comparator
median; geometric mean ≤ 1.0 per area, no row above 1.25). The 50k-event rows use 3 launches of 1 warm-up and 3 iterations.

## Comparators and what each side does

Ical.Net 5.2.3 and FolkerKinzel.VCards 8.2.0, released NuGet packages used through their public API, referenced only from `bench/`.

| Area | OneMail.Calendar | comparator |
|---|---|---|
| ICS parse | `ContentDocument.Parse` over the bytes, then a typed pass over every event (uid, summary, description, location, start, end, rule count, attendees) | `Calendar.Load` over the text, then the same fields |
| ICS build-write | create the events from fields (typed setters), `ToArray` | create the events, `CalendarSerializer` |
| ICS edit-write | parsed calendar, set every SUMMARY, write (unchanged lines go out as read) | parsed calendar, set every Summary, serialise (everything is re-rendered) |
| vCard parse | `ContentDocument.Parse`, then name, FN, e-mails, phones, addresses | `Vcf.Parse`, then the same |
| vCard build-write / edit-write | as for ICS | `VCardBuilder` and `Vcf.AsString` (3.0) |
| Recurrence | `CalendarRecurrence.Expand` over a parsed calendar and a window | `Calendar.GetOccurrences` over a loaded one, stopping at the window end |
| jCal / jCard | `JsonForms.ToJson` / `FromJson`: OneMail only | none |

`cverify` checks equal work before any timing: the parse and expansion checksums are equal on all 10 rows (events, cards and
occurrence counts), and the written sizes agree to within 39 bytes (ICS) or 4% (vCard, whose comparator writes extra defaults).
Inputs: a generated invite (1 event, 833 B), the largest Thunderbird holiday calendar (700 events, 291 KB), a generated 50k-event
file (32.8 MB), a generated card (430 B) and a 10k-card address book (4.5 MB); recurrence cases daily UTC, weekly floating, monthly
last-Friday in New York, yearly second Sunday of March in Berlin and daily in New York over 50 years.

## Results, x64 (AMD Ryzen 9 7950X3D, 32 cores, .NET 10.0.402, BenchmarkDotNet 0.15.8, load about 2.6 before, during and after)

| Area | Input | OneMail ns | comparator ns | ratio | OneMail B/item | comparator B/item |
|---|---|---:|---:|---:|---:|---:|
| ICS parse | invite | 3,917 | 38,967 | 0.101 | 9,328 | 113,264 |
| ICS parse | holidays | 931,933 | 14,621,595 | 0.064 | 3,177 | 45,284 |
| ICS parse | 50k events | 420,817,599 | 4,160,494,118 | 0.101 | 6,033 | 66,904 |
| ICS build-write | 1 / 1k / 50k | 3,044 / 4.39 M / 632.6 M | 23,819 / 49.5 M / 3,876 M | 0.128 / 0.089 / 0.163 | 8,856 (50k) | 65,449 |
| ICS edit-write | invite (holidays and 50k in the job's report; worst 0.058) | 460 | 18,922 | 0.024 | 3,424 | 53,033 |
| vCard parse | 1 card | 2,242 | 2,548 | 0.880 | 6,256 | 8,856 |
| vCard parse | 10k cards | 64,428,106 | 61,611,407 | 1.046 | 5,500 | 7,770 |
| vCard build-write | 1 / 10k | 2,059 / 79.2 M | 5,296 / 127.8 M | 0.389 / 0.620 | 7,466 | 12,171 |
| vCard edit-write | 1 / 10k | 248 / 6.79 M | 3,263 / 47.6 M | 0.076 / 0.143 | 2,278 | 6,283 |
| Recurrence | daily UTC 3,653 / weekly 1,566 / monthly NY 240 / yearly Berlin 50 / daily NY 18,263 occurrences | 515 µs / 230 µs / 168 µs / 41 µs / 9.97 ms | 4.31 ms / 1.67 ms / 371 µs / 88 µs / 22.5 ms | 0.119 / 0.138 / 0.452 / 0.470 / 0.443 | 2–200 B/occurrence | 3.3–4.6 KB |

Geometric mean / worst: ICS parse 0.087 / 0.101, ICS build-write 0.123 / 0.163, ICS edit-write 0.040 / 0.058, vCard parse **0.959 / 1.046**,
vCard build-write 0.491 / 0.620, vCard edit-write 0.104 / 0.143, recurrence 0.274 / 0.470. Every area is inside the bar.
jCal/jCard (OneMail only): write 6.8 µs for the invite, 3.8 ms for the holidays, 1.30 s for 50k events; read 10.8 µs, 4.6 ms, 2.77 s.
Native reference (libical 3.0.17 from the distribution package, parse only): invite 14.1 µs, holidays 3.88 ms, 50k events 526 ms
(OneMail's parse to the tree and typed pass: 3.9 µs, 0.93 ms, 421 ms).

## Tuning (before / after, same machine)

| Row | before | after | change |
|---|---:|---:|---|
| vCard parse 10k | ratio 1.882, 11,484 B/card | 1.046, 5,500 B/card | shared strings for known names, no per-line parameter list or object until asked, parameter values read straight into their array, a card's item list allocated once, typed views without LINQ |
| vCard parse 1 card | 1.516 | 0.880 | same |
| Recurrence yearly Berlin | 1.284 | 0.470 | a yearly rule with `BYMONTH` scans the selected months, not 365 days |
| ICS parse (all) | 0.133 geomean | 0.087 | same parse changes |

## Results, arm64 (Apple M2, 8 cores, .NET 10.0.400, load 1.7-2.1 before, 3.0-4.3 shortly after start, 1.9-2.4 at the end of each group)

Three filtered groups, each its own queued bench job of at most 20 minutes (ICS parse and edit; ICS build and vCard; recurrence and
jCal), run after the tuning above. The first column is before tuning (one 43-minute matrix), the second after.

| Area | geomean before | geomean after | worst after | status |
|---|---:|---:|---:|---|
| ICS parse | 0.132 | 0.104 | 0.151 | pass |
| ICS build-write | 0.132 | 0.153 | 0.248 | pass |
| ICS edit-write | 0.059 | 0.040 | 0.074 | pass |
| vCard parse | 1.530 (worst 1.707) | **1.093** | **1.181** (10k cards; 1 card 1.011) | **misses the bar** |
| vCard build-write | 0.442 | 0.558 | 0.674 | pass |
| vCard edit-write | 0.121 | 0.111 | 0.146 | pass |
| Recurrence | 0.412 (worst 1.527) | 0.334 | 0.589 (daily NY; yearly Berlin 0.551) | pass |

vCard parse, 10k cards: 79.4 ms against 67.2 ms, 5,500 B per card against 7,770. The M2 runs the comparator's parser relatively
faster than the x64 machine does (x64 ratio 1.046 on the same rows); a further cut needs fewer objects per property in the tree.


<!-- https://recursiveai.org/onemail/docs/ambiguities/ -->

# Ambiguity table (version 1)

Real mail is full of messages that the RFCs call invalid or leave open, and the established libraries each pick their own
answer. OneMail picks **one documented answer per case**, reports it as a typed defect, and keeps the original bytes so
nothing is lost whichever answer a caller prefers. This file lists each case, what three released libraries do with it, and
the resolution OneMail takes.

## How the table was made

- Each case is a small invented message in [`testdata/differential`](https://github.com/recursive-ai-oss/onemail-csharp/tree/bb4c9543917ab4ad466b5200d132e1899cb3ca14/testdata/differential) (generated by
  [`tools/corpus/make_differential.py`](https://github.com/recursive-ai-oss/onemail-csharp/blob/bb4c9543917ab4ad466b5200d132e1899cb3ca14/tools/corpus/make_differential.py); `cases.tsv` there describes every file).
- Each message was run through MimeKit 4.18.2, Python 3.13 `email` (`policy.default`, `message_from_bytes`) and Go 1.26
  `net/mail` + `mime/multipart`, through the drivers in [`tools/oracles`](https://github.com/recursive-ai-oss/onemail-csharp/tree/bb4c9543917ab4ad466b5200d132e1899cb3ca14/tools/oracles). The columns below are what
  those runs showed. Where the three agree, OneMail does the same.
- Where they differ, the rule is: prefer the answer two of the three give; otherwise prefer the answer that loses no
  bytes and follows the RFC text most closely. No mail client has been observed for any entry yet, so **every resolution
  that is not "all three agree" is provisional**: it is a default, to be revisited when a client's behaviour is observed.
  Provisional entries are marked *prov.*
- "Defect" is the typed defect the parser raises; every repair and every guess carries the byte offset. Names are the
  intended catalogue names; `Ambiguous*` marks a choice between valid readings, the others mark a repair.
- Compared forms, normalisation and the difference classes are in [`tools/oracles/README.md`](https://github.com/recursive-ai-oss/onemail-csharp/blob/bb4c9543917ab4ad466b5200d132e1899cb3ca14/tools/oracles/README.md).

Abbreviations: **MK** MimeKit, **Py** Python, **Go** Go standard library.

## Boundaries and multipart structure

| ID | Probe (`testdata/differential/…`) | MK | Py | Go | Resolution | Defect | Basis |
|---|---|---|---|---|---|---|---|
| B1 | `bnd-02` a content line `--abcdef` when the boundary is `abc` | content | content | content | A delimiter line is `--` + boundary + optional trailing whitespace; a longer line is content. | none | all agree |
| B2 | `bnd-03` trailing spaces and tabs after a delimiter | delimiter | delimiter | delimiter | Trailing space and tab are padding (RFC 2046 5.1.1). | none | all agree |
| B3 | `bnd-04` `--abc xyz`: text after the boundary | content | content | content | Not a delimiter; the line stays in the current part. | `AmbiguousBoundaryLikeLine` | all agree |
| B4 | `bnd-14` delimiter in another case (`--ABC`) | content | content | content | Boundaries are case-sensitive. | none | all agree |
| B5 | `bnd-07`, `bnd-08` one boundary is a prefix of the other | nested correctly | same | same | Longest-exact-match wins: a line closes or starts a part only if it equals a boundary in scope. | none | all agree |
| B6 | `bnd-06` text after the closing delimiter, then another `--abc` | epilogue | epilogue | epilogue | Everything after the closing delimiter is epilogue, including later delimiter-looking lines. | `EpilogueHasDelimiter` | all agree |
| B7 | `bnd-05` closing delimiter missing | last part keeps its final line break (5 bytes) | strips it (3) | strips it (3) | The last part runs to end of input and keeps every byte; no line break is assumed to belong to a delimiter that is not there. | `MissingFinalBoundary` | prov. (MK; keeps all bytes) |
| B8 | `bnd-09` multipart with no `boundary` | multipart, no parts, body not exposed | leaf, body kept | leaf, body kept | Treated as a leaf of the declared type with the body kept; no parts. | `MultipartWithoutBoundary` | 2 of 3 |
| B9 | `bnd-10` `boundary=""` | no parts | leaf | leaf | Same as B8. | `MultipartWithoutBoundary` | 2 of 3 |
| B10 | `bnd-25` multipart whose body has no delimiter at all | no parts | leaf, body kept | no parts | No parts; the whole body is the preamble, bytes preserved. | `MissingStartBoundary` | 2 of 3 |
| B11 | `bnd-11` boundary parameter that ends in a space | none found | finds a part | 2 parts | The boundary is matched literally and also with trailing space trimmed. | `BoundaryHasTrailingSpace` | prov. |
| B12 | `bnd-12` unquoted `boundary=a/b=c` | accepted whole | cut at `=` | media type rejected | The unquoted value runs to `;` or end of value. | none (not raised) | prov. (MK) |
| B13 | `bnd-13` two `boundary` parameters | first | first | media type rejected (leaf) | First occurrence wins. | `DuplicateParameter` | 2 of 3 |
| B14 | `bnd-15` multipart with only `--abc--` | 0 parts | leaf, empty | 0 parts | 0 parts. | none | 2 of 3 |
| B15 | `bnd-16` first delimiter directly after the headers, no blank line | header block never ends, 0 parts | 1 part | header error | The first line that is not a header field or continuation ends the header block and starts the body (see H5). | `MissingHeaderBodySeparator` | prov. (Py) |
| B16 | `bnd-17` LF-only delimiters inside a CRLF message | 2 parts | 2 parts | 1 part | Each line may end in LF or CRLF independently. | `MixedLineEndings` | 2 of 3 |
| B17 | `bnd-18` part with headers and no blank line before the next delimiter | 2 parts | 2 parts | 0 parts | Part with headers and an empty body. | `MissingHeaderBodySeparator` | 2 of 3 |
| B18 | `bnd-23` delimiters separated by bare CR | no parts | 1 part | no parts | A bare CR is not a line break (see L1). | `BareCarriageReturnInHeader` | 2 of 3 |
| B19 | `bnd-26` inner multipart reuses the outer boundary | nested | outer swallows inner | nested | The innermost open multipart is tried first, so the shared line closes the inner part. | `AmbiguousDuplicateBoundary` | prov. (MK, Go) |
| B20 | `bnd-19`, `bnd-20`, `bnd-21`, `bnd-22`, `bnd-24` empty part headers; blank line before delimiter; delimiter at end of file without line break; whitespace-only preamble and epilogue; 80-character boundary | same | same | same | The line break before a delimiter belongs to the delimiter; a part can have no headers (defaults apply); the boundary length limit is not enforced; whitespace-only preamble/epilogue counts as present but empty. | `BoundaryTooLong` for over 70 | all agree |

## Line endings and raw octets

| ID | Probe | MK | Py | Go | Resolution | Defect | Basis |
|---|---|---|---|---|---|---|---|
| L1 | `eol-02` whole message with CR-only line ends; `eol-05` CR inside a header value; `eol-09` CR CR LF | CR-only: one long header; CR in value dropped from the decoded value | CR is a line break | CR kept in the value | A line ends at LF or CRLF only. A lone CR is data: it stays in the raw value and is dropped from the decoded value. | `BareCarriageReturnInHeader` | prov. (MK, Go) |
| L2 | `eol-01`, `eol-03`, `eol-04`, `eol-06`, `eol-07`, `eol-08` LF-only; mixed CRLF/LF; no final newline; headers end at EOF | same | same | same | Accept any mix of LF and CRLF; a body's bytes are never changed. | `MixedLineEndings` when mixed | all agree |
| L3 | `raw-01`, `raw-09` NUL, form feed and vertical tab in a header value | kept | kept | kept | Kept in raw and decoded values. | `ControlCharacterInHeader` | all agree |
| L4 | `raw-02` NUL in a header name | header | body starts here | header | Accepted as a header name; the raw bytes are kept. | `InvalidHeaderName` | 2 of 3 |
| L5 | `raw-03`, `raw-07`, `raw-08` NUL in a body; 3000-character header; 20000-character body line | kept | kept | kept | No length limits below the configured resource limits; the defect `LineTooLong` marks lines over 998. | `LineTooLong` | all agree |
| L6 | `raw-04`, `raw-05`, `raw-11`, `adr-24` raw 8-bit octets in headers | UTF-8, else Latin-1 | `U+FFFD` for non-ASCII | invalid bytes become `U+FFFD`; one address list fails | A raw 8-bit header value is decoded as UTF-8 if valid, otherwise as Windows-1252 (RFC 6532 for the UTF-8 case). | `AmbiguousEightBitHeader` | prov. (MK) |
| L7 | `raw-10` UTF-8 byte-order mark before the first header | header found | body starts here | the BOM stays in the first name, which then goes unrecognised | The BOM is ignored when the name is read (the raw bytes keep it), so the first field is found. | `InvalidHeaderName` | prov. (MK) |
| L8 | `raw-06`, `cte-14`, `ct-14` 8-bit body declared 7bit or without charset | bytes kept | bytes kept | bytes kept | Body bytes are never altered; charset choice is the encodings layer's. | `EightBitIn7BitBody` | all agree |

## Header block

| ID | Probe | MK | Py | Go | Resolution | Defect | Basis |
|---|---|---|---|---|---|---|---|
| H1 | `hdr-01`, `hdr-02`, `hdr-03` duplicate `Content-Type`, `MIME-Version`, `Content-Transfer-Encoding` | first | first | first | The first occurrence is the one in force; all stay in the tree. | `DuplicateHeader` | all agree |
| H2 | `hdr-04`, `adr-25` duplicate `From`, `To`, `Subject`, `Date` | `From`/`To` lists merge; all `Subject`s kept | first | first | The typed accessors use the first occurrence; every header stays in the tree. | `DuplicateHeader` | 2 of 3 |
| H3 | `hdr-07` whitespace before the colon (`Subject : x`) | header | body starts here | header | Accepted; the name is trimmed (RFC 5322 4.2 obsolete syntax). | `WhitespaceBeforeHeaderColon` | 2 of 3 |
| H4 | `hdr-14` space inside a header name | header | body starts here | header | Accepted as written; the name is kept (Latin-1) and `InvalidHeaderName` is reported. | `InvalidHeaderName` | 2 of 3 |
| H5 | `hdr-05`, `hdr-06`, `hdr-09` a line without a colon in or after the headers | line skipped (or parse error on the first line) | body starts at the line | error | The first non-header line ends the header block and starts the body, so no byte is dropped. | `MissingHeaderBodySeparator` | prov. (Py) |
| H6 | `hdr-10` body line that looks like a header (`Note: x`) with no blank line | header | header | error | It is a header. | `MissingHeaderBodySeparator` | all agree where parsed |
| H7 | `hdr-08` continuation line before the first header | parse error | ignored | error | A line that starts with white space where a field should begin ends the header block and starts the body. | `HeaderContinuationWithoutHeader` | prov. |
| H8 | `hdr-11` `From ` envelope line first | skipped | skipped | skipped | Kept as the entity's envelope line (its own reader event), not a header; the header block starts after it. | none | all agree |
| H9 | `hdr-12` blank lines before the first header | empty header block | empty header block | empty header block | An empty header block; everything after is the body. | `NoHeaders` | all agree |
| H10 | `hdr-16`, `hdr-15`, `hdr-21` folding with tabs, runs of spaces and blank continuation lines | keeps white space | keeps white space | collapses to one space | Unfolding removes the line break only; interior white space stays. | none | 2 of 3 |
| H11 | `hdr-25` trailing white space on a value | trimmed | kept in the decoded value | trimmed | Trimmed in the decoded value, kept in raw. | none | 2 of 3 |
| H12 | `hdr-18` empty file (`hdr-19`, `hdr-20`: blank line only and headers only agree) | error | empty message | error | An empty message; the parser never throws. | `EmptyMessage` | prov. (Py) |
| H13 | `hdr-13`, `hdr-17`, `hdr-22`, `hdr-23`, `hdr-24`, `hdr-26` empty values, no MIME-Version, colons in values, odd case, 2000 headers, trace order | same | same | same | Names compare case-insensitively; a missing `MIME-Version` does not stop MIME parsing; order is preserved. | `MissingMimeVersion` | all agree |
| H14 | `nest-13` message cut inside a header name | error | body | error | The partial line is kept as the start of the body. | `TruncatedHeader` | prov. (Py) |

## Content-Type

| ID | Probe | MK | Py | Go | Resolution | Defect | Basis |
|---|---|---|---|---|---|---|---|
| C1 | `ct-02`, `ct-03`, `ct-04`, `ct-09` `text` without subtype, `text/`, garbage, empty value | `application/octet-stream` | `text/plain` (`text/` kept literal for `ct-03`) | `text/plain` or `text` | A value that is not `type/subtype` is treated as absent: `text/plain; charset=us-ascii`, raw header kept. | `InvalidContentType` | 2 of 3 |
| C2 | `ct-05` parameter without `=` | dropped | `charset=""` | dropped | Dropped. | `InvalidParameter` | 2 of 3 |
| C3 | `ct-07` comments in the value | one comment stays in the value | comment stays in the type | removed | Comments are removed (RFC 2045 5.1). | none | prov. (Go) |
| C4 | `ct-01`, `ct-06`, `ct-08`, `ct-10`, `ct-11`, `ct-12` odd case and spacing, unterminated quote, duplicate `charset`, `;;`, unknown charset, raw UTF-8 in a parameter | same | same | same | Lower-case types and parameter names; first duplicate wins; an unterminated quote runs to the end of the value; extra semicolons are ignored; unknown charsets are kept as written. | `UnterminatedQuote`, `DuplicateParameter` | all agree |
| C5 | `ct-13` top-level `message/rfc822` | nested message | nested message | nested message | Parsed as a message. | none | all agree |

## Content-Transfer-Encoding

| ID | Probe | MK | Py | Go | Resolution | Defect | Basis |
|---|---|---|---|---|---|---|---|
| E1 | `cte-01`, `cte-04`, `cte-19`, `cte-17`, `cte-12`, `cte-10`, `cte-11` unknown, odd-case, empty, `binary`; QP lower-case hex, soft breaks, invalid escapes | same | same | same | Unknown, empty, 7bit, 8bit, binary: identity. Names are case-insensitive and trimmed. QP: invalid escapes pass through as written. | `UnknownTransferEncoding`, `InvalidQuotedPrintable` | all agree |
| E2 | `cte-02`, `cte-03` composite type declared `base64` or `8bit` | parts parsed | parts parsed | decodes first, no parts | The declared encoding is ignored for `multipart/*`: parts are parsed from the raw body. | `AmbiguousEncodedComposite` | 2 of 3 |
| E3 | `cte-05` base64 with characters outside the alphabet | 9 bytes | 10 bytes | 3 bytes | Characters outside the alphabet are skipped, the rest is decoded as one stream. | `InvalidBase64` | prov. (Py) |
| E4 | `cte-06` base64 without padding | drops the last partial group (9) | decodes it (11) | drops it (9) | The final partial group is decoded. | `MissingBase64Padding` | prov. (Py; keeps data) |
| E5 | `cte-07`, `cte-20` white space and line breaks inside base64 | skipped | skipped | stops at white space | White space is skipped anywhere. | none | 2 of 3 |
| E6 | `cte-08` data after `=` padding | continues | continues | stops | Decoding continues after padding; each padded run is decoded on its own. | `DataAfterBase64Padding` | 2 of 3 |
| E7 | `cte-13` trailing spaces before a QP line end | kept | kept | stripped | Kept (they are data in transit; RFC 2045 6.7 rule 3 only forbids producing them). | none | 2 of 3 |
| E8 | `cte-15` `message/rfc822` declared base64 | not decoded | decoded, parsed | decoded, parsed | Stays a leaf whose body is the encoded bytes (decoding it first would break the byte-for-byte tree). | `AmbiguousEncodedComposite` | prov. (MK) |
| E9 | `cte-16` uuencode | decoded | decoded | not decoded | Decoded (`x-uuencode`, `uuencode`, `x-uue`, `uue`). | none | 2 of 3 |
| E10 | `cte-18` comment after the name (`base64 (note)`) | decoded | decoded | not decoded | Comments are removed first. | none | 2 of 3 |

## Parameters, encoded words and file names

| ID | Probe | MK | Py | Go | Resolution | Defect | Basis |
|---|---|---|---|---|---|---|---|
| N1 | `enc-01`, `enc-30` RFC 2047 encoded word inside a quoted or token parameter value | decoded | decoded (quoted only) | literal | Decoded, although RFC 2047 5 forbids it in parameters; every mail client does. | `AmbiguousEncodedWordInQuotedString` | 2 of 3 |
| N2 | `enc-04` `filename=` and `filename*=` together | extended | plain | extended | The extended (RFC 2231) value wins. | `AmbiguousParameterForms` | 2 of 3 |
| N3 | `enc-02`, `enc-03`, `enc-09`, `enc-05` RFC 2231 charset, continuations, out-of-order continuations | same | same | same | Continuations are joined by index before charset decoding (RFC 2231 3). | none | all agree |
| N4 | `enc-06` unknown charset in RFC 2231 | decoded as Latin-1 | `U+FFFD` | parameter dropped | Fall back to Windows-1252. | `AmbiguousUnknownCharset` | prov. |
| N5 | `enc-07` `filename*=` without the charset quote marks | Latin-1 mojibake | UTF-8 | parameter dropped | Percent-decode, then UTF-8 if valid, else Windows-1252. | `MalformedExtendedParameter` | prov. (Py) |
| N6 | `enc-08` continuations starting at `*1` | joined | dropped | dropped | Joined in index order. | `AmbiguousParameterSegments` | prov. (MK) |
| N7 | `enc-14` two `filename` parameters | first | first | parameter dropped | First wins. | `DuplicateParameter` | 2 of 3 |
| N8 | `enc-15` unquoted file name with a space | whole value | cut at the space | rejected | The value runs to `;` or end of value. | none (not raised) | prov. (MK) |
| N9 | `enc-10`, `enc-11` `name` versus `filename` | disposition filename, else type name | same | same | Disposition `filename` first, then Content-Type `name`. | none | all agree |
| N10 | `enc-12`, `enc-13` path separators, `..`, backslash escapes | as written | as written | as written | The name is kept as written; making it safe to write to disk is the caller's or the export layer's job. | `UnsafeFilename` | all agree |
| N11 | `enc-16`–`enc-20`, `enc-22`–`enc-24`, `enc-28`, `enc-29` encoded words: Q and B, adjacent, mixed with text, unknown charset, over-long, split by a fold, empty, in `Comments`/`Keywords` | same | same | same | RFC 2047 decoding; white space between adjacent encoded words is dropped. | `EncodedWordTooLong`, `AmbiguousSplitMultibyte` | all agree |
| N12 | `enc-21` encoded word with invalid base64 | word removed | word replaced by a space | kept literally | Left in the text as written (RFC 2047 6.3 allows displaying it as ordinary text). | `InvalidEncodedWordBase64`, `InvalidEncodedWordQuotedPrintable` | prov. (Go) |
| N13 | `enc-25` encoded word in a quoted display name | decoded | decoded | literal | Decoded. | `AmbiguousEncodedWordInQuotedString` | 2 of 3 |
| N14 | `enc-27` ISO-2022-JP encoded word | escape sequences left | decoded | not decoded | Decoded. | none | prov. (Py) |

## Addresses

| ID | Probe | MK | Py | Go | Resolution | Defect | Basis |
|---|---|---|---|---|---|---|---|
| A1 | `adr-01`, `adr-02`, `adr-27` group syntax, empty group | flattened | flattened | flattened | Groups are kept as groups in the model; the shared view flattens members. | none | all agree |
| A2 | `adr-03` unquoted comma in a display name | one mailbox, name `Doe, John` | two: `Doe` (no domain) and `John` | list rejected | One mailbox named `Doe, John`: a comma before `<` stays in the phrase. | `AmbiguousUnquotedCommaInPhrase` | prov. (MK) |
| A3 | `adr-04` bare address after plain words | second mailbox dropped | second mailbox with a wrong local part | list rejected | The words become the display name and the address is kept. | `MissingAngleBrackets` | prov. |
| A4 | `adr-05` name given as a trailing comment | name from the comment | no name | name from the comment | The comment becomes the display name when there is none. | none | 2 of 3 |
| A5 | `adr-06`, `adr-07`, `adr-20` nested comments, obsolete source route, spaces around `@` and `.` | same | same | list rejected | Comments are stripped, the route is dropped, CFWS inside the address is removed. | `ObsoleteRoute` | 2 of 2 that parse |
| A6 | `adr-08` quoted local part | quotes kept | quotes kept | unquoted | The address keeps its quoting as written; an unquoted form is available alongside. | none | 2 of 3 |
| A7 | `adr-09` empty address `<>` | mailbox dropped | kept, address `<>` | list rejected | A mailbox with an empty address and its name is kept. | `EmptyAddress` | prov. |
| A8 | `adr-10` local part without a domain | kept | kept | rejected | Kept as written: `john` and `mary@` are two mailboxes, each with `MissingDomain` (dropping a recipient loses data). | `MissingDomain` | prov. |
| A9 | `adr-11`, `adr-13`, `adr-14`, `adr-16`, `adr-21`, `adr-22`, `adr-26`, `adr-28` IP literals; empty list elements; unclosed `<`; dotted names; 50 entries; escaped quoted names; several `From`; empty `Cc`/`Bcc` | same | same | mostly same | Accepted as written; empty elements are skipped; an unclosed `<` is closed at the end of the value. | `UnclosedAngle` | all agree where parsed |
| A10 | `adr-12`, `adr-24` raw UTF-8 or Latin-1 in an address | name decoded; local part with UTF-8 dropped | `U+FFFD` | rejected | RFC 6532: UTF-8 in local part and domain is accepted; names follow L6. | `AmbiguousEightBitHeader` | prov. |
| A11 | `adr-15` unclosed quoted name | name `John`, address kept | whole value as address | rejected | Recover as MimeKit does: the open quote ends at the `<`. | `UnterminatedQuote` | prov. (MK) |
| A12 | `adr-17` `@` inside an unquoted display name | name kept | wrong address | rejected | The address is the one in angle brackets. | none | prov. (MK) |
| A13 | `adr-18` two address-specs in one mailbox (`<a@x><b@y>`) | both | first only | rejected | Both are kept as two mailboxes. | `AmbiguousAdjacentAngleAddrs` | prov. (MK) |
| A14 | `adr-19` semicolon as a separator | first only | first only | rejected | Only the first address is parsed; the remainder is kept as unparsed text in the defect. | `UnparsedAddressText` | 2 of 2 that parse |
| A15 | `adr-23` leading, trailing and doubled dots in a local part (`.a@x`, `b.@x`, `c..d@x`) | address dropped | kept | rejected | Kept as written. These are common in old corporate mail, and dropping them loses recipients. | `ObsoleteLocalPart` | prov. (Py; in about 9% of messages in a 517,000-message public corpus, MimeKit drops such recipients) |
| A16 | gateway-rewritten addresses seen in the Enron corpus: `<"a"@b.com@gw>`, `munn"."mary@gw`, `"d@x".@gw@gw` | MimeKit keeps the part up to the second `@`, or drops the address; Go rejects the list | Python gives `<>` or a mangled address | Kept as written, with `InvalidAddressSyntax`. | `InvalidAddressSyntax` | prov. (no common reading) |

## Dates

All forms are read into an instant plus the written zone; the source text is kept with the value.

| ID | Probe | MK | Py | Go | Resolution | Defect | Basis |
|---|---|---|---|---|---|---|---|
| D1 | `date-01`–`date-13`, `date-18`, `date-24`, `date-26`, `date-28`, `date-33` standard, no seconds, no weekday, wrong weekday, `EST`/`GMT`/`UT`, comment zone, lower-case month, extra white space, ±14:00 | same | same | same | Accept; the weekday is ignored; named zones follow RFC 5322 4.3 (`EST` −5, `PDT` −7 …); with two `Date` headers the first is used. | `WeekdayMismatch` | all agree |
| D2 | `date-34`–`date-37`, `date-03`–`date-05` two-digit years | 00–68 → 20xx; 69 → 2069; 70+ → 19xx | 00–68 → 20xx; 69–99 → 19xx | same as Py | 00–68 → 2000–2068, 69–99 → 1969–1999. All three libraries read `50` as 2050, which differs from RFC 5322 4.3 (50–99 → 19xx); OneMail follows the libraries. | `AmbiguousTwoDigitYear` | 2 of 3 (69) |
| D3 | `date-14` three-digit year | rejected | year 101 | rejected | Add 1900, as RFC 5322 4.3 says (101 → 2001). | `ThreeDigitYear` | prov. (RFC only) |
| D4 | `date-10`, `date-12`, `date-21`, `date-25` `-0000`, military letters, `GMT+2`, missing zone | UTC | UTC, zone unknown for `-0000` and none | `Z` and none rejected; `GMT+2` read as +2 | UTC with the zone marked unknown (RFC 5322 4.3: military zones are unreliable and read as `-0000`). | `UnknownTimeZone` | 2 of 3 |
| D5 | `date-16` seconds = 60 | 59 | rejected | rejected | Accept as a leap second, instant clamped to :59. | `LeapSecond` | prov. (MK) |
| D6 | `date-17` zone `+2400` | UTC | rejected | +24:00 | Out-of-range zones are read as unknown. | `InvalidTimeZone` | prov. |
| D7 | `date-19` `2001-05-14 23:39:00` | parsed as UTC | rejected | rejected | Rejected (not RFC 5322); the header stays as text. | `InvalidDate` | 2 of 3 |
| D8 | `date-22` hour 24 | wrong date (00:00 on the 14th) | rejected | rejected | Rejected. | `InvalidDate` | 2 of 3 |
| D9 | `date-27` year 1899 | rejected | accepted | accepted | Accepted. | none | 2 of 3 |
| D10 | `date-29` `PDT` | −7 | −7 | rejected | Accepted (obsolete zone, RFC 5322 4.3). | none | 2 of 3 |
| D11 | `date-30` fractional seconds | wrong time (00:00) | rejected | accepted | Fraction dropped. | `FractionalSeconds` | prov. (Go) |
| D12 | `date-15`, `date-20`, `date-23`, `date-31`, `date-32` garbage, ISO with `T…Z`, 31 April, empty, day 0 | rejected | rejected | rejected | Rejected; the value reads as absent and the header stays as text. | `InvalidDate` | all agree |
| D13 | `date-38` four-digit year with leading zeros (`0002`), which a client that once wrote a two-digit year can produce | 2002 | 2002 | year 2 | Read as a two-digit year (rule D2). | `AmbiguousTwoDigitYear` | 2 of 3 |

## Nested messages and structure

| ID | Probe | MK | Py | Go | Resolution | Defect | Basis |
|---|---|---|---|---|---|---|---|
| M1 | `nest-01`, `nest-02`, `nest-05`, `nest-17` `message/rfc822` plain, multipart inside, 8bit, 30 levels | nested | nested | nested | Parsed recursively, bounded by the nesting limit. | none | all agree |
| M2 | `nest-03`, `nest-04` `message/rfc822` that is empty or has no headers | message part without inner message, or body in an empty inner message | inner message with empty body | no inner message | An inner message exists whenever the part is `message/rfc822`; text without headers becomes its body. | `NoHeaders` | prov. |
| M3 | `nest-06`, `nest-07` `multipart/digest` children without a type | `message/rfc822` | `message/rfc822` | `text/plain` | `message/rfc822` (RFC 2046 5.1.5). | none | 2 of 3 |
| M4 | `nest-08`, `nest-09` `message/delivery-status`, `message/partial` | bytes | groups parsed as headers | bytes | Bytes are kept; the status groups are parsed on demand. | none | 2 of 3 |
| M5 | `nest-12` cut inside a part's header | partial part | partial part | no part | The part exists with its partial header. | `TruncatedHeader` | 2 of 3 |
| M6 | `nest-14`, `nest-15`, `nest-16` 10 and 200 nested multiparts, 3000 parts | all parsed | all parsed | all parsed | None of the three limits depth (200) or part count (3000). OneMail's limits are on by default and are listed in the resource-limit notes. | `DepthLimitExceeded`, `EntityLimitExceeded` | no oracle limit |
| M7 | `nest-10`, `nest-11`, `nest-18`, `nest-19`, `nest-20`, `nest-21` alternative/related, signed, QP HTML, inline and unknown dispositions, a lying `Content-Length`, mbox-like trailer lines | same | same | same | Structure as written; `Content-Length` is advisory and ignored; an unknown disposition is kept as written. | none | all agree |

## Defect names in this table that are not raised, and why

The decoders and the tree keep what they repair without a defect where the repair is the specified reading, or report it by another route.

| Name in the table | Where it is reported instead |
|---|---|
| `InvalidBase64`, `MissingBase64Padding`, `DataAfterBase64Padding`, `InvalidQuotedPrintable` (E1, E3, E4, E6) | Body decoding is lazy, so the decoders report `TransferIssues` flags on the decoding stream, not tree defects. |
| `DuplicateHeader` (H1, H2) | Not raised: all copies stay in the tree; only a repeated `Content-Type` or `Content-Transfer-Encoding` is a defect (`AmbiguousDuplicateContentType`, `AmbiguousDuplicateTransferEncoding`). |
| `NoHeaders`, `EmptyMessage`, `TruncatedHeader` (H9, H12, H14, M2) | Not raised: an empty or cut-off message is a valid empty tree; `MissingHeaderBodySeparator` marks a header block that ran into the end of input. |
| `MissingMimeVersion` (H13) | Not raised: a missing `MIME-Version` does not change parsing. |
| `LineTooLong`, `EightBitIn7BitBody` (L5, L8) | Not raised: body bytes are never judged. |
| `EnvelopeLine` (H8) | Not a defect: the reader emits an `EnvelopeLine` event. |
| `UnsafeFilename` (N10) | Not raised: a file name is kept as written. |
| `EncodedMessagePart` (E8) | Reported as `AmbiguousEncodedComposite`. |
| `UnquotedParameterSpecials` (B12, N8) | Not raised: an unquoted value runs to `;`. |

## Differences the differential run reports, and how each is explained

`tools/oracles/explained.tsv` maps every difference between OneMail and each oracle to an entry above, to a documented normalisation of
the comparison (`NORM-*`) or to a named limitation of that oracle (`LIMIT-*`). A difference with no line fails the run.

## Calendar and contacts (ICS, vCard)

Observed black-box on Python `icalendar` 7.3.0 (**Py-ical**) and `vobject` 0.9.9 (**Py-vobj**), over the public corpus and our probes (`tools/calendar/oracles`). Ical.Net 5.2.3 and FolkerKinzel.VCards 8.2.0 expose typed models only, so they settle value questions, not line syntax. The tree always keeps every byte (untouched input writes back identical); the rows below govern the decoded view and the defects. Every row is our default, marked provisional where fewer than two oracles agree.

| ID | Case | Py-ical | Py-vobj | Resolution | Defect | Basis |
|---|---|---|---|---|---|---|
| K1 | Line ends: LF only, lone CR, CR CR LF | accepts LF | accepts LF | CRLF, LF and lone CR all end a line. A run of two or more CRs then LF is one line end, so a fold after it still joins and no CR reaches the value. Reported once per file, at the first. | `BareLineFeed`, `BareCarriageReturn`, `AmbiguousDoubleCarriageReturn` | all agree for LF; CR prov.; CR CR LF 6 of 7 (probe below) |
| K2 | Fold with two whitespace characters after the break | one removed (`ab cd`) | one removed | Exactly one space or tab is removed (RFC 5545 §3.1); the rest is content. | none | all agree |
| K3 | Fold inside a multi-byte UTF-8 sequence | n/a | n/a | Lines are unfolded as bytes before decoding, so the character is whole. | none | RFC 5545 §3.1 |
| K4 | Line with no colon | skipped | parse error | Kept in place as a malformed line (written back as read); not a property. | `MissingColon` | prov. (lossless) |
| K5 | Space before the colon (`SUMMARY :x`) | accepts | parse error | Accepted; the name is trimmed. | `WhitespaceAroundDelimiter` | prov. (1 of 2) |
| K6 | Lower-case names and `begin:vcalendar` | accepts | accepts | Names and component names compare ASCII case-insensitively; case is kept as written. | none | all agree |
| K7 | `END:` that does not match the open component | error | error | Closes up to the matching open component; the components in between are left unclosed. With no match, the line is kept as malformed. | `MissingEnd`, `UnmatchedEnd` | prov. (both reject; ours must not) |
| K8 | RFC 6868 caret sequences in a parameter value (`^n`, `^^`, `^'`) | decoded | literal | Decoded. | `AmbiguousCaretEncoding` | prov. (1 of 2) |
| K9 | The same parameter twice (`ROLE=A,B;ROLE=C`) | last wins | merged | Both are kept in order as written; readers that want one value take the first. | none | lossless |
| K10 | Unknown text escape (`\q`, `\"`, `\:`) | `\q` and `\"` kept, `\:` becomes `:` | `\q` and `\:` kept, `\"` becomes `"` | The backslash and the character stay as written. Only `\\ \; \, \n \N` are resolved (vCard 2.1: `\\ \;`). | none | prov.; `\q` agrees |
| K11 | Escapes in `X-` properties | raw | unescaped | Unescaped like any TEXT value (RFC 5545 §3.8.8.2 gives `X-` properties the TEXT type). | none | prov. (1 of 2) |
| K12 | Invalid UTF-8 text | U+FFFD | U+FFFD | The line is read as Windows-1252, the reading clients use for legacy files, unless `CHARSET` names a charset we can decode. The source bytes are untouched. | `AmbiguousNonUtf8Text` | prov. (our default) |
| K13 | vCard 2.1 `CHARSET` we cannot decode | n/a | n/a | Read as UTF-8, or Windows-1252 when that is invalid. | `UnknownCharset` | our default |
| K14 | vCard 2.1 quoted-printable soft break (`=` at line end) | n/a | joined | Joined whenever the line is marked `QUOTED-PRINTABLE`, in vCard and ICS alike. | `QuotedPrintableSoftBreak` | prov. |
| K15 | vCard 2.1 base64 data on unindented lines, ended by an empty line | n/a | joined | Joined when the line carries `ENCODING=BASE64`/`B`/bare `BASE64` and the next line holds only base64 characters. The ending empty line belongs to the property. | `Base64ContinuationWithoutFold` | prov. |
| K16 | vCard 2.1 bare parameters (`TEL;WORK;VOICE`) | n/a | dropped on some inputs | Kept as bare parameters; the typed view lists them as types together with `TYPE=` values. | none | lossless |
| K17 | Comma lists with an empty element (`ADR` street `5,`) | n/a | element dropped | The empty element is dropped in the typed view; the raw value keeps it. | none | prov. |
| K18 | Properties inside `VCALENDAR` after `BEGIN:VCARD` junk, stray lines around components | n/a | n/a | Kept as items of the document root, in order. | `MissingColon`, `UnmatchedEnd` | lossless |
| K19 | A date-time with a `TZID` and a `Z` suffix | UTC | UTC | UTC wins; the `TZID` stays a parameter. | none | all agree |
| K20 | Nesting beyond 32 levels | unlimited | unlimited | Deeper `BEGIN`/`END` lines are kept as ordinary properties. | `DepthLimitExceeded` | limit |
| CV1 | Unknown escape in a typed vCard 3.0/4.0 value (`\:`, `\"`, `\q`) | vobject drops the backslash of `\"` only; FolkerKinzel drops that of `\:` only | | The typed view (not the raw value) drops the backslash and keeps the character, for every character after `\\ \; \, \n \N`. vCard 2.1 unescapes `\\` and `\;` only. | none | prov. (oracles disagree) |
| CV2 | `TEL;TYPE=VOICE,MSG,WORK` imported to a contact | n/a | n/a | Every matching rule fires: Business and Other (MS-OXVCARD 2.1.3.4.1 example, literally). A fax alone fills Primary Fax, `pref` fills Primary, no type fills Other. | none | spec example; no client observed |
| CV3 | `ADR` with no recognised type | n/a | n/a | Business and Other; `postal`, `dom`, `intl`, `parcel` fill Other; a fourth address is dropped. | `AdditionalValueNotMapped` | spec example; prov. |
| CV4 | A second or later `TEL`/`EMAIL` of a kind with one slot | n/a | n/a | The second work (home) number goes to Business2 (Home2); later ones are reported, not stored. The first three emails fill the slots in card order. | `AdditionalValueNotMapped`, and on export `PhoneSlotChanged`, `EmailSlotChanged` | spec; prov. |
| CV5 | `BDAY` as a partial date or with a time | n/a | n/a | `PidTagBirthday` is the date at 11:59 UTC, `PidLidBirthdayLocal` the date at 0:00 (Outlook's own pair); the time of day is dropped; a date without a year is not stored. | `TimeOfDayDropped`, `PartialDate` | spec 2.1.3.2.5; prov. |
| CV6 | An empty text property in a contact (`PidTagDepartmentName` = "") | n/a | n/a | Treated as absent: no vCard property is written and the report says so. | `EmptyValueNotWritten` | agent default |
| CV7 | CR CR LF in a vCard | vobject, ez-vcard join the fold; Ical.Net and biweekly do for iCalendar; ical.js keeps a CR in the value; FolkerKinzel cuts the value | n/a | One line end (K1): a folded value that contains it is joined. | `AmbiguousDoubleCarriageReturn` | 6 of 7 (probe below) |

**CR CR LF probe (K1, CV7).** `tools/calendar/crcrlf/run.sh` writes one iCalendar and one vCard four ways (all CRLF; every line end CR CR LF; only the end before a fold; only some other line ends) and reads them with every pinned oracle. Before the change we cut the folded value; the readers split as follows (each reads one or both of the formats):

| Reading of CR CR LF + space | Oracles | Count |
|---|---|---|
| One line end: the fold joins and no CR reaches the value | biweekly 0.6.8, Ical.Net 5.2.3, vobject 0.9.9, ez-vcard 0.12.2 | 4 |
| The fold joins but a CR stays in the value (`Hello\rWorld`; a trailing CR on plain lines) | ical.js 2.2.1, icalendar 7.3.0 (which also refuses the file when plain lines end that way) | 2 |
| The value is cut at the line end; the rest becomes another line | FolkerKinzel.VCards 8.2.0 (vCard) | 1 |

Majority: the fold joins (6 of 7), and with 4 of those the CR is not data. Resolution: a run of CRs then LF is one line end. The bytes are untouched.

### JSON and XML forms (jCal, jCard, xCal, xCard)

Observed black-box on ical.js 2.2.1, Python `icalendar` 7.3.0, biweekly 0.6.8 and ez-vcard 0.12.2 (`tools/calendar/oracles/forms`); the rows say how we resolve what the RFCs leave open or what the oracles do differently. Reasons and the full class list: [`jcal-xcal.md`](https://recursiveai.org/onemail/docs/jcal-xcal/) and `explained.tsv`.

| ID | Case | Oracles | Resolution | Defect or loss | Basis |
|---|---|---|---|---|---|
| J1 | Value that does not fit its type (`DTSTAMP:19970901T1300Z`, `TRIGGER:19980403T120000`) | ical.js, biweekly print a guessed, often garbled typed value | Carried as `unknown` with the raw text; the declared `VALUE` stays as a parameter, so the text writes back as read | `ValueNotRepresentable` | lossless |
| J2 | `DTSTART:19971102` with no `VALUE=DATE` | ical.js, biweekly: date-time with a garbled time | A date (RFC 7265 Appendix B.1 shows it) | `AmbiguousValueType` | RFC example |
| J3 | `ENCODING=BASE64` without `VALUE=BINARY` on `ATTACH` or an extension property; `VALUE=BINARY` without `ENCODING` | n/a | Both are `binary`; `ENCODING=BASE64` is added in the second case | `AmbiguousValueType` | RFC 5545 §3.3.1 |
| J4 | `TZ:-0500` | ical.js text | `utc-offset` (RFC 7095 Appendix B.1); `VALUE` is not written back | none | RFC example |
| J5 | Parameter with no value (`ATTENDEE;MAILTO:`) | ical.js, icalendar drop it | One empty value, in JSON and XML | none | lossless |
| J6 | Parameter given twice | ical.js, icalendar keep the last | Values merged in order | `DuplicateParameterMerged` | lossless |
| J7 | Same JSON object key twice | n/a | Both kept | `DuplicateKey` | lossless |
| J8 | `TYPE`, `PID`, `SORT-AS` values, also a quoted `"a,b"` | ez-vcard keeps one string | Split into arrays (RFC 7095 §3.3.1.1) | none | RFC |
| J9 | Inline vCard base64 (`PHOTO`, `KEY`) | ical.js keeps `binary` and `ENCODING` | `data:` URI; media type from `TYPE`, else `application/octet-stream` | `InlineBinaryAsDataUri` | RFC 7095 §3.5, RFC 2397 |
| J10 | Backslash escapes in a vCard URI value (`URL:http\://x`) | ical.js and ez-vcard unescape | Unescaped (`\\ \, \; \:`); a backslash cannot occur in a URI | none | 2 of 2 |
| J11 | vCard 2.1/3.0 data in xCard | ez-vcard writes 4.0 everywhere | 4.0 forms for `GEO`, `TYPE=pref`; `VERSION` dropped. jCard keeps the source version and type names | `VersionNotCarried`, `VersionSpecificForm` | RFC 6351 |
| J12 | Group prefix on iCalendar data | ical.js keeps `GROUP.NAME` as the name | Split into the `group` parameter | `GroupNotCarried` in XML | RFC 7095 §3.3.1.2 |
| J13 | Property, parameter, component or type name that is not an XML name | biweekly, ez-vcard drop or rename | Left out of xCal/xCard (JSON carries it) | `PropertyNotCarried`, `ParameterNotCarried`, `ComponentNotCarried`, `TypeNameNotCarried` | XML 1.0 |
| J14 | Character XML cannot hold (U+000C, NUL, lone surrogate) | n/a | U+FFFD | `InvalidXmlCharacter` | XML 1.0 |
| J15 | vCard 2.1 quoted-printable `=0D=0A` | ez-vcard keeps CR LF in `LABEL` | Line feed in the form; the text writer uses quoted-printable again and never folds a line after `=` | none | RFC 6350 §3.4 |
| J16 | Components outside a `VCALENDAR` (bare `VEVENT` examples) | biweekly writes calendars only | Carried at the top level of the form | none | lossless |
| J17 | Durations, UTC offsets, `TZID` values, repeated `FREEBUSY` periods, URI scheme case, local times with `TZID` | icalendar, biweekly normalise (`P7W` to `P49D`, `-00:00` to `+00:00`, local to UTC, `MAILTO:` to `mailto:`) | Kept as written | none | lossless |

## Recurrence and time zones

Ours against `recurring-ical-events` 3.8.2 on `icalendar` 7.3.0 (**rec**), `python-dateutil` 2.9.0.post0 `rrule` through our glue (**du**) and Ical.Net 5.2.3 (**ical**), black-box, over every recurring component of the public calendar corpus plus our generated fixtures: 2,384 components, each in a fixed window (`tools/calendar/run_expand_diff.sh`; method in `docs/recurrence.md`). A component is *identical* when the sorted start lists are equal, *differing* otherwise, *not expanded* when the oracle errors or times out (330 libical components have an invalid DTSTART that every oracle rejects). Counts are components. Each difference gets exactly one id, the first cause that applies; the id is the entry below. Basis as in the other sections: all agree, 2 of 3, or provisional (our default).

| Oracle | Compared | Identical | Differing | Explained | Unexplained | Not expanded |
|---|---|---|---|---|---|---|
| rec | 2,372 | 2,233 | 139 | 139 | 0 | 12 (2 broken property, 8 timeout, 2 value error) |
| du | 2,378 | 2,238 | 140 | 140 | 0 | 6 (1 broken property, 3 type error, 2 value error) |
| ical | 2,188 | 1,409 | 779 | 779 | 0 | 196 (130 timeout, 33 argument, 30 out of range, 3 other) |

| ID | Case | Oracles | Resolution | Defect | Differences (rec / du / ical) | Basis |
|---|---|---|---|---|---|---|
| R1 | `BYWEEKNO` with no `BYDAY`, `BYMONTHDAY` or `BYYEARDAY` | rec and du: every day of the week; Ical.Net: the weekday of DTSTART | The weekday of DTSTART (RFC 5545 §3.3.10: a finer part that is absent takes DTSTART's value). | none | 32 / 32 / 0 | prov. (RFC; Ical.Net agrees) |
| R2 | A part the §3.3.10 table marks N/A for the frequency (`BYWEEKNO` with MONTHLY, `BYYEARDAY` with DAILY/WEEKLY/MONTHLY, `BYMONTHDAY` with WEEKLY, `BYSETPOS` alone, ordinal `BYDAY` where §3.3.10 forbids it) | the three disagree with each other and with us on some; 3 of the Python differences are one occurrence at the COUNT boundary of `WEEKLY;BYMONTHDAY;BYSETPOS;COUNT` | The part filters, never expands. | none | 102 / 102 / 449 | prov. (the table gives no result) |
| R3 | DTSTART does not fit its own rule (§3.8.5.3: undefined) | rec and du include DTSTART; Ical.Net omits it | DTSTART is always the first instance and counts toward COUNT. | `AmbiguousUnsynchronizedStart` | 0 / 0 / 75 (rec and du give our list on all 94 such components, apart from those with an R1 or R2 part) | 2 of 3 |
| R4 | `UNTIL` of another value type than DTSTART (DATE vs date-time, UTC vs floating) | the oracles differ with each other | A UTC UNTIL is an instant for zoned and UTC series and a wall-clock value for floating ones; a DATE UNTIL covers its whole day. | `AmbiguousUntilType` | 2 / 1 / 1 | prov. (§3.3.10 requires equal types) |
| R5 | Wall times in a clock change: the skipped hour and the repeated hour | rec and du: skipped shifted, repeated first; Ical.Net differs | Skipped: shifted by the offset before the gap (RFC 5545 §3.3.5), or dropped with `SkippedTimePolicy.Skip`. Repeated: the first. | `AmbiguousSkippedLocalTime`, `AmbiguousRepeatedLocalTime` | 0 / 0 / 57 (rec and du give our list on all 85 components that hit a clock change; ical on 26 of 83) | 2 of 3 |
| R6 | `EXDATE` as a DATE on a date-time series | not in the corpus | The whole day is removed. | `AmbiguousExdateType` | 0 / 0 / 0 (unit test only) | prov. (clients treat a date as a day) |
| R7 | `WEEKLY` with `BYSETPOS` in the first, partial week | rec and du position inside the part of the week after DTSTART | Positions count in the whole week (§3.3.10), so an earlier instant of the first week takes a position and is then dropped as before DTSTART. | none | 1 / 1 / 0 | prov. (RFC) |
| Z1 | A `TZID` named both by a VTIMEZONE in the file and by the runtime's database | not compared (the oracles resolve zones in their own way; the zone fixtures use one or the other) | The VTIMEZONE wins. | none | n/a | prov. |
| Z2 | A vendor-prefixed `TZID` (`/softwarestudio.org/Olson_20011030_5/America/New_York`) | not compared | The longest tail of up to three segments that names a zone. | `AmbiguousZoneName` | n/a | prov. |
| Z3 | A Windows zone name with no VTIMEZONE | not compared; instants checked for 5 names (`W. Europe`, `Eastern`, `AUS Eastern`, `New Zealand`, `Pacific`) | The CLDR default-territory IANA zone. | `AmbiguousWindowsZone` | n/a | prov. (CLDR) |
| Z4 | A `TZID` that resolves nowhere | not compared | The values stay floating; no instant is claimed. | `UnknownTimeZone` | n/a | prov. |
| Z5 | An IANA `TZID` on a host whose runtime knows only Windows zone ids (Windows without ICU, or with `InvariantGlobalization`) | not compared | The runtime lookup is repeated under the Windows zone CLDR assigns the IANA name (default-territory row first, then any territory). The zone keeps its IANA `Id` and `ZoneSource.Iana`. Windows zone data has fewer historic rules than IANA (US dates before 2007 follow the current rules), so such instants can differ from the same file read on Linux or macOS | none | n/a | our default (CLDR) |

Limits of one oracle (not ambiguities; no entry in the resolution column):

| ID | Oracle | Behaviour | Differences |
|---|---|---|---|
| L-oracle-ignores-exrule | rec, du | EXRULE is not applied | 1 / 1 / n/a |
| L-du-glue-has-no-overrides | du | our glue expands the master only, so RECURRENCE-ID components are not applied | 0 / 3 / n/a |
| L-ical-duplicate-instants | ical | an instant produced by several BYxxx combinations is returned several times; the set of instants is ours | n/a / n/a / 10 |
| L-ical-window-wall-clock | ical | the window start is compared with the wall time of a zoned instant, so in zones east of UTC an instant before the window gets through; without those instants the lists are equal | n/a / n/a / 102 |
| L-ical-weekno-interval | ical | `YEARLY` with `BYWEEKNO` and `INTERVAL` above 1 lands off the interval grid (2026, 2029, 2031 for `INTERVAL=2`) | n/a / n/a / 8 |
| L-outvoted-by-the-other-two | any | the oracle differs and at least one other oracle gives our list | 1 / 0 / 77 |

The `AmbiguousUntilType` and `AmbiguousExdateType` resolutions are the ones most likely to change when Outlook is observed (the Outlook oracle, before Outlook writing). Counts come from one run of the 2,384 components; `tools/calendar/run_expand_diff.sh` repeats it. Zone behaviour is checked in the DST subset instead: 108 `dst.ics` components in America/New_York, Europe/Berlin, Australia/Sydney and Pacific/Auckland (both transitions each) and 9 with Windows names give our list on 108/108 and 9/9 for rec and du.

## Mailbox files (mbox, Maildir, EMLX)

Black-box runs of Python 3.13.17 `mailbox` (mbox and Maildir) and MimeKit 4.18.2's mbox mode (`MimeParser` with `MimeFormat.Mbox`;
`tools/oracles/mailbox-driver`) over small probes (`tools/mailbox/gen_mboxes.py`, `gen_stores.py`) and the dovecot-crlf file
(3,240 `From ` lines). Go has no mbox reader. Neither oracle is a mail client, so every row that is not "agree" is provisional.
The reader's behaviour is in [`mailboxes.md`](https://recursiveai.org/onemail/docs/mailboxes/). MK = MimeKit, Py = Python.

| ID | Case | Py | MK | Resolution | Defect | Basis |
|---|---|---|---|---|---|---|
| S1 | A line starting `From ` that is not an envelope line (`From the desk of x`, `From maildir-storage.c, line 325:`), unquoted in the body | splits | splits, then often fails on the next "message" | Only a line with a sender and a date after `From ` (RFC 4155) starts a message; the rest is text. `MboxFromLineMode.Any` splits like both oracles. dovecot-crlf has three such lines: Py 3,240 messages, OneMail 3,237, MK raises at message 556 | `AmbiguousFromLine` | prov.; real-world file |
| S2 | A `Content-Length` header (mboxcl/mboxcl2, or in any file) | ignored | ignored | Used when the file's variant is unknown, mboxcl or mboxcl2 and the length ends exactly at an envelope line or the end of the file (a blank line may sit between). Otherwise ignored. mboxo and mboxrd ignore it. Two different values: the first | `ContentLengthMismatch`, `AmbiguousContentLength` | prov. |
| S3 | A CRLF file: the empty line after a message | keeps the `CR LF` in the message | the message runs to the next envelope line | The empty line (LF or CRLF) before the next envelope line or the end of the file is the separator, not message text | none | prov. |
| S4 | A body that the oracle's split leaves unparseable | n/a | raises `FormatException` | The tree is OneMail's job, not the splitter's: a message is never rejected | none | oracle limit |
| S5 | A line that is only `From ` (no sender or date) | splits | does not split | Not an envelope line in strict mode; a delimiter in `Any` mode | `AmbiguousFromLine` | 1 of 2 |
| S6 | mboxcl2 (unquoted bodies, `Content-Length` only) | splits inside messages that hold envelope-looking lines | same | Read correctly through `Content-Length`; OneMail writes it only when asked for the variant | none | oracle limit |
| S7 | Bytes before the first envelope line | skipped | skipped | Kept as the store's preamble, so entries plus preamble tile the file | `PreambleBeforeFirstMessage` | all agree |
| S8 | An envelope line with no empty line before it | splits | splits | Splits; the previous message keeps its last line | `MissingSeparatorLine` | all agree |
| S9 | Unquoting a stored body when the variant is not known | n/a | n/a | mboxrd's rule (every `>+From ` loses one `>`); an entry that a `Content-Length` delimited is returned as stored, because mboxcl and mboxcl2 differ | none | our default |
| S10 | Maildir info flags not in ASCII order or repeated (`:2,SF`) | read as written | n/a | Read as written, reported; written back sorted | `UnsortedFlags` | our default |
| S11 | Maildir file in `new` | subdir `new`, no flags | n/a | Flags from the name, plus `Recent` | none | all agree |
| S12 | Mail's `flags` word and `date-sent`/`date-received` reals in EMLX plists | n/a | n/a | Bits 0, 1, 2, 4, 6, 8 map to seen, deleted, answered, flagged, draft, forwarded; the raw word is always kept. A real is Unix seconds from 1e9, otherwise seconds since 2001-01-01. **No real Mail store has been inspected**, so this is a default to check against one | none | our default |
| S13 | Courier `courierimapkeywords/:list` | n/a | n/a | Keyword names, an empty line, then `unique:n n` rows with zero-based positions. Written from Courier's published description of the directory; the line format inside the files is not in that document, so this is unchecked against Courier | `MalformedControlFile` | our default |
| S14 | DBX: the length a message record states (field 0x11) differs from the chain's | n/a | n/a | undbx 0.21 writes the chain's bytes whatever the record states (3 records: stated 10 vs 3,024 bytes, 99,999 vs 63, 0 vs 723). The chain wins; the notes warn the field "can be wrong" | `AmbiguousDbxDeclaredLength` | undbx 0.21 on generated stores, and on a real store (NIST CFReDS Hacking Case): 33 of the 34 news posts that have text state a length 5 to 380 bytes different from the chain's, and undbx's file equals the chain in all 34 |
| S15 | DBX: the character set of record strings (subject, sender, folder names) | n/a | n/a | windows-1252. The notes name none; the strings are in the system ANSI code page of the machine that wrote them, which the file does not record | none | our default; not observable through undbx |
| S16 | DBX: a message file in the store folder that `Folders.dbx` does not name | n/a | n/a | Read, as a folder named after the file | `DbxUnlistedFile` | undbx 0.21 also extracts every file in the directory; our default for the folder name |
| S17 | DBX: a message record with no text chain (a header that was never downloaded) | n/a | n/a | An empty message in the folder, flagged. undbx 0.21 writes a 0-byte file for it (its `-i` option skips empty messages) | `AmbiguousDbxNoMessageText` | undbx 0.21 on a generated store; the real store above has 2,019 header-only records among 2,053 (news headers), and undbx writes each as a 0-byte file |
| S18 | DBX: field 0x09 of a folder record | n/a | n/a | Read as the special-folder kind (1 Inbox, 2 Outbox, 3 Sent, 4 Deleted, 5 Drafts). The notes describe the field twice (a per-folder index and the kind) | none | a real `Folders.dbx` (35,527 records) has the kinds on Inbox 1, Outbox 2, Sent Items 3, Deleted Items 4, Drafts 5 and absent on the rest |

| S19 | The character before the Maildir info section (`unique:2,FS` or `unique;2,FS`) | Py `mailbox` writes and reads `:` only | n/a | Read: the first `:`, else the first `;` that is followed by `2,`. Written: `MaildirStoreOptions.InfoSeparator`, default `;` on Windows (where a file name cannot hold `:`) and `:` elsewhere; isync's `InfoDelimiter` documents the same default. No Windows mail client has been observed | none | our default (isync documentation) |

## Reports, authentication results and MHTML

Probes are `testdata/reports/…` (`cases.tsv` describes each); MK is MimeKit 4.18.2's public API, Py is Python 3.13 `email`
(plus `urllib.parse.urljoin` for MH6–MH8). Go has no report or MHTML support and is not compared. All entries are provisional
(no client observed). Method: [reports-and-mhtml.md](https://recursiveai.org/onemail/docs/reports-and-mhtml/).

| ID | Probe | MK | Py | Resolution | Defect | Basis |
|---|---|---|---|---|---|---|
| RP1 | `dsn-11`, `dsn-12`, `dsn-32` extra blank lines between or around blocks | skipped | each is an empty block | Runs of blank lines are skipped; one defect per run. | `AmbiguousReportBlankLines` | prov. (MK, RFC) |
| RP2 | MDN/ARF body with a leading blank line, or several blocks | empty first block | empty first block | The first block is empty; further blocks are kept as extra blocks. | `AmbiguousReportExtraBlocks` | MK and Py agree |
| RP3 | `dsn-13` a line without a colon in a block | kept as a field named by the line | block ends there | Skipped, reported. | `ReportMalformedLine` | prov. (no two agree) |
| RP4 | `dsn-34` a name with a space | kept as a field | block ends there | Kept under its text before the colon. | `InvalidHeaderName` | prov. |
| RP5 | `dsn-18` whitespace before the colon | trimmed | field lost | Trimmed, reported. | `WhitespaceBeforeHeaderColon` | prov. (MK, RFC 5322 obs-) |
| RP6 | `dsn-14` a fold before any field | ignored | n/a | Skipped, reported. | `ReportMalformedLine` | prov. |
| RP7 | `dsn-26` a lone CR in a value | kept | line split at CR | Stays data. | `BareCarriageReturnInHeader` | MK, same as the header reader |
| RP8 | `dsn-21` an empty delivery-status body | no report | one empty block | A report with no groups. | `ReportMissingGroup` | prov. |
| RP9 | a whitespace-only line inside a block | fold | fold | A fold, not a block end. | none | all agree |
| RP10 | repeated single-use field (`dsn-15` Action twice) | both kept in order | n/a | Both stay in the block; the typed accessor reads the first. | `AmbiguousReportDuplicateField` | prov. (lossless) |
| AR1 | `auth-26`, `auth-08` first token is `name=…` (no authserv-id) | no id | n/a | No authserv-id; the token starts the first result. | `AuthenticationResultsMissingId` | MK |
| AR2 | `auth-08` a property with no ptype (`action=none`) | dropped | n/a | Kept with an empty ptype. | `AmbiguousAuthenticationResultsProperty` | prov. (lossless) |
| AR3 | `auth-11,12,14,15,16,18,19,27` broken fields (bad version, empty clause, garbage, `none` beside results, unterminated quote or comment) | the whole field fails | n/a | The clauses that can be read are kept, each repair is a defect. | `AuthenticationResultsInvalidVersion`, `InvalidAuthenticationResults`, `AmbiguousAuthenticationResultsNone`, `UnterminatedQuote` | lossless |
| AR4 | `auth-10` quoted authserv-id, property value or reason (valid per RFC 8601 §2.2) | the field fails | n/a | Parsed. | none | RFC |
| RS1 | `spf-06,07,10` unknown result, words that are not pairs, bad `client-ip` | no oracle | no oracle | Unknown result kept as `SpfResult.Unknown` with the text; bad pairs skipped. | `InvalidReceivedSpf` | prov. (RFC 7208 §9.1 only) |
| MH1 | `mht-01` `cid:` references, with brackets, `%40`, or case | no match | n/a (no MHTML) | Matches `Content-ID` only, brackets stripped, percent-decoded, exact then ASCII case-insensitive. | none | RFC 2557 §8.3 |
| MH2 | `mht-02,06,08,09,10,11,15,19` base from the part's or an enclosing entity's `Content-Location` | base only from the multipart's `Content-Base` | n/a | Base order of RFC 2557 §5 as written in the method note. | none | RFC |
| MH3 | `mht-03` dot segments, no base | no match | n/a | RFC 3986 §5.2.4 on `thismessage:/`. | none | RFC |
| MH4 | `mht-07` percent escapes, space, case of `%hex` | exact only | n/a | Exact first, then normalised. | none | prov. |
| MH5 | `mht-17` `mid:` | no match | n/a | Needs a matching Message-ID and Content-ID. | none | RFC 2392 |
| MH6 | `urljoin` with base `thismessage:/…` | n/a | returns the reference unchanged (scheme not hierarchical for it) | Resolved per RFC 3986. | none | RFC |
| MH7 | `urljoin` of `http:g` against `http:` base | n/a | non-strict: `http://a/b/c/g` | Strict: `http:g` (RFC 3986 §5.2.2). | none | RFC |
| MH8 | `urljoin` with an upper-case scheme or host in the reference | n/a | lower-cases the scheme | Left as written; `Normalize` lower-cases when matching. | none | lossless |
| MH9 | two parts with the same location or content id (`mht-06`) | first | n/a | First wins. | `AmbiguousMhtmlDuplicateLocation`, `AmbiguousMhtmlDuplicateContentId` | prov. |
| MH10 | `start` names no part (`mht-05`) | n/a | n/a | The first part is the root. | `MhtmlStartNotFound` | RFC 2387 |

Documented oracle limits (not ambiguities; in `explained.tsv` as `LIMIT-*`): Python does not read
`message/global-delivery-status` or transfer-encoded report parts as reports.

## The shared read view (bodies and attachments)

Checked against Python 3.13 `email` with `policy.default` (`get_body`/`iter_attachments`) over the probes, the Apache James test
messages, the dovecot mbox and a sample of the CMU Enron maildir; `tools/oracles/viewcheck.sh` reproduces the run. The view's
choices below are provisional: no mail client has been observed for them yet. The view raises no defects of its own; it
chooses between readings of a valid tree. The tree always keeps every part (`docs/shared-view.md`).

| ID | Case | Python `email` | Resolution |
|---|---|---|---|
| V1 | Which parts are attachments | Immediate children of the top multipart, except the first `text/plain`, `text/html`, `multipart/related` and `multipart/alternative`; a nested multipart is one attachment; a message that is not multipart has none | A leaf part or `message/*` part that is not the chosen text or HTML body, found by walking every multipart, in document order. A message that is a single leaf part (not text-body) is its own attachment. |
| V2 | Body inside `multipart/signed` or a nested `multipart/mixed` | `get_body` does not look inside `multipart/signed`; the nested part's text is listed as an attachment | The body is searched in the signed content (the first child) and in nested mixed parts, in order; the signature part is an attachment. |
| V3 | Body in `multipart/alternative` with several text parts | The last alternative that matches | The last alternative that matches (RFC 2046 5.1.4). The other text renderings are not attachments unless marked `Content-Disposition: attachment`. |
| V4 | Inline parts | Parts of a `multipart/related` are not listed | Listed, with `IsInline` set when the part is not marked `attachment` and has a `Content-ID`, so callers can resolve `cid:` references. |
| V5 | A text part with `Content-Disposition: attachment` | An attachment | An attachment; never the body. A text part with a file name but no disposition is still a body when it is the first match (the same as Python). |
| V6 | Text body without a usable charset | `us-ascii`, bytes above 127 replaced | UTF-8 when the bytes are valid UTF-8, otherwise Windows-1252 (N5's rule). |
| V7 | Address groups in `From`/`To`/… | Group name kept | Flattened to the members, the group name dropped (A1). The tree keeps the group. |
| V8 | Attachment file name | `get_filename()`: disposition `filename`, else type `name`, with the parameter rules of N1–N8 | The same order (N9). Names are kept as written, not made safe for a file system (N10); no name is `null`, never an empty string. |
| V9 | A message with a TNEF part (winmail.dat) | Python: winmail.dat is one attachment. MimeKit (black box, `MimeKitTnef`): `MimeMessage.Attachments` lists winmail.dat; `TnefPart.ExtractAttachments` returns the body (an unnamed RTF, HTML or text entry) and the attachments | A part that qualifies under MS-OXCMAIL 2.2.3.8 (the conversion's rule, `conversion.md`) is not listed; the attachments of its TNEF take its place in document order, `RtfBody` and the bodies the MIME side lacks come from it, and the part stays reachable (`MimeMailItem.TnefParts`, the tree). A part that does not qualify stays an attachment. Outlook shows a TNEF message this way (2.2.3.8); the attachments equal MimeKit's extraction, tnefparse's, tnef's and ytnef's on 64 wrapped messages (252 checks, 0 unexplained). Prov.: no client observed. |

## MAPI and MSG

Observed on the Apache Tika and POI MSG files (55) and our generated ones, against extract-msg 0.55.0 as a black-box oracle.
No Outlook has been observed yet, so each resolution is provisional.

| ID | Case | Other reader | Resolution | Defect | Basis |
|---|---|---|---|---|---|
| X1 | Same property tag twice in one property stream | n/a | Both kept in the bag; lookups return the last | `AmbiguousDuplicateProperty` | prov. |
| X2 | Value stream with no property entry (POI fuzz-minimised file) | ignores it | Read as a property; the stream is the data | `AmbiguousOrphanValueStream` | prov., lossless |
| X3 | String type differs from the store support mask | follows the entry's type | The entry's own type decides | `AmbiguousStringType` | MS-OXMSG 2.4.2 |
| X4 | 8-bit strings, no `PidTagMessageCodepage` | fixed Latin default | Locale's ANSI page, then Internet code page, then enclosing message, then 1252 | `AmbiguousCodePage` | 36 corpus strings differ, all in files with only a locale or Internet code page |
| X5 | String size field: stream length or stream plus terminator | n/a | Both accepted | none | ANSI files store the NUL in the stream |

## PST/OST messaging and LTP

Observed on the Tika testPST files, the CFReDS OST and our generated files against `pffexport` 20180714 as a black-box oracle. No Outlook
has been observed yet, so each resolution is provisional. L-entries are the lists-tables-properties choices of `docs/pst-ltp.md`.

| ID | Case | Other reader | Resolution | Defect | Basis |
|---|---|---|---|---|---|
| P1 | Message has attachment subnodes but no attachment table | no attachments | The attachment subnodes found are used | `AmbiguousAttachmentTableMissing` | prov., lossless |
| P2 | Embedded-message object property holds a bare subnode NID, not the (NID, size) structure | n/a | The NID is used | `AmbiguousObjectReference` | MS-PST 2.3.3.5 states the structure; prov. |
| P3 | Table row identifier differs from the NID the object is stored under | trusts the node | The node decides; the mismatch is reported | `RowIdMismatch` | prov. |
| L1 | Rows per block of a subnode row matrix | n/a | Taken from the first block's own size, which equals the formula for 8 KiB blocks | none | prov.; real files agree |
| L2 | BTH key that does not increase | n/a | Skipped, so a node reached twice gives no duplicates | none | prov. |
| L3 | Property or cell of an unknown type | n/a | Read as an HNID in a PC; kept inline when one, two or eight bytes | none | prov. |
| L4 | Positions in a heap's fill-level bitmap blocks (HNBITMAPHDR) | n/a | Allocations may not start before the block's header (12, 66 or 136... bytes, 2 elsewhere) | none | exercised on our files only |

## RTF bodies (compressed RTF, de-encapsulation)

Probes are the 45 RTF bodies of the public Tika and POI `.msg` files, the specification examples (`testdata/rtf`), and 3,000
damaged streams. Oracles: compressed-rtf 1.0.7 (decompress, compress) and RTFDE 0.1.2.2 (de-encapsulation), run as released
packages; the message's own `PidTagHtml`/`PidTagBody` is a third reference. Where the specification is explicit the entry says so.
Entries without an observed Outlook-side result are provisional. Method: [rtf.md](https://recursiveai.org/onemail/docs/rtf/).

| ID | Case | Oracle / own body | Resolution | Defect | Basis |
|---|---|---|---|---|---|
| RT1 | `\par` and `\line` in the extracted text | RTFDE writes LF; own plain-text bodies use CRLF (4 of 4) | CRLF (MS-OXRTFEX 2.2.3.2 and 2.2.3.3) | none | spec; seen in 31 HTML and 4 text results vs RTFDE |
| RT2 | A literal no-break space (`\'a0`) in visible text | RTFDE writes `&nbsp;` | The character U+00A0. A string has no code page to lose it in | none | prov.; 4 files vs RTFDE |
| RT3 | The message's own `PidTagHtml` carries `<meta http-equiv=Content-Type …>` that the RTF's HTML lacks | own body | Not part of the encapsulated HTML; nothing to resolve. Callers who want the stored body read `PidTagHtml` | none | 5 of 6 own HTML bodies |
| RT4 | Own `PidTagHtml` is a re-serialised copy of the RTF's HTML (tag case, dropped `</tbody>`) | own body | The encapsulated HTML is returned as written; same text (checked by words) | none | 1 of 6 (`bug66335`); RTFDE agrees with OneMail on it |
| RT5 | `\_` | spec table: soft hyphen in `\htmltag`; RTF 1.9.1: non-breaking hyphen | U+00AD inside `\htmltag`, U+2011 elsewhere | none | spec |
| RT6 | `\htmlrtf` active when an `\htmltag` group starts; code page of `\htmltag` text | spec: content MUST be copied; default code page MUST be used | Copied whatever the state; decoded with the document's default code page, not the font's | none | spec |
| RT7 | `\uN` and `\lquote`, `\~`… inside `\htmltag` | spec table gives HTML text (`&#x20AC;`, `&lsquo;`); 2.2.3.2 says "Unicode characters" | The characters. A .NET string needs no entities, and both denote the same text | none | prov. |
| RT8 | How many characters `\ucN` skips | RTF 1.9.1 says "characters", with `\'hh` as one | Text bytes and `\'hh` escapes count; control words do not; `{` or `}` cancels the rest; `\uc` is group-scoped | none | prov. |
| RT9 | Code page of text bytes | spec: "the code page of the current font" | Font `\cpgN`, else `\fcharsetN` (0 = 1252 even under another `\ansicpg`; 1 = document), else `\ansicpg`/`\ansi`/`\mac`/`\pc`/`\pca`, else 1252; before any `\f` the `\deff` font | none | spec example uses `\fcharset0` under `\ansicpg1251` for exactly this; the corpus Cyrillic body decodes to its own body |
| RT10 | `\htmlrtf` in a `\fromtext` document | spec defines it for HTML only | Ignored | `AmbiguousHtmlRtfInPlainText` | prov. |
| RT11 | A code page with no decoder (symbol, platform gap) | spec silent | The document's default code page | `AmbiguousUnknownCodePage` | prov. |
| RT12 | Detection: `\fromhtml`/`\fromhtml0`, text before the control word, an eleventh token, CR/LF before the first token | spec 2.2.3.1 (SHOULD) | Only `\fromhtml1` or `\fromtext` within ten group-start/control-word tokens after `{\rtf`; line breaks are skipped as in RTF | `NotEncapsulated`, `NotRtf` | spec |
| RT13 | Destinations | spec: skip standard destinations without visible text, keep `\fonttbl`; ignorable `\*` groups other than `\htmltag` skipped | A fixed list of body-less destinations is skipped; an unknown destination that is not ignorable is read as text (it cannot be told from a formatting group) | none | prov. |
| RT14 | RAWSIZE far above the data (a decompression bomb) | compressed-rtf decodes the data and ignores the claim | Refused before allocating: above 256 MiB, or above 20 × the input length | `OutputLimitExceeded`, `ExpansionRatioExceeded` | 123 of 3,000 damaged streams differ for this reason only |
| RT15 | COMPSIZE disagrees with the bytes present | compressed-rtf ignores it | Contents are bytes 16 to COMPSIZE + 4 when that is within the input, else to the end; the CRC covers exactly those | `CompressedSizeMismatch` | spec 2.1.3.1.1; prov. for the extra bytes |
| RT16 | Data ends before the end marker | compressed-rtf returns what it decoded | The decoded bytes, with a defect | `MissingEndMarker` | prov. |
| RT17 | RAWSIZE disagrees with the output; `MELA` contents | spec: MELA CRC not validated, size may be read either way | Output is what the data holds (growing up to the limit), with a defect; MELA is the contents as stored | `RawSizeMismatch` | spec 2.2.3.1 |
| RT18 | Compressing nothing | spec note: Outlook writes `02 00 0D 00` | `01 0C F0` (end marker only) is written; `02 00 0D 00` is read as one NUL byte | none | spec |
| RT19 | `\fcharset0`/`\fcharset1` fonts under another `\ansicpg` (native RTF to HTML) | pandoc and unrtf decode every `\'hh` as 1252; LibreOffice 24.2.7 uses 1252 for `\fcharset0`, LibreOffice 26.2.6 follows `\ansicpg` | Follows the document's `\ansicpg` (0 and 1 both mean "the document's ANSI code page"). The de-encapsulator keeps RT9 (0 = 1252) because it reads what Outlook wrote, where the spec example shows `\fcharset0` under `\ansicpg1251`; a document authored by Word names a Central European font with its own `\fcharset238`, so `\fcharset0` there is Latin text typed under that page | none | no client observed; oracles split, the newer one agrees; provisional |
| RT20 | Header, footer, footnote and endnote destinations | LibreOffice puts them in the body; pandoc/unrtf vary | Omitted from the mail body, text and HTML | none | provisional: a message body has no page furniture |
| RT21 | Hidden text (`\v`) | unrtf writes it | Omitted | none | spec: hidden text is not displayed |
| RT22 | A table inside a table cell | oracles flatten or nest differently | Flattened into the outer cell's text | `NestedTableFlattened` | provisional |
| RT23 | Shapes (`\shp`) | pandoc shows `\shprslt` only | The `\shpinst` text box content is read; `\shprslt` is skipped when a `\shpinst` is present | none | RTF 1.9.1 (the result is the fallback for readers that skip `\shpinst`) |
| RT24 | Picture whose bytes disagree with the declared `\pngblip`/`\jpegblip` | none | The bytes decide (magic numbers); EMF/WMF are not rendered and become an attachment-style placeholder | `AmbiguousPictureFormat`, `InvalidPictureData` | provisional |
| RT25 | A `\cell` outside any row | LibreOffice joins the words on each side | A tab between them | `AmbiguousBareCell` | provisional |
| RT26 | `\object` with a result | LibreOffice drops the result picture | The result is shown; an object with only its placeholder becomes an attachment placeholder | none | provisional |
| RT27 | Empty paragraphs; content after the root group | none | An empty paragraph is `<p><br></p>`, trailing empty paragraphs are dropped; anything after the root group's closing brace is ignored | none | provisional |
| RT28 | List level beyond the list table, or an `\ls` with no list table | pandoc drops the list | Level clamped to the deepest defined; an undefined `\ls` renders as a bullet list | `AmbiguousListLevel`, `UndefinedReference` | provisional |
| RT29 | Headings | pandoc and unrtf make none from `\s` styles | A paragraph style whose stylesheet name is "heading N" becomes `<hN>`; the stylesheet is read for names only and its formatting is not inherited | none | provisional |
| RT30 | `\objattph` | none | The attachment position is a placeholder, and the single space after it is consumed | none | spec |
| RT31 | Text in a symbol font (`\fcharset2`, Symbol, Wingdings) | none | Private-use code points are kept, except the bullet | `AmbiguousSymbolFont` | provisional |
| RT32 | `HYPERLINK \l "x"` | none | `href="#x"` | none | spec (field switches) |
| RT33 | `\caps`/`\scaps` | pandoc upcases the text | The text is unchanged and a CSS `text-transform` carries the effect | none | provisional |

## TNEF

Observed on the public Apache POI and Tika TNEF files (6 distinct) and our generated ones, against tnefparse 1.4.0, tnef 1.4.18 and ytnef 2.1.2
as black-box oracles. No Outlook has been observed, so each resolution is provisional unless it follows the specification's own text.

| ID | Case | Other reader | Resolution | Defect | Basis |
|---|---|---|---|---|---|
| T1 | A message-level legacy attribute appears twice | n/a | The last one is used; both stay in the attribute layer | `AmbiguousDuplicateAttribute` | prov., lossless |
| T2 | A legacy attribute and the encapsulated property differ (e.g. attDateRecd vs PidTagMessageDeliveryTime in winmail-sample1.dat) | n/a | The encapsulated property wins; the legacy value is not added | `AmbiguousLegacyConflict` | MS-OXTNEF 2.1.3; seen in 1 corpus file |
| T3 | PtypNull value in a property list | tnef and ytnef reject it (0 and 4 bytes alike) | Takes no bytes | none | MS-OXCDATA 2.11.1; prov. |
| T4 | Legacy message class names (`IPM.Microsoft Mail.Note`) | n/a | Mapped to the modern class | `AmbiguousMessageClass` | MS-OXTNEF 2.3.3.4; the spec sample and corpus files use it |
| T5 | Wrong checksum on the message class attribute | n/a | Ignored, the data is used | `AmbiguousMessageClassChecksum` | MS-OXTNEF 2.3.3.4 |
| T6 | An embedded-message or OLE attachment also has attAttachData | n/a | The attribute is ignored | `AmbiguousAttachDataIgnored` | MS-OXTNEF 2.3.3.7 |
| T7 | attOemCodepage differs from PidTagMessageCodepage | n/a | attOemCodepage decides how 8-bit strings are read | `AmbiguousCodePage` | MS-OXTNEF 2.3.3.2 order; no corpus file has the conflict, so no client observation |
| T8 | attOwner with no meeting class | n/a | Maps to the PidTagSentRepresenting group | `AmbiguousOwner` | prov. |

## MS-OXCMAIL conversion (MAPI to MIME and back)

The rules are in [`conversion.md`](https://recursiveai.org/onemail/docs/conversion/). No Outlook has been observed yet; the two oracles are black-box MimeKit 4.18.2 and
Python 3.13 reading what we write, and the corpus is the Tika and POI MSG files and the Tika and CFReDS PST/OST files. Each
resolution is provisional until Outlook can be run.

| ID | Case | What the spec says | Resolution | Defect or report entry | Basis |
|---|---|---|---|---|---|
| OX1 | An `EX` sender or recipient has no SMTP address property | 2.1.3.1.7: look it up in the address book directory | Without a directory: the same distinguished name elsewhere in the message, then the matching transport `From`/`Sender` field (one mailbox), then a unique display name in the transport headers. Never derived from the distinguished name | `AddressUnresolved` when nothing resolves | on the 45 MSG and 25 PST/OST items, 16 of 28 EX senders and 50 of 50 EX recipients resolve; the 12 senders that do not (all MSG) have no SMTP property and no transport From field; 9 PST senders have no address at all |
| OX2 | Transport headers and properties disagree | 2.2.3.2.28 keeps the received block; 2.1.3.2 builds fields from properties | The received field stays when it agrees; the properties win when it does not (a user edit makes the properties newer) | `TransportHeaderOverridden`, `NamedPropertyValueSetAside` | 2 fields and 5 named-property values across the 70 corpus items |
| OX3 | Default values (`PidTagSensitivity` 0, false receipt flags) | 2.1.3.2.6 and 2.1.3.1.5: no field | No field; reading back leaves the property absent | `DefaultValueNotWritten` | 2.1.3.2.6, 2.1.3.1.5; 151 entries over the 70 exported items |
| OX4 | `Importance: Normal` | 2.1.3.2.5: MAY be omitted | Written whenever the property exists (0, 1, 2) | none | 2.1.3.2.5 (MAY omit); writing it keeps the property on the way back, 0 silent differences in the 70-item property round trip |
| OX5 | RTF that encapsulates nothing | 2.1.3.3: not covered | A `text/rtf` part when there is no HTML, else not written | `RtfCarriedAsTextRtf`, `RtfRedundant` | 2.1.3.3 is silent; 5 and 13 entries over the 70 exported items; no client observed |
| OX6 | Case of a `PS_INTERNET_HEADERS` name | 2.2.3.2.29: "the header name" | As the message wrote it. Stores differ (Exchange stores lower case, Outlook as written), so comparisons are case-insensitive | none | 2.2.3.2.29; the MSG and PST corpora hold both spellings, so the property round trip compares case-insensitively |
| OX7 | Missing or invalid `Date` on import | 2.2.3.2.2: the current time | No `PidTagClientSubmitTime`; a conversion must be reproducible. Export falls back to the delivery time | `DateFromDeliveryTime` | deviates from 2.2.3.2.2 on purpose (reproducible output); 4 MSG items and 1 Mime4j message hit it |
| OX8 | Non-ASCII display name | 2.1.3.1: RFC 2047 | The whole name as one run of encoded words (never leaving "@" or "(" in plain text beside them) | none | MimeKit and Python read it; a plain-text remainder broke an address in a first attempt |
| OX9 | Which MIME part is the body | 2.2.3.3.2 | The part the shared view chooses (`shared-view.md`), a named text part included | none | 2.2.3.3.2; body hashes equal the view's on 70 of 70 exported items in both oracles |
| OX10 | Composite media type on flat attachment data | not covered | `application/octet-stream` | `CompositeTypeOnFlatData` | spec silent; a `multipart/signed` without a boundary broke the parse in MimeKit and Python; 1 MSG item and 6 Mime4j messages hit it |
| OX11 | Bcc | 2.1.3.1.1.2 | Not written unless asked; a `Bcc` field on import is Bcc recipients | `BccNotWritten` | 2.1.3.1.1.2; 2 MSG items, 2 Mime4j and 1,272 Enron-sample messages hit it |
| OX12 | `multipart/signed` on import | 2.2.3.4 | Class `IPM.Note.SMIME.MultipartSigned`; content and signature become a body and an attachment | `SignatureNotVerifiable` | 2.2.3.4; 213 of 1,430 Dovecot messages; the Outlook form, while signed MIME stays bytes elsewhere |
| OX13 | Subject prefix | 2.2.3.2.6.1: one to three characters other than colon, blank or digit, then colon and blanks | As stated, with an empty `PidTagSubjectPrefix` when there is none | none | 2.2.3.2.6.1; 23 PST items carry a prefix marker the export removes (`SubjectPrefixMarkerRemoved`), none differ in the oracles |
| OX14 | Embedded message names | not covered | The attachment's own name only; the subject is not invented as a name | none | spec silent; attachment names of all 70 exported items equal the source in both oracles |
| OX15 | Reply-To entries | 2.1.3.1.2: names ignored when the counts differ | Only one-off SMTP entries are written; address-book entries need a directory | `ReplyToUnresolved` | 2.1.3.1.2; 9 entries over the 70 exported items |
| OX16 | A TNEF property that a header field also sets (subject, sender, reply-to) | 2.2.3.8 step 2.1: ignore attributes that set a property already set by the headers | Per property, and a group set by any header (subject, sender, sent-representing sender, reply-to) is the headers' whole, so two sources are never mixed in one sender or subject. Ignored values that differ are reported | `TnefPropertyIgnored` | 2.2.3.8; 152 of 1,596 properties over 52 wrapped corpus messages (none in the bare shape, which has no headers); prov. for groups |
| OX17 | Plain text in MIME, richer body in the TNEF | 2.2: the plain text MUST NOT be preferred to a richer TNEF body | A body property is added when the MIME side has none of that kind; the MIME text stays `PidTagBody`; `PidTagNativeBody` and `PidTagInternetCodepage` are the TNEF's (the author's) unless the HTML is the MIME side's, then the richest body | `TnefPropertyIgnored` for a differing body of the same kind | 2.2; prov. for the codepage rule |
| OX18 | Message class | the class is a property; header processing gives `IPM.Note` | The default `IPM.Note` gives way to the TNEF's `IPM.*` class; a class the MIME structure gave (report, S/MIME) stays | `TnefMessageClassUsed` | spec silent on defaults; needed for meeting requests carried in TNEF; prov. |
| OX19 | Recipients in the headers and in the TNEF | 2.2.3.8 step 2.1 | The headers' recipients win; the TNEF table is used only when the headers gave none | `TnefRecipientIgnored` | 2.2.3.8; prov. |
| OX20 | A TNEF stream ending in CR LF | 2.2.3.8 step 3: SHOULD ignore the stream (a SHOULD, no client observed) | Unpacked, as tnef 1.4.18, ytnef 2.1.2, tnefparse 1.4.0 and MimeKit 4.18.2 all do (4 of 4 oracles on both `bug52400` files); reported, not hidden. Prov. until a client is observed | `AmbiguousTnefTrailingCrLf` (view defect), `TnefTrailingCrLfIgnored` (`Internal` loss entry) | 2.2.3.8 step 3; the two `bug52400` corpus files; the earlier choice (keep as attachment) was reversed on the oracle evidence |
| OX21 | Correlation key vs `X-MS-TNEF-Correlator` | 2.2.3.8 step 4: a mismatch means pure MIME; the field is spelled `X-MS-TNEF-Correlator` (2.1.3.2.26) and `X-MS-Tnef-Correlation` (2.2.3.8) | Both spellings are read; the binary key matches when its bytes (trailing NULs and blanks removed) equal the field as ASCII text or as hexadecimal digits; either one missing is a match | `TnefNotUsed` | 2.1.3.2.26, 2.2.3.8; none of the corpus files has a key; prov. |
| OX22 | Writing a TNEF message | 2.1.3.7; the key is "unique" | The key of the message's own `PidTagTnefCorrelationKey` when it has one, else the first 16 bytes of SHA-256 of Message-ID (or subject) and submit time in hex; no top-level recipient table in the TNEF; an empty `text/plain` when there is no text body | `TnefWriterLimit`, `TnefPlainTextNotRendered` | 2.1.3.7, 2.1.3.1.1; 45 MSG and 25 PST/OST corpus items through `OxcMailTool rt-tnef`: 10,496 of 11,142 properties exact, 227 the same text under another string type, the rest in the loss report, 0 silent (pure MIME: 6,453 exact, 4,290 reported) |

## Outlook calendar items and MS-OXCICAL

No Outlook has been observed yet: every row is an agent default settled by the spec text and, for the expansion, by the oracles named in [`calendar-items.md`](https://recursiveai.org/onemail/docs/calendar-items/).

| ID | Case | What the spec says | Resolution | Defect or report entry | Basis |
|---|---|---|---|---|---|
| OC1 | A monthly pattern on day 29 or 30 | MS-OXOCAL 2.2.1.44.1: the instance falls on the last day of a month that is too short | The RRULE `BYMONTHDAY=29`/`30` is written (RFC 5545 skips short months); day 31 is written `BYMONTHDAY=-1` | `AmbiguousShortMonth` (Approximated) | MS-OXCICAL 2.1.3.2.2; test `MonthlyDay30IsAnAmbiguityNotASilentChange`; prov. |
| OC2 | The series start is not a day the pattern allows (a Tuesday start on a Monday-only weekly pattern) | 2.2.1.44.1.2 starts from the first valid day | The first valid day is the first instance and `DTSTART` | `AmbiguousFirstInstance` | test `AFirstInstanceOffThePatternStartsOnTheFirstAllowedDay`; prov. |
| OC3 | A floating time or a DATE value has no zone in an Outlook item | MS-OXOCAL: times are UTC plus a zone struct | UTC is assumed for the series zone and reported | `FloatingTimeAssumedUtc` | prov. |
| OC4 | `WKST` when an RRULE has none | MS-OXCICAL: a missing WKST means Sunday; RFC 5545 3.3.10: Monday | `WKST` is always written from the pattern's first day of week when exporting; a missing WKST is read as RFC 5545 says (Monday) by the shared engine | none | dateutil and recurring-ical-events agree on all 54 generated series (the two-week patterns with Sunday and Monday differ without it); test `RRuleExpansionEqualsTheSpecDayWalk` weekly-2w cases; prov. |
| OC5 | The UID of an appointment | 2.1.3.1.1.20.26: a global object ID whose data (from byte 40) starts with `vCal-Uid` and 0x01000000 carries the UID text from byte 52; otherwise the UID is the hex of the ID with the instance date zeroed | Exactly that; an item with no usable ID gets a UID made from a hash of subject and times | `UidSynthesized` (Approximated) | spec text; tests; prov. |
| OC6 | The zone of a VEVENT with a `VTIMEZONE` | 2.1.3.1.1.20.2: the zone becomes a TZDEFINITION | The rules are recovered by sampling the zone hourly in the DTSTART year: two changes are a DST rule, none a fixed offset, anything else the fixed offset at DTSTART | `UnknownTimeZone` when the TZID is neither defined nor known | prov. |


## OLM (Outlook for Mac)

Observed on the two public sample archives (Outlook 16 layout) and our generated ones, against the released olmreader 1.9.0 jar and the
olm-reader 1.0.0 npm package as black-box oracles. No Outlook for Mac has been observed, so each resolution is provisional. Details: [`olm.md`](https://recursiveai.org/onemail/docs/olm/).

| ID | Case | Other reader | Resolution | Defect | Basis |
|---|---|---|---|---|---|
| O1 | Dates have no zone (`2018-06-20T23:49:52`) | the jar returns the text and leaves the reading to the caller; the npm package reads it in the machine's zone (UTC when `TZ=UTC`, and then writes it as GMT) | UTC, `ZoneKnown` false | `AmbiguousDateZone` | prov.; the npm package agrees under `TZ=UTC` on 478 + 7 messages |
| O2 | `OPFMessageCopyBody` and `OPFMessageCopyHTMLBody` hold the same text | the npm package writes text and strips tags | `OPFMessageGetHasHTML` decides: true gives an HTML body, false a plain one; with no flag, a body that differs from the HTML element or looks like HTML is HTML | `AmbiguousBodyKind` (flag absent only) | prov.; 470 of 483 and 7 of 7 sample messages have the two elements equal |
| O3 | Two attachments of one message share a name | the jar keys bytes by name and keeps one | Both are listed, each with its own bytes | none | oracle limit (1 message in the Enron sample) |
| O4 | The declared attachment size (`OPFAttachmentContentFileSize`, written as `5.0614E4`) differs from the entry's size | not used by either oracle | The entry's size is the size; the declared one stays in the XML | none | every attachment in the Outlook-example sample differs |
| O5 | An address element has a name and no address | the npm package writes `unknown@unknown.com` for a missing sender | Name kept, address empty | `AddressMissing` | 12 recipients in the Enron sample |
| O6 | A date of year 4501 or later | n/a | No date (Outlook's "none" value) | none | MS-OXOCAL; prov. |
| O7 | Contact e-mail slot type 0, 1, 2 | n/a | work, home, other | none | prov.; the sample's addresses are named after their slot |
| O8 | Recurrence with both a count and an end date | n/a | The count (`COUNT`) when the record says it is numbered, else the end date as a UTC `UNTIL` | `RecurrenceEnd` (loss entry) | prov.; all four sample patterns are numbered |
| O9 | An attachment URL that matches no entry | the jar returns no bytes | Listed with no content | `AttachmentMissing` | prov. |


<!-- https://recursiveai.org/onemail/docs/changelog/ -->

# Changelog

All notable changes are listed here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project follows [Semantic Versioning](https://semver.org/).

## [Unreleased]

## [0.3.0-preview.2]

A fix release for 0.2.0-preview.2, on top of 0.3.0-preview.1.

### Added

- `RecursiveAI.OneMail.Display.CalendarCard`: the compact card mail clients show for a calendar object, as an HTML fragment and as plain text. It covers iMIP/iTIP methods (invitation, update, cancellation, reply, counter-proposal, ...), the time in a caller-chosen zone with the organizer's time kept, a plain-English recurrence sentence ("Every 2 weeks on Monday and Wednesday, until 30 Nov 2026", or "Repeats (custom rule)" when it cannot be said exactly) with the next occurrences, location and conference URLs, organizer, attendees and a truncated description, from `.ics` files, `text/calendar` parts and Outlook appointment and meeting items (`FromIcs`, `FromMime`, `FromMapi`, `FromItem`). The HTML is built from escaped text and passes the display sanitiser unchanged. See `docs/calendar-card.md`.
- `OutlookNames.NetShowUrl` (`PidLidNetShowUrl`).
- `IMailAttachment.ContentTypeInferred` (a default interface member): true when `ContentType` was guessed from the file name's extension because the item stored no specific type.

### Fixed

- `RecursiveAI.OneMail` now depends on `RecursiveAI.OneArchive` 1.1.0 (it was 0.7.0-preview.1), so an app that also uses OneArchive 1.x no longer hits a version conflict. The compound-file and zip readers are no longer preview APIs, so the build opt-outs are gone.
- Attachments with no stored content type, or the generic `application/octet-stream` (Outlook, TNEF, OLM, and MIME parts), report a type from a built-in table when the file name's extension is known (`.pdf` is `application/pdf`); a stored specific type always wins. See `docs/shared-view.md`.
- The package page (`PACKAGE.md`) links to the files of the release tag instead of `main`; the link is stamped from the package version when packing.
- The README lists only the packages that are on nuget.org.

## [0.3.0-preview.1]

On the local feed. Calendar and contacts close with this preview (speed rows in docs/benchmarks.md); it also carries the bodies, display and conversion work done so far. What is checked, and the known gaps, are in [docs/compatibility.md](https://recursiveai.org/onemail/docs/compatibility/).

### Added

- `RecursiveAI.OneMail.Composition`: `CreateReply`, `CreateReplyAll` and `CreateForward` (inline or as a `message/rfc822` attachment) on any item. They make the threading headers, recipients, subject prefix (never stacked), quoted text or HTML, inline images, and for Outlook items the MS-OXOMSG conversation index level, as MIME and, for Outlook originals, as a MAPI message; hostile originals are bounded and reported as typed defects. `ConversationIndex` and `SubjectPrefixes` are public. See `docs/composition.md`.
- `StoreConverter` and `Mail.ConvertAsync`/`Mail.Convert`: any readable store or directory to EML, mbox, Maildir++, Apple Mail bundles, ICS, vCard, the j/x forms or TNEF, keeping folders, flags and keywords where the target has them, with progress, cancellation, an aggregate loss report and an `IStoreSink` extension point; the fidelity matrix is in `docs/fidelity.md`.
- `onemail convert` and `onemail export`, with `--json`.
- `MaildirStore.DeliverPath`: delivers and returns the path (`Deliver` re-listed the directory per message).
- Outlook calendar and task items (MS-OXOCAL, MS-OXOTASK): appointments, recurrence blobs with exceptions, time zones, meeting requests, responses and cancellations, and tasks in the shared view, with conversion to and from iCalendar per MS-OXCICAL and loss reports.
- Outlook for Mac OLM reading (`RecursiveAI.OneMail.Stores.Olm`): archives over the zip reader, every item kind with a lossless element tree, calendar and contact mapping with loss reports, and `Mail.Open` on `.olm`.
- Display HTML (`RecursiveAI.OneMail.Display`): an allowlist sanitiser on the HTML DOM, a remote-content policy (block, allow or a rewrite callback), `cid:` and attachment resolution, and optional header and attachment blocks. It has not had its independent review yet.
- Calendar and contact benchmark rows against Ical.Net and FolkerKinzel.VCards, with a libical reference, in `docs/benchmarks.md`.
- `RtfHtmlConverter`: RTF written as RTF (no encapsulated HTML or text) becomes HTML and plain text with typed defects and limits; the Outlook item view uses it when no other body exists (`MailBody.Origin`).

### Changed

- Content lines: a run of carriage returns before a line feed is one line end (most readers join a fold written that way), reported as `AmbiguousDoubleCarriageReturn`; the bytes are kept. Parsing allocates less.
- `MimeWriter` keeps the white space inside a header value when it writes one (runs of spaces and tabs were collapsed); untouched messages still write from their raw bytes.

## [0.2.0-preview.2]

The first preview on nuget.org. It has everything in 0.2.0-preview.1 plus the changes below. What is checked, and the known gaps, are in [docs/compatibility.md](https://recursiveai.org/onemail/docs/compatibility/).

### Added

- jCal, jCard, xCal and xCard (`RecursiveAI.OneMail.Calendar.Forms`): read and write both directions from ICS and vCard trees with a loss report, typed defects and resource limits; `MailFormatSniffer` names the four forms.
- Outlook Express 5/6 DBX reading (`RecursiveAI.OneMail.Stores.Dbx`): message databases, the folder tree from `Folders.dbx`, lazy message streams in constant memory, content sniffing, and `Mail.Open` on a `.dbx` file or a store folder.
- `MaildirStoreOptions.InfoSeparator`: Maildir file names use `;` before the info section on Windows (where `:` is not allowed in file names) and `:` elsewhere; both are read on every platform.

### Fixed

- IANA time-zone names now resolve on Windows when the runtime knows only Windows zone ids (for example with invariant globalization), through the CLDR Windows-zone map; Windows names resolve through the same map on every platform.
- Maildir delivery no longer fails on Windows.

## [0.2.0-preview.1]

Outlook formats on read: MSG and OFT, PST and OST, TNEF and compressed RTF, the MAPI property model, and conversion between MAPI and MIME. Calendar recurrence and time zones. What is checked, and the known gaps, are in [docs/compatibility.md](https://recursiveai.org/onemail/docs/compatibility/).

### Fixed

- Encoding to ISO-8859-1 and US-ASCII code pages (the runtime's own, not in the code page provider) no longer throws in `MailEncoding.GetBytes`.

### Added

- The MAPI property model (`RecursiveAI.OneMail.Mapi`): typed property bags with named properties, code-page handling and the MS-OXPROPS tag tables.
- MSG and OFT reading (MS-OXMSG) over the compound file reader of OneArchive, with embedded messages and attachments.
- The PST/OST lists, tables and properties layer (`RecursiveAI.OneMail.Pst.Ltp`: heap-on-node, BTH, property and table contexts) over the node database, reading through the random-access sources.
- TNEF reading and writing (MS-OXTNEF): the attribute layer, MAPI messages and embedded messages.
- Compressed RTF (MS-OXRTFCP) both ways and HTML and text de-encapsulation from RTF (MS-OXRTFEX), in a namespace with no dependencies on the rest of the library.
- Conversion between MAPI messages and MIME per MS-OXCMAIL in both directions, with a loss report listing every value that did not carry over (`RecursiveAI.OneMail.Conversion`).
- Calendar recurrence expansion (RRULE, RDATE, EXDATE, overrides) and time-zone resolution through VTIMEZONE, the IANA database and Windows zone names.
- Benchmark rows for PST and OST enumeration and extraction, MSG read and TNEF read ([docs/benchmarks.md](https://recursiveai.org/onemail/docs/benchmarks/)).
- Contacts: `VCardView` covers the rest of RFC 6350 (`Gender`, `Geos`, `Impps`, `Related`, `Keys`, `Sounds`, repeated properties, lazy `VCardMedia`); `MapiContact` and `MapiDistributionList` read Outlook contacts and lists (also as `MapiMailItem.Contact` / `.DistributionList`); `ContactToVCardConverter` and `VCardToContactConverter` convert both ways by MS-OXVCARD with loss reports ([docs/contacts.md](https://recursiveai.org/onemail/docs/contacts/)).

- The PST/OST messaging layer (`RecursiveAI.OneMail.Pst.Messaging`): store, named properties, folders, messages, recipients, attachments and embedded messages as MAPI property bags, in ANSI, Unicode and 4 KiB files.
- MAPI items (MSG, PST, OST) in the shared view; `Mail.Open` reads them and `onemail inspect` lists them.
- TNEF parts inside MIME messages: `MimeToMapiConverter` merges them into the MAPI message by the rules of MS-OXCMAIL 2.2.3.8 (headers win, richer bodies kept, loss-report entries for every property set aside), the shared view shows their attachments and bodies in place of winmail.dat (`MimeMailItem.TnefParts`), and `OxcMailExportOptions.Tnef` writes a TNEF message.
- The browser smoke app covers MSG, PST, TNEF, compressed RTF and de-encapsulation with legacy code pages, recurrence through a VTIMEZONE and the IANA database, and the MS-OXCMAIL conversions.

## [0.1.0-preview.1]

First preview: the MIME core, mailbox stores and the one-line reading path. What is checked, and the known gaps, are in [docs/compatibility.md](https://recursiveai.org/onemail/docs/compatibility/).

### Added

- Random-access sources (memory, stream, file, ranged reads) under every file format.
- `MimeReader` and `MimeWriter` (forward-only events with byte offsets, streaming writes) and the lossless entity tree: untouched messages write back byte for byte, every repair is a typed defect, and resource limits are on by default.
- Charsets: the WHATWG encodings plus UTF-7, IMAP modified UTF-7 and the ISO-2022-JP variants; RFC 2047 encoded words and RFC 2231 parameters.
- Structured headers: addresses and groups, dates, message identifiers, content type and disposition parameters.
- multipart/report views (delivery status, disposition notification, feedback reports) with builders; MHTML resolution; `Authentication-Results` and `Received-SPF` parsing.
- Mailbox stores: mbox (mboxo, mboxrd, mboxcl, mboxcl2, Gmail Takeout labels), Maildir with Courier and Dovecot keywords, EMLX and Apple Mail bundles, read and written; content-based store sniffing.
- The shared read view (`IMailItem`), `Mail.Open` and `Mail.Parse`, and content-based format sniffing.
- `RecursiveAI.OneMail.Calendar`: ICS and vCard content lines and lossless component trees with typed views.
- The PST/OST node database layer (header, pages, B-trees, data and subnode trees, allocation maps, scan mode).
- The `onemail inspect` command.
- The ambiguity table (`docs/ambiguities.md`), the differential and round-trip tools, fuzz targets and the MIME benchmark rows.
- Package skeleton: the `RecursiveAI.OneMail`, `.Calendar`, `.Client`, `.Cryptography` and `.Render` packages and the `onemail` tool as buildable placeholders, with tests, a benchmark shell, a fuzz shell and NativeAOT and browser WebAssembly smoke apps.


<!-- https://recursiveai.org/onemail/cookbook/read-msg-file/ -->

# Read an Outlook MSG file in C#

Open the file with `Mail.Open` and take its one item: you get the subject, sender, recipients, date, bodies and attachments as plain .NET values. No Outlook, COM or native library is involved. The program below reads a public test message from Apache POI.

```shell
dotnet new console -o ReadMsg && cd ReadMsg
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
curl -LO https://raw.githubusercontent.com/apache/poi/REL_5_4_1/test-data/hsmf/attachment_test_msg.msg
```

OneMail is a preview, so `dotnet add package` needs `--version` or `--prerelease`. Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneMail;

// Mail.Open reads the format from the bytes, not the extension:
// .msg, .oft, .eml, .pst and mbox files all open the same way
using MailContainer file = Mail.Open("attachment_test_msg.msg");
IMailItem msg = file.Enumerate().First();

Console.WriteLine($"Format:  {file.Format}");
Console.WriteLine($"Subject: {msg.Subject}");
Console.WriteLine($"From:    {msg.From.FirstOrDefault()}");
Console.WriteLine($"To:      {string.Join(", ", msg.ToRecipients)}");
Console.WriteLine($"Sent:    {msg.Sent?.Value:u}");

Console.WriteLine($"Text body: {msg.TextBody?.ReadText().Trim()}");
Console.WriteLine($"HTML body: {msg.HtmlBody?.ReadText().Length} characters");
Console.WriteLine($"RTF body:  {(msg.RtfBody is null ? "no" : "yes")}");

foreach (IMailAttachment a in msg.Attachments)
    Console.WriteLine($"Attachment: {a.FileName} ({a.Size:N0} bytes)");
```

```text
Format:  Msg
Subject: test pièce jointe 1
From:    Nicolas1 23456 <nicolas1.23456@free.fr>
To:      'nicolas1.23456@free.fr' <nicolas1.23456@free.fr>
Sent:    2009-04-22 14:36:33Z
Text body: contenu
HTML body: 1033 characters
RTF body:  yes
Attachment: test-unicode.doc (24,064 bytes)
Attachment: pj1.txt (89 bytes)
```

`Mail.Open` reads the first bytes to decide the format, so the same call opens `.oft` templates, `.eml` files, PST stores and mbox files. The items share one interface, `IMailItem`.

## The bodies

An Outlook message can carry plain text, HTML and RTF. `TextBody`, `HtmlBody` and `RtfBody` give each one that is present. When Outlook stored the HTML wrapped inside the RTF body, as in this file, `HtmlBody` returns that HTML. When the RTF is the only body, `HtmlBody` is HTML converted from it and its `Origin` is `RtfConverted`: see [Convert an RTF email body to HTML](https://recursiveai.org/onemail/cookbook/convert-rtf-to-html/). Call `ReadText()` for a string, or `Open()` for a stream.

## Attachments

`Attachments` lists files and embedded messages in order. `Size` is the decoded size and `Open()` returns the content. An attached message has `Embedded` set to an `IMailItem` you can read the same way. To save the files, see [Extract attachments from email](https://recursiveai.org/onemail/cookbook/extract-attachments/).

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **Reading only.** OneMail can’t write MSG files yet. To get a file other programs read, [convert it to EML](https://recursiveai.org/onemail/cookbook/convert-msg-to-eml/).
- **Damaged files.** The project’s own fidelity table lists this gap: “two fuzz-minimised files from the corpus are refused by the compound file reader (typed defect)”.
- **Exchange senders.** A sender with only an Exchange (X.500) address and no SMTP form has an empty `Address`; `Name` and `X500Address` are still set.

Everything checked, and every known gap: [Fidelity and known gaps](https://recursiveai.org/onemail/docs/compatibility/). How the format is read: [MSG and OFT](https://recursiveai.org/onemail/docs/msg/).


<!-- https://recursiveai.org/onemail/cookbook/read-pst-file/ -->

# Read a PST file in C#

`Mail.Open` opens a PST (or OST) store and `Enumerate()` walks every message in every folder. Messages are read one at a time as you reach them, so a large store doesn’t have to fit in memory. The program below lists Apache Tika’s public test store.

```shell
dotnet new console -o ReadPst && cd ReadPst
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
curl -LO https://raw.githubusercontent.com/apache/tika/4.2.0/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/test/resources/test-documents/testPST.pst
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneMail;

// Opening reads only the store header; each message is read as you reach it
using MailContainer pst = Mail.Open("testPST.pst");

int count = 0;
foreach (IMailItem msg in pst.Enumerate())
{
    count++;
    MailAddress from = msg.From.FirstOrDefault();
    Console.WriteLine($"{msg.Sent?.Value:yyyy-MM-dd}  {from.Name}: {msg.Subject}");
    Console.WriteLine($"            folder: {msg.Provenance.Folder}, " +
                      $"attachments: {msg.Attachments.Count}");
}
Console.WriteLine($"{count} messages");
```

```text
2014-02-26  Jörn Kottmann: Re: Feature Generators
            folder: Début du fichier de données Outlook, attachments: 0
2014-02-25  Barry Olddog: Re: init tokenizer fails: "Bad type in putfield/putstatic"
            folder: Début du fichier de données Outlook, attachments: 0
2014-02-25  Kenig Ma: Feature Generators
            folder: Début du fichier de données Outlook, attachments: 0
2014-02-26  Nick Burch (JIRA): [jira] [Resolved] (TIKA-1249) Vcard files detection
            folder: Début du fichier de données Outlook, attachments: 0
2014-02-26  Gary Murphy (JIRA): [jira] [Commented] (TIKA-1250) Process loops infintely processing a CHM file
            folder: Début du fichier de données Outlook, attachments: 0
2014-02-24  Couchbase: [WEBINAR] - "Introducing Couchbase Server 2.5"
            folder: Début du fichier de données Outlook, attachments: 0
2020-11-26  Luis Filipe da Cruz Nassif: FW: First email
            folder: Début du fichier de données Outlook, attachments: 1
7 messages
```

Each item is an `IMailItem`, the same view as an MSG or EML file: `TextBody`, `HtmlBody`, `Attachments` and the rest work as in [Read an Outlook MSG file](https://recursiveai.org/onemail/cookbook/read-msg-file/). `Provenance.Folder` is the folder path inside the store. The last message’s sender has only an Exchange address, so its name prints and its SMTP address is empty.

## List the folders

For the folder tree itself, open the store with `PstStore`:

```csharp
using RecursiveAI.OneMail.Pst.Messaging;

if (!PstStore.TryOpen("testPST.pst", out PstStore? store, out var defects))
{
    Console.WriteLine($"Not a PST: {string.Join(", ", defects)}");
    return;
}
using (store)
{
    foreach (PstFolder folder in store!.EnumerateFolders())
    {
        if (folder.Depth == 0) continue;   // the unnamed root
        string indent = new(' ', 2 * (folder.Depth - 1));
        int messages = folder.EnumerateMessageIds().Count();
        string search = folder.IsSearchFolder ? " (search folder)" : "";
        Console.WriteLine($"{indent}{folder.Name}: {messages}{search}");
    }
}
```

```text
Début du fichier de données Outlook: 7
  Éléments supprimés: 0
Racine (pour la recherche): 0
SPAM Search Folder 2: 0 (search folder)
IPM_COMMON_VIEWS: 0
ItemProcSearch: 0 (search folder)
```

`Mail.Open` returns the ordinary messages of ordinary folders. Search folders only link to messages stored elsewhere, so their contents aren’t listed again. Hidden items (rules, views) come from `folder.EnumerateMessages(PstMessageSet.Associated)`.

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **Reading only.** OneMail can’t write PST files yet. To export messages, [convert a PST to EML files](https://recursiveai.org/onemail/cookbook/convert-pst-to-eml/).
- **Calendar and task items** come through the read view with `Kind` set to `Appointment` or `Task`, a subject and a body. For their times, attendees and recurrence, [convert them to ICS](https://recursiveai.org/onemail/cookbook/convert-msg-to-ics/).
- **Layouts checked on generated files only.** In the project’s words: “no public ANSI or 512-byte-page Unicode PST from Outlook exists; those layouts and 4 KiB multi-map stores are checked only on files we generate.”
- **Exchange addresses.** There is no directory lookup: a sender with no SMTP address in the store keeps only its name and X.500 address.

OST files open the same way; the project checked them on the NIST CFReDS OST. Everything checked, and every known gap: [Fidelity and known gaps](https://recursiveai.org/onemail/docs/compatibility/). The store layers: [PST and OST messages](https://recursiveai.org/onemail/docs/pst-messaging/).


<!-- https://recursiveai.org/onemail/cookbook/read-olm-file/ -->

# Read an OLM file in C#

Open the archive with `Mail.Open` and enumerate it. An `.olm` file is what Outlook for Mac exports, and it holds mail, calendar items, contacts, tasks and notes. Each record comes back as an `IMailItem` with its folder, so mail reads exactly like a PST. The program below reads the example archive of the OLMReader project (BSD-2-Clause).

```shell
dotnet new console -o ReadOlm && cd ReadOlm
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
curl -LO https://raw.githubusercontent.com/teverett/OLMReader/c3f14c3e49d181f180750a2125984fb67a82fd58/src/test/resources/Outlook16/outlook_example.olm
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneMail;

// Mail.Open reads the format from the bytes: an .olm is a zip of XML records
using MailContainer olm = Mail.Open("outlook_example.olm");
Console.WriteLine($"Format: {olm.Format}");

// Records are read one at a time, in the order they sit in the archive
List<IMailItem> items = olm.Enumerate().ToList();
foreach (var kind in items.GroupBy(i => i.Kind))
    Console.WriteLine($"{kind.Key}: {kind.Count()}");

foreach (IMailItem msg in items.Where(i => i.Kind == MailItemKind.Message))
{
    Console.WriteLine();
    Console.WriteLine($"{msg.Provenance.Folder}: {msg.Subject}");
    foreach (IMailAttachment a in msg.Attachments)
        Console.WriteLine($"  {a.FileName}  {a.Size:N0} bytes  {a.ContentType}");
}
```

```text
Format: Olm
Other: 10
Contact: 5
DistributionList: 2
Appointment: 8
Task: 5
Note: 1
Message: 7

Outlook PST/Inbox: [ubuntu-ko] 우분투 시작 안내서의 10. Tips & Trick 의 번역이 일단락되었습니다.

Outlook PST/Drafts: パーティーは何時ですか？
  017091.docx  37,440 bytes  application/vnd.openxmlformats-officedocument.wordprocessingml.document
  037025.docx  95,167 bytes  application/vnd.openxmlformats-officedocument.wordprocessingml.document
  000038.xls  71,680 bytes  application/vnd.ms-excel

Outlook PST/Drafts: What time is party?
  000046.xls  74,240 bytes  application/vnd.ms-excel
  018375.pptx  111,596 bytes  application/vnd.openxmlformats-officedocument.presentationml.presentation
  000006.doc  67,584 bytes  application/msword

Outlook PST/Drafts: באיזה שעה המסיבה?
  000100.doc  58,368 bytes  application/msword
  005227.pdf  147,455 bytes  application/pdf
  008708.pdf  148,328 bytes  application/pdf

Outlook PST/Archive: Email Test

Drafts: Draft with low priority

Drafts: Mail with followup
```

`Kind` says what each record is (`Other` here is the archive’s list of categories), and `Provenance.Folder` is its folder in the archive. Two of the attachment types weren’t stored and come from the file extension, so `ContentTypeInferred` is true for them. Messages have `From`, `ToRecipients`, `Sent`, `TextBody`, `HtmlBody` and `Attachments`, like every other format. An attachment’s `Open()` streams it from the archive.

## Calendar items and contacts

Appointments, tasks and notes have a subject and a body in the read view. For their times, attendees and recurrence, convert them: `Mail.Convert(item, ConvertFormat.Ics)` gives the `.ics` bytes and `ConvertFormat.VCard` the `.vcf` bytes of a contact, each with a loss report. To turn the whole archive into mbox files plus `.ics` and `.vcf` files, see [Convert a PST file to mbox](https://recursiveai.org/onemail/cookbook/convert-pst-to-mbox/): the same call takes an `.olm`.

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **Reading only.** OneMail never writes OLM files.
- **Checked on few archives.** In the project’s words: “no Outlook 2011 layout sample; no archive with a raw MIME source, attendees or exceptions seen; a damaged zip is refused (recovery comes later)”.
- **Dates.** OLM stores times in UTC with no time zone, so `Sent` is UTC.
- **Memory.** The zip directory stays in memory, about 1 KB per entry in the archive.

Everything about the format: [OLM (Outlook for Mac)](https://recursiveai.org/onemail/docs/olm/). Everything checked, and every known gap: [Fidelity and known gaps](https://recursiveai.org/onemail/docs/compatibility/).


<!-- https://recursiveai.org/onemail/cookbook/open-winmail-dat/ -->

# Open winmail.dat in C#

A `winmail.dat` attachment is TNEF, Outlook’s wrapper for attachments and rich text. `Mail.Open` reads it as one message whose `Attachments` are the files inside. The program below unpacks Apache POI’s public test file.

```shell
dotnet new console -o Winmail && cd Winmail
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
curl -LO https://raw.githubusercontent.com/apache/poi/REL_5_4_1/test-data/hmef/quick-winmail.dat
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneMail;

// winmail.dat is TNEF: Outlook's wrapper for attachments and rich text
using MailContainer tnef = Mail.Open("quick-winmail.dat");
Console.WriteLine($"Format: {tnef.Format}");

IMailItem msg = tnef.Enumerate().First();
Directory.CreateDirectory("winmail");
foreach (IMailAttachment a in msg.Attachments)
{
    string name = Path.GetFileName(a.FileName ?? "attachment.bin");
    string path = Path.Combine("winmail", name);
    using (Stream content = a.Open())
    using (FileStream file = File.Create(path))
        content.CopyTo(file);
    Console.WriteLine($"{path}  {a.Size:N0} bytes");
}
Console.WriteLine($"RTF body: {(msg.RtfBody is null ? "none" : "yes")}");
```

```text
Format: Tnef
winmail/quick.doc  19,968 bytes
winmail/quick.html  428 bytes
winmail/quick.pdf  18,638 bytes
winmail/quick.txt  235 bytes
winmail/quick.xml  143 bytes
RTF body: yes
```

All five files are byte-identical to the reference copies Apache POI keeps beside the test file. The rich-text body is in `RtfBody`.

## winmail.dat inside an email

You usually meet TNEF as an `application/ms-tnef` part of an `.eml`. You don’t need to find and unpack it: `Mail.Parse` and `Mail.Open` list the files inside it as the message’s attachments, in place of `winmail.dat`. Code for saving them: [Extract attachments from email](https://recursiveai.org/onemail/cookbook/extract-attachments/).

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **TNEF inside MIME** is checked on generated messages only. The project says: “no TNEF-in-MIME message in the public corpora; checked on generated wrappers (64 messages)”.
- **Subject and sender.** A bare `winmail.dat` often carries no subject or sender: those live in the email around it. This one has neither.

Everything checked, and every known gap: [Fidelity and known gaps](https://recursiveai.org/onemail/docs/compatibility/). The format: [TNEF (winmail.dat)](https://recursiveai.org/onemail/docs/tnef/).


<!-- https://recursiveai.org/onemail/cookbook/convert-rtf-to-html/ -->

# Convert an RTF email body to HTML in C#

Read the message’s `RtfBody` and pass it to `RtfHtmlConverter.Convert`: you get HTML and plain text, with no Word or Outlook. Many Outlook messages, notes and tasks carry only an RTF body. For those, the read view already does this, and `HtmlBody.Origin` says `RtfConverted`. The program below converts the RTF bodies of two Apache POI test messages: one written as RTF, and one in Cyrillic (code page 1251) that has only text and RTF.

```shell
dotnet new console -o RtfToHtml && cd RtfToHtml
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
curl -LO https://raw.githubusercontent.com/apache/poi/REL_5_4_1/test-data/hsmf/quick.msg
curl -LO https://raw.githubusercontent.com/apache/poi/REL_5_4_1/test-data/hsmf/cyrillic_message.msg
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneMail;
using RecursiveAI.OneMail.Rtf;

foreach (string path in new[] { "quick.msg", "cyrillic_message.msg" })
{
    using MailContainer file = Mail.Open(path);
    IMailItem msg = file.Enumerate().First();
    Console.WriteLine($"{path}: HTML body {msg.HtmlBody?.Origin.ToString() ?? "none"}");
    if (msg.RtfBody is null) continue;

    // RtfBody is the decompressed RTF; the converter never throws on bad input
    using var rtf = new MemoryStream();
    using (Stream s = msg.RtfBody.Open()) s.CopyTo(rtf);
    RtfHtmlResult result = RtfHtmlConverter.Convert(rtf.ToArray());

    string html = Path.ChangeExtension(path, ".html");
    File.WriteAllText(html, result.Html);
    Console.WriteLine($"  {html}: {result.Html.Length:N0} characters, code page {result.DefaultCodePage}, {result.Defects.Count} defects");
    Console.WriteLine($"  {result.Text.Split('\n')[0].Trim()}");
}
```

```text
quick.msg: HTML body RtfConverted
  quick.html: 173 characters, code page 1252, 0 defects
  The quick brown fox jumps over the lazy dog
cyrillic_message.msg: HTML body none
  cyrillic_message.html: 874 characters, code page 1251, 0 defects
  Ваше сообщение с темой "FW: RTM" было успешно получено брокером подсистемы обмена данными.
```

`cyrillic_message.html` holds the automatic reply, decoded from code page 1251:

```html
<div class="rtf-body" style="font-family:'Arial',sans-serif;font-size:10pt">
<p style="margin-top:0;margin-bottom:0">Ваше сообщение с темой &quot;FW: RTM&quot; было успешно получено брокером подсистемы обмена данными.</p>
<p style="margin-top:0;margin-bottom:0">&nbsp;&nbsp;В нем были обнаружены следующие вложения:</p>
<p style="margin-top:0;margin-bottom:0">10595000.58B</p>
<p style="margin-top:0;margin-bottom:0">10595000.56B</p>
<p style="margin-top:0;margin-bottom:0">10595000.59B</p>
<p style="margin-top:0;margin-bottom:0">10595000.57B</p>
<p style="margin-top:0;margin-bottom:0"><br></p>
<p style="margin-top:0;margin-bottom:0">Сообщение передано в обработку 18.05.2010 11:38:39</p>
<p style="margin-top:0;margin-bottom:0">------------</p>
<p style="margin-top:0;margin-bottom:0">&nbsp;Сообщение сгенерировано автоматически, не следует отвечать на него.</p>
</div>
```

## What converts

Paragraphs, headings from the stylesheet, bold, italic and the other run styles, fonts and colours, bulleted and numbered lists, tables, hyperlinks and pictures. Pictures become `data:` URIs by default; set `new RtfHtmlOptions { ImageMode = RtfImageMode.ContentId }` to get `cid:` links and the bytes in `result.Images`. `result.Text` is the same document as plain text. The output is one `<div class="rtf-body">` fragment, and the same input always gives the same HTML.

The project compared the converter with LibreOffice, pandoc and unrtf on 123 documents, by words, styles, lists, tables, links and headings, with no unexplained difference.

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **Not sanitised.** The HTML comes from the sender’s RTF. Treat it like any HTML from an email before you show it in a browser. OneMail’s display sanitiser (`ToDisplayHtml`) has not had its independent security review yet: don’t rely on it to make untrusted mail safe.
- **Placeholders.** EMF and WMF pictures and the positions of attachments inside the RTF (`\objattph`) become placeholders.
- **Limits on size.** Group depth, node count, output length and picture sizes are capped (`RtfHtmlOptions`); a hit is a typed defect, and the output so far is kept.

How the converter works, and how it was checked: [Compressed RTF and RTF to HTML](https://recursiveai.org/onemail/docs/rtf/). Reading the rest of the message: [Read an Outlook MSG file](https://recursiveai.org/onemail/cookbook/read-msg-file/).


<!-- https://recursiveai.org/onemail/cookbook/parse-eml/ -->

# Parse an EML file in C#

Pass the bytes to `Mail.Parse`. Headers come back decoded (encoded words, charsets) and the bodies as .NET strings, with base64 and quoted-printable undone. The program below parses a public test message from Apache Tika with two attachments.

```shell
dotnet new console -o ParseEml && cd ParseEml
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
curl -LO https://raw.githubusercontent.com/apache/tika/4.2.0/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-mail-module/src/test/resources/test-documents/testRFC822_multipart_attachments.eml
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneMail;

IMailItem msg = Mail.Parse(File.ReadAllBytes("testRFC822_multipart_attachments.eml"));

Console.WriteLine($"Subject: {msg.Subject}");
Console.WriteLine($"From:    {msg.From.FirstOrDefault()}");
Console.WriteLine($"To:      {string.Join(", ", msg.ToRecipients)}");
Console.WriteLine($"Date:    {(msg.Sent is { } d ? d.Value.ToString("u") : "none")}");

// Bodies come decoded (base64, quoted-printable, charset) as .NET strings
string? text = msg.TextBody?.ReadText();
string? html = msg.HtmlBody?.ReadText();
Console.WriteLine($"Text:    {text?.ReplaceLineEndings(" ").Trim()}");
Console.WriteLine($"HTML:    {html?.Length} characters");

foreach (IMailAttachment a in msg.Attachments)
    Console.WriteLine($"Attachment: {a.FileName}, {a.ContentType}, {a.Size} bytes");

// Malformed input is repaired, never thrown; each repair is listed
Console.WriteLine($"Defects: {msg.Defects.Count()}");
```

```text
Subject: EML 1
From:    Bugs Bunny <Bugs.Bunny@acme.com>
To:      Bugs Bunny <Bugs.Bunny@acme.com>
Date:    none
Text:    Hi,  This is test email for EML1  Thanks, Chetan
HTML:    151 characters
Attachment: Test TxtA.txt, text/plain, 33 bytes
Attachment: Test TxtB.txt, text/plain, 33 bytes
Defects: 0
```

This message has no `Date` header, so `Sent` is null. For a file that might be an `.eml`, an `.msg` or a mailbox, use `Mail.Open(path)` instead: it decides from the content.

## What you get

- **Addresses** as `MailAddress` values (`Name`, `Address`), with groups flattened: `From`, `ToRecipients`, `Cc`, `Bcc`, `ReplyTo`, `Sender`.
- **Ids and threading:** `MessageId`, `InReplyTo`, `References`, without angle brackets.
- **Bodies:** `TextBody` and `HtmlBody` are the candidates the message carries; neither is converted into the other.
- **Repairs, not exceptions.** Accessors never throw on malformed content. Every repair is a `MailDefect` with its byte offset.
- **Lossless.** Parsing doesn’t copy the bodies, and an untouched message writes back byte for byte. The lower layer, `MimeEntity.Parse`, gives the full part tree and every header.

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **Ambiguous mail.** Where mail clients disagree on how to read broken input, OneMail picks one reading and marks the defect as an ambiguity. The project calls these choices “provisional until mail clients are observed”.
- **No signatures or encryption.** S/MIME and OpenPGP parts are read as parts; checking or decrypting them isn’t in this preview.

Everything checked, and every known gap: [Fidelity and known gaps](https://recursiveai.org/onemail/docs/compatibility/). The reader: [Reading and writing MIME](https://recursiveai.org/onemail/docs/mime-reading/).


<!-- https://recursiveai.org/onemail/cookbook/read-mbox/ -->

# Read an mbox file in C#

`Mail.Open` recognizes an mbox file from its content and `Enumerate()` returns its messages one by one. Each is an `IMailItem`, as from an `.eml` file. The program below reads a public mbox from Apache Tika’s tests: three posts from a Hadoop mailing list.

```shell
dotnet new console -o ReadMbox && cd ReadMbox
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
curl -LO https://raw.githubusercontent.com/apache/tika/4.2.0/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-mail-module/src/test/resources/test-documents/complex.mbox
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneMail;

// mboxo, mboxrd, mboxcl and mboxcl2 are told apart by content
using MailContainer mbox = Mail.Open("complex.mbox");
Console.WriteLine($"Format: {mbox.Format}");

foreach (IMailItem msg in mbox.Enumerate())
{
    Console.WriteLine($"{msg.Sent?.Value:u}  {msg.From.FirstOrDefault().Address}");
    Console.WriteLine($"  {msg.Subject}");
    string text = msg.TextBody?.ReadText() ?? "";
    string first = text.Split('\n').Select(l => l.Trim()).First(l => l != "");
    if (first.Length > 40) first = first[..40] + "…";
    Console.WriteLine($"  {text.Length} characters: {first}");
}
```

```text
Format: Mbox
2009-06-01 04:26:31Z  jothipn@yahoo-inc.com
  Re: question about when shuffle/sort start working
  621 characters: When a Mapper completes, MapCompletionEv…
2009-06-01 05:30:49Z  jianmin_woo@yahoo.com
  Re: question about when shuffle/sort start working
  1259 characters: Thanks a lot for your explanation, Jothi…
2009-06-01 06:03:13Z  jothipn@yahoo-inc.com
  Re: question about when shuffle/sort start working
  1732 characters: No you cannot raise this event yourself,…
```

The four mbox variants differ in how a `From ` line inside a body is escaped and whether lengths are stored. OneMail tells them apart and undoes the escaping, so bodies come back as written. A Gmail Takeout export is an mbox too, and `MboxMetadata` reads its `X-Gmail-Labels`. Use `await foreach (IMailItem msg in mbox)` to read asynchronously.

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **MMDF isn’t supported**, as the project’s fidelity table states.

Maildir folders, Apple Mail `.emlx` files, Outlook Express `.dbx` files and Outlook for Mac `.olm` archives open with the same `Mail.Open` call. Everything checked, and every known gap: [Fidelity and known gaps](https://recursiveai.org/onemail/docs/compatibility/). The stores: [Mailboxes](https://recursiveai.org/onemail/docs/mailboxes/).


<!-- https://recursiveai.org/onemail/cookbook/create-eml/ -->

# Create an EML file in C#

Build the message from `MimeMultipart` and `MimePart`, add the headers, and call `WriteTo`. The program below writes an email with plain text and HTML bodies and a CSV attachment, then reads it back.

```shell
dotnet new console -o CreateEml && cd CreateEml
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using System.Globalization;
using System.Text;
using RecursiveAI.OneMail;
using RecursiveAI.OneMail.Mime;
using RecursiveAI.OneMail.Mime.Transfer;
using RecursiveAI.OneMail.Text;

var message = new MimeMultipart("mixed");
// Address headers are written as given: encode a non-ASCII display name yourself
message.Headers.Add("From", $"{Rfc2047.Encode("Zoë Martín")} <zoe@example.com>");
message.Headers.Add("To", "Sam Lee <sam@example.org>");
// Free text such as the subject is encoded for you
message.Headers.Add("Subject", "Q3 report – final");
string date = DateTime.UtcNow.ToString("ddd, dd MMM yyyy HH:mm:ss +0000",
    CultureInfo.InvariantCulture);
message.Headers.Add("Date", date);
message.Headers.Add("Message-ID", $"<{Guid.NewGuid():N}@example.com>");
message.Headers.Add("MIME-Version", "1.0");

// Plain text and HTML versions of the same body
var body = new MimeMultipart("alternative");
var text = new MimePart("text/plain; charset=utf-8");
text.SetDecodedContent(Utf8("Hi Sam,\r\n\r\nThe report is attached.\r\n"),
    ContentTransferEncoding.QuotedPrintable);
var html = new MimePart("text/html; charset=utf-8");
html.SetDecodedContent(Utf8("<p>Hi Sam,</p><p>The report is attached.</p>"),
    ContentTransferEncoding.QuotedPrintable);
body.Add(text);
body.Add(html);
message.Add(body);

// An attachment, base64-encoded (the default)
var csv = new MimePart("text/csv; name=\"q3.csv\"");
csv.Headers.Add("Content-Disposition", "attachment; filename=\"q3.csv\"");
csv.SetDecodedContent(Utf8("month,revenue\r\nJul,1200\r\nAug,1350\r\n"));
message.Add(csv);

using (FileStream file = File.Create("report.eml"))
    message.WriteTo(file);
Console.WriteLine($"report.eml: {new FileInfo("report.eml").Length:N0} bytes");

// Read it back
IMailItem check = Mail.Parse(File.ReadAllBytes("report.eml"));
Console.WriteLine($"{check.From[0]} -> {check.ToRecipients[0]}: {check.Subject}");
string plain = check.TextBody!.ReadText();
Console.WriteLine($"text: {plain.ReplaceLineEndings(" ").Trim()}");
IMailAttachment attached = check.Attachments[0];
Console.WriteLine($"attachment: {attached.FileName}, {attached.Size} bytes");
Console.WriteLine($"defects: {check.Defects.Count()}");

static byte[] Utf8(string s) => Encoding.UTF8.GetBytes(s);
```

```text
report.eml: 1,019 bytes
Zoë Martín <zoe@example.com> -> Sam Lee <sam@example.org>: Q3 report – final
text: Hi Sam,  The report is attached.
attachment: q3.csv, 35 bytes
defects: 0
```

`report.eml` opens in any mail program; Python’s `email` package also reads it with no defects. A boundary for each multipart is generated for you, and lines end in CRLF as MIME requires.

## How the parts fit

- `multipart/mixed` holds the body and the attachments.
- `multipart/alternative` inside it holds versions of the same body, plainest first. Mail programs show the last one they support.
- `SetDecodedContent` takes the raw bytes and encodes them: base64 by default, or quoted-printable for text people may read in the source.
- Header values must be one line. A line break throws, so user input can’t inject headers.

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **A low-level API.** There is no message builder yet. You set each header yourself, and you encode non-ASCII display names in address headers with `Rfc2047.Encode`, as above. Subjects and other free text are encoded for you.
- **No sending.** SMTP isn’t in this preview. The file is ready to hand to any SMTP client.
- **No Outlook formats.** OneMail can’t write MSG or PST files yet.

Reading messages back: [Parse an EML file](https://recursiveai.org/onemail/cookbook/parse-eml/). The writer: [Reading and writing MIME](https://recursiveai.org/onemail/docs/mime-reading/).


<!-- https://recursiveai.org/onemail/cookbook/convert-msg-to-eml/ -->

# Convert MSG to EML in C#

Open the `.msg` with `MsgFile.TryOpen`, convert its message with `MapiToMimeConverter`, and write the result. The conversion follows Microsoft’s published mapping between Outlook properties and MIME (MS-OXCMAIL). It also returns a loss report naming everything that didn’t carry over. The program below converts a public test message from Apache POI.

```shell
dotnet new console -o MsgToEml && cd MsgToEml
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
curl -LO https://raw.githubusercontent.com/apache/poi/REL_5_4_1/test-data/hsmf/attachment_test_msg.msg
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneMail;
using RecursiveAI.OneMail.Conversion;
using RecursiveAI.OneMail.Conversion.OxcMail;
using RecursiveAI.OneMail.Msg;

if (!MsgFile.TryOpen("attachment_test_msg.msg", out MsgFile? msg, out var defects))
{
    Console.WriteLine($"Not an Outlook message: {string.Join(", ", defects)}");
    return;
}
using (msg)
{
    // MAPI to MIME by Microsoft's rules (MS-OXCMAIL); Convert never throws on content
    var result = new MapiToMimeConverter().Convert(msg.Message);

    using (FileStream eml = File.Create("attachment_test_msg.eml"))
        result.Value.WriteTo(eml);
    long size = new FileInfo("attachment_test_msg.eml").Length;
    Console.WriteLine($"Wrote attachment_test_msg.eml, {size:N0} bytes");

    // Everything that did not carry over is in the loss report
    foreach (var group in result.Report.GroupBy(e => e.Kind))
        Console.WriteLine($"{group.Key}: {group.Count()} properties");
}

// Read it back as any mail client would
IMailItem copy = Mail.Parse(File.ReadAllBytes("attachment_test_msg.eml"));
Console.WriteLine($"{copy.Subject} | {copy.From[0]} | {copy.Sent?.Value:u}");
foreach (IMailAttachment a in copy.Attachments)
    Console.WriteLine($"  {a.FileName} ({a.Size:N0} bytes)");
```

```text
Wrote attachment_test_msg.eml, 35,184 bytes
Approximated: 5 properties
Internal: 46 properties
Dropped: 46 properties
test pièce jointe 1 | Nicolas1 23456 <nicolas1.23456@free.fr> | 2009-04-22 14:36:33Z
  test-unicode.doc (24,064 bytes)
  pj1.txt (89 bytes)
```

The `.eml` has the sender, recipient, date, subject, a plain text and an HTML body, and both attachments. The HTML was wrapped inside the message’s RTF body, and the converter unwraps it. Python’s `email` package and the `onemail` tool read the file too.

## The loss report

`result.Report` lists every source property that wasn’t carried exactly, each with a `Kind`, a rule `Code` and a reason:

- **Dropped** here are mostly attachment bookkeeping that MIME has no field for (`NoInternetMapping`), such as each attachment’s creation time.
- **Approximated** are values carried in a weaker form: the `Date` header holds whole seconds (`DateSubSecondDropped`), and the HTML came from the RTF body (`HtmlFromRtf`).
- **Internal** are store bookkeeping such as entry IDs and search keys, listed so nothing is silent.

Nothing goes missing without an entry. The project sent 70 Outlook items through MIME and back and accounted for all 11,142 properties: 0 silent losses.

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **Exchange addresses.** In the project’s words: “EX addresses with no SMTP form in the item are left out and reported”. Each one is an `AddressUnresolved` entry in the report.
- **Bcc** isn’t written by default. Set `new OxcMailExportOptions { IncludeBcc = true }` to keep it.
- **One way for files.** EML to MSG would need an MSG writer, which isn’t in this preview. `MimeToMapiConverter` converts to the MAPI model in memory.

For a whole mailbox, see [Convert a PST file to EML](https://recursiveai.org/onemail/cookbook/convert-pst-to-eml/). Every rule: [MAPI and MIME conversion](https://recursiveai.org/onemail/docs/conversion/).


<!-- https://recursiveai.org/onemail/cookbook/convert-pst-to-eml/ -->

# Convert a PST file to EML in C#

Walk the store’s folders with `PstStore`, convert each message with `MapiToMimeConverter`, and write one `.eml` per message. The program below exports Apache Tika’s public test store.

```shell
dotnet new console -o PstToEml && cd PstToEml
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
curl -LO https://raw.githubusercontent.com/apache/tika/4.2.0/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/test/resources/test-documents/testPST.pst
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneMail.Conversion.OxcMail;
using RecursiveAI.OneMail.Pst.Messaging;

if (!PstStore.TryOpen("testPST.pst", out PstStore? store, out var defects))
{
    Console.WriteLine($"Not a PST: {string.Join(", ", defects)}");
    return;
}

var converter = new MapiToMimeConverter();
int written = 0, unresolved = 0;
using (store)
{
    foreach (PstFolder folder in store!.EnumerateFolders())
    {
        foreach (PstMessage message in folder.EnumerateMessages())
        {
            // One folder per PST folder, one .eml per message
            string dir = Path.Combine(["eml", .. folder.Path.Split('/').Select(Safe)]);
            Directory.CreateDirectory(dir);
            var result = converter.Convert(message);
            string file = Path.Combine(dir, $"{++written:D5}.eml");
            using (FileStream eml = File.Create(file))
                result.Value.WriteTo(eml);

            // Exchange-only senders with no SMTP address are left out and reported
            unresolved += result.Report.Count(e => e.Code == "AddressUnresolved");
            Console.WriteLine($"{file}  {new FileInfo(file).Length,7:N0} bytes");
        }
    }
}
Console.WriteLine($"{written} messages, {unresolved} unresolved addresses");

static string Safe(string name) =>
    string.Concat(name.Select(c =>
        Path.GetInvalidFileNameChars().Contains(c) ? '_' : c));
```

```text
eml/Début du fichier de données Outlook/00001.eml    5,899 bytes
eml/Début du fichier de données Outlook/00002.eml    9,006 bytes
eml/Début du fichier de données Outlook/00003.eml    4,493 bytes
eml/Début du fichier de données Outlook/00004.eml    4,745 bytes
eml/Début du fichier de données Outlook/00005.eml    5,277 bytes
eml/Début du fichier de données Outlook/00006.eml   18,368 bytes
eml/Début du fichier de données Outlook/00007.eml   21,015 bytes
7 messages, 2 unresolved addresses
```

Every file opens in Python’s `email` package and with the `onemail` tool. Messages that arrived over the Internet keep the headers they arrived with. The forwarded message in `00007.eml` is still attached as a message. Messages are read one at a time, so a large store doesn’t have to fit in memory.

## The two unresolved addresses

Message 7 was sent inside an Exchange organization. Its sender has only an Exchange (X.500) address, with no SMTP form in the store, so the `.eml` has no `From` header. The report lists it as `AddressUnresolved`. The project reports the same gap on its own test set: 30 such entries over 70 items.

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **Exchange addresses.** There is no directory lookup: “EX addresses with no SMTP form in the item are left out and reported”.
- **Bcc** isn’t written by default (`OxcMailExportOptions.IncludeBcc`).
- **Mail only.** This loop writes messages. To write appointments and tasks as `.ics` files too, convert the whole store with `Mail.ConvertAsync`, as in [Convert a PST file to mbox](https://recursiveai.org/onemail/cookbook/convert-pst-to-mbox/).
- **Search folders and hidden items** are skipped, so no message is written twice.

To write one mbox instead of many files, see [Convert a PST file to mbox](https://recursiveai.org/onemail/cookbook/convert-pst-to-mbox/). One message at a time: [Convert MSG to EML](https://recursiveai.org/onemail/cookbook/convert-msg-to-eml/). Every rule: [MAPI and MIME conversion](https://recursiveai.org/onemail/docs/conversion/).


<!-- https://recursiveai.org/onemail/cookbook/convert-pst-to-mbox/ -->

# Convert a PST file to mbox in C#

Call `Mail.ConvertAsync(source, ConvertFormat.Mbox, destination)`. It reads the store one message at a time and writes one mbox file per Outlook folder, the format most mail programs import. The same call takes OST, OLM, DBX, mbox and Maildir sources, and writes EML, Maildir, Apple Mail, ICS or vCard instead. The program below converts the PST test file of Apache Tika.

```shell
dotnet new console -o PstToMbox && cd PstToMbox
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
curl -LO https://raw.githubusercontent.com/apache/tika/4.2.0/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/test/resources/test-documents/testPST.pst
```

Put this in `Program.cs` and run `dotnet run` (or `dotnet run -- yourfile.pst`):

```csharp
using RecursiveAI.OneMail;
using RecursiveAI.OneMail.Conversion.Stores;

string source = args.Length > 0 ? args[0] : "testPST.pst";

// One .mbox per folder under "mbox"; calendar items and contacts go to mbox/_items
StoreConversionReport report = await Mail.ConvertAsync(source, ConvertFormat.Mbox, "mbox");

Console.WriteLine($"read {report.ItemsRead}, written {report.ItemsWritten}, skipped {report.ItemsSkipped}");
foreach (var (folder, count) in report.Folders)
    Console.WriteLine($"  {folder}: {count.Written} of {count.Read}");

// What didn't carry over, by code, most frequent first
foreach (LossTotal loss in report.Losses.Take(5))
    Console.WriteLine($"  {loss.Kind} {loss.Code} in {loss.Items} items, e.g. {loss.Example}");

foreach (string file in Directory.GetFiles("mbox", "*", SearchOption.AllDirectories))
    Console.WriteLine($"{file}  {new FileInfo(file).Length:N0} bytes");
```

```text
read 7, written 7, skipped 0
  Début du fichier de données Outlook: 7 of 7
  Approximated DefaultValueNotWritten in 7 items, e.g. PidTagSensitivity
  Dropped NoInternetMapping in 7 items, e.g. PidTagPriority
  Internal StoreBookkeeping in 7 items, e.g. PidTagSentRepresentingSearchKey
  Approximated SubjectPrefixMarkerRemoved in 7 items, e.g. PidTagSubject
  Internal FlagsWrittenAsHeaders in 7 items, e.g. Status
mbox/Début du fichier de données Outlook.mbox  69,216 bytes
```

Python’s `mailbox` module reads the seven messages back from the file.

## The report

Every message is accounted for: items read always equal items written plus items skipped, and `SkippedByReason` says why any were skipped. `Losses` totals what didn’t carry over across the run, by code. Outlook properties with no internet form (`NoInternetMapping`), the transport headers rewritten for the new message, and reply-to entries with no address are the usual ones. Pass an `IProgress<StoreItemResult>` to see each message’s own report as it’s written.

## Options

- **One file.** A destination with an extension, such as `"all.mbox"`, writes a single mbox; folders are flattened and the report says so.
- **Other formats.** `ConvertFormat.Eml`, `Maildir` (Maildir++ subfolders with flags), `AppleMail`, `Ics`, `VCard` and their JSON and XML forms, or `Tnef`.
- **Skip calendars and contacts.** `new StoreConversionOptions { OtherItems = OtherItemsMode.Skip }`.
- **One folder.** `FolderFilter = "Inbox"` converts that folder and the folders under it.
- **From the command line**, the same conversion: `onemail convert --to mbox testPST.pst mbox`.

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **Not to Outlook.** In the project’s words: “MSG and PST are not destinations until 0.5”.
- **Outlook flags.** “flags and keywords from Outlook items (`PidTagFlagStatus`, last verb) are mapped by rule and not yet observed in Outlook”.
- **Exchange addresses.** There is no directory lookup: “EX addresses with no SMTP form in the item are left out and reported”.

What carries over for each source and target: [Store to store and fidelity](https://recursiveai.org/onemail/docs/fidelity/). One message at a time: [Convert a PST file to EML](https://recursiveai.org/onemail/cookbook/convert-pst-to-eml/).


<!-- https://recursiveai.org/onemail/cookbook/extract-attachments/ -->

# Extract attachments from email in C#

Open the file with `Mail.Open`, and for each message copy `attachment.Open()` to a file. The same loop works for `.eml`, Outlook `.msg`, `winmail.dat`, PST stores and mbox files. The program below saves the attachments of three public test files from Apache Tika and Apache POI.

```shell
dotnet new console -o ExtractAttachments && cd ExtractAttachments
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
curl -LO https://raw.githubusercontent.com/apache/tika/4.2.0/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-mail-module/src/test/resources/test-documents/testRFC822_multipart_attachments.eml
curl -LO https://raw.githubusercontent.com/apache/poi/REL_5_4_1/test-data/hsmf/attachment_test_msg.msg
curl -LO https://raw.githubusercontent.com/apache/poi/REL_5_4_1/test-data/hmef/quick-winmail.dat
```

Put this in `Program.cs` and run `dotnet run` (or `dotnet run -- yourfile.eml`):

```csharp
using RecursiveAI.OneMail;

// Any message file or mailbox works the same: .eml, .msg, winmail.dat, .pst, mbox
string[] inputs = args.Length > 0 ? args :
[
    "testRFC822_multipart_attachments.eml",
    "attachment_test_msg.msg",
    "quick-winmail.dat",
];

foreach (string input in inputs)
{
    string dir = Path.Combine("attachments", Path.GetFileNameWithoutExtension(input));
    Directory.CreateDirectory(dir);

    using MailContainer mail = Mail.Open(input);
    foreach (IMailItem msg in mail.Enumerate())
    {
        foreach (IMailAttachment a in msg.Attachments)
        {
            // FileName is what the sender wrote: keep only the name, never a path
            string name = Path.GetFileName(a.FileName ?? "");
            if (name == "") name = "attachment.bin";
            string path = Path.Combine(dir, name);
            for (int i = 2; File.Exists(path); i++)
                path = Path.Combine(dir, $"{i}-{name}");

            using (Stream content = a.Open())   // decoded content
            using (FileStream file = File.Create(path))
                content.CopyTo(file);
            Console.WriteLine($"{path}  {new FileInfo(path).Length:N0} bytes  {a.ContentType}");
        }
    }
}
```

```text
attachments/testRFC822_multipart_attachments/Test TxtA.txt  33 bytes  text/plain
attachments/testRFC822_multipart_attachments/Test TxtB.txt  33 bytes  text/plain
attachments/attachment_test_msg/test-unicode.doc  24,064 bytes  application/msword
attachments/attachment_test_msg/pj1.txt  89 bytes  text/plain
attachments/quick-winmail/quick.doc  19,968 bytes  application/msword
attachments/quick-winmail/quick.html  428 bytes  text/html
attachments/quick-winmail/quick.pdf  18,638 bytes  application/pdf
attachments/quick-winmail/quick.txt  235 bytes  text/plain
attachments/quick-winmail/quick.xml  143 bytes  text/xml
```

`Open()` returns the decoded content: base64 and quoted-printable are undone, and Outlook attachments come straight from the file. The five files from `quick-winmail.dat` are byte-identical to the copies Apache POI keeps beside it.

## Things to know

- **Never use the name as a path.** `FileName` is what the sender wrote, and it can hold `../` or a full path. `Path.GetFileName` keeps only the name; two attachments with the same name don’t overwrite each other.
- **Inline images** (pictures the HTML body shows through `cid:` links) are attachments too, with `IsInline` set. Skip them with `if (a.IsInline) continue;`.
- **Attached emails** have `Embedded` set: read that `IMailItem` the same way to reach the files inside it.
- **winmail.dat inside an email** is unpacked for you: the message lists the files in it instead.
- **Content types.** `ContentType` is the type the message stored. When Outlook stored none, or only `application/octet-stream`, a known file extension decides it (`.pdf` is `application/pdf`) and `ContentTypeInferred` is true.

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **`Size` costs a pass.** It decodes the content once to measure it, so prefer the file length after copying, as above.

Reading the rest of a message: [Parse an EML file](https://recursiveai.org/onemail/cookbook/parse-eml/), [Read an Outlook MSG file](https://recursiveai.org/onemail/cookbook/read-msg-file/). The shared view: [Mail.Open and the read view](https://recursiveai.org/onemail/docs/shared-view/).


<!-- https://recursiveai.org/onemail/cookbook/convert-msg-to-ics/ -->

# Convert an Outlook appointment (MSG) to ICS in C#

Open the `.msg` with `Mail.Open` and call `Mail.Convert(item, ConvertFormat.Ics)`. You get the iCalendar bytes, ready for any calendar app, and a report of every Outlook property that didn’t carry over. Meetings, tasks and recurring series convert the same way, following Microsoft’s published MS-OXCICAL rules. The program below converts the appointment test file of Apache POI.

```shell
dotnet new console -o MsgToIcs && cd MsgToIcs
dotnet add package RecursiveAI.OneMail --version 0.3.0-preview.2
curl -LO https://raw.githubusercontent.com/apache/poi/REL_5_4_1/test-data/hsmf/msgClassAppointment.msg
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneMail;
using RecursiveAI.OneMail.Conversion;
using RecursiveAI.OneMail.Conversion.Stores;

using MailContainer file = Mail.Open("msgClassAppointment.msg");
IMailItem item = file.Enumerate().First();
Console.WriteLine($"{item.Kind}: {item.Subject}");

// Appointments, meetings and tasks become iCalendar; a plain email gives null
ConversionResult<byte[]?> ics = Mail.Convert(item, ConvertFormat.Ics);
if (ics.Value is null) { Console.WriteLine("Not a calendar item"); return; }
File.WriteAllBytes("appointment.ics", ics.Value);
Console.WriteLine($"appointment.ics  {ics.Value.Length:N0} bytes");

// What didn't carry over: content first, store bookkeeping only counted
foreach (var loss in ics.Report.Where(l => l.Kind != LossKind.Internal)
                               .GroupBy(l => (l.Kind, l.Code)))
    Console.WriteLine($"  {loss.Key.Kind} {loss.Key.Code}: {loss.Count()}");
Console.WriteLine($"  Internal: {ics.Report.Count(l => l.Kind == LossKind.Internal)}");
```

```text
Appointment: Quick brown fox
appointment.ics  942 bytes
  Dropped OrganizerWithoutSmtpAddress: 1
  Dropped RtfBodyNotCarried: 1
  Dropped NoIcalMapping: 49
  Internal: 33
```

The file it writes:

```text
BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Recursive AI//OneMail//EN
METHOD:PUBLISH
BEGIN:VTIMEZONE
TZID:Eastern Standard Time
BEGIN:STANDARD
DTSTART:20071104T020000
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20070311T020000
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
UID:040000008200E00074C5B7101A82E008000000003042F07DC891D201000000000000000
 01000000091B31F0C0B120942B66C023A533DD16C
DTSTAMP:20170228T184317Z
DTSTART;TZID=Eastern Standard Time:20170228T133000
DTEND;TZID=Eastern Standard Time:20170228T140000
SUMMARY:Quick brown fox
LOCATION:under lazy dog
DESCRIPTION:This is a meeting to discuss jumping
CLASS:PUBLIC
PRIORITY:5
TRANSP:OPAQUE
X-MICROSOFT-CDO-BUSYSTATUS:BUSY
SEQUENCE:0
CREATED:20170228T184317Z
LAST-MODIFIED:20170228T184317Z
END:VEVENT
END:VCALENDAR
```

## What carries over

Python’s `icalendar` 7.3.0 reads the file back as 13:30 to 14:00 on February 28, 2017, at UTC−05:00, US Eastern time. Start and end keep the item’s own time zone, written as a `VTIMEZONE`. Subject, location, description, attendees with their roles and responses, the organizer, reminder, busy status, privacy and the recurrence pattern with its exceptions all map to their iCalendar properties. The UID comes from Outlook’s global object ID, so updates to the same meeting match up in other calendars.

The report names the rest. Here the organizer had no SMTP address in the file, so `ORGANIZER` is left out (`OrganizerWithoutSmtpAddress`), and the RTF body isn’t carried (its text is the `DESCRIPTION`). `NoIcalMapping` entries are Outlook properties iCalendar has no field for. `Internal` entries are store bookkeeping such as entry IDs.

## A whole calendar

To convert every appointment of a PST, OST or OLM, pass the store to `Mail.ConvertAsync(path, ConvertFormat.Ics, "calendar")`: one `.ics` per item, in a folder tree like the store’s. From the command line: `onemail convert --to ics archive.pst calendar`. For the reverse direction, `IcsToMapiConverter` turns iCalendar into an Outlook item in memory.

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **No Outlook-written meetings yet.** The project checked this on the two appointments of the public test files and on generated items: “the corpus holds 2 distinct appointment and 2 task items and no meeting or recurring item: those are covered by generated items following the MS-OXOCAL examples”.
- **No MSG out.** iCalendar to Outlook works in memory only; OneMail can’t write MSG or PST files yet.

Every mapping and loss code: [Outlook calendar and task items](https://recursiveai.org/onemail/docs/calendar-items/). Reading the `.ics` back: [Read an ICS calendar file](https://recursiveai.org/onemail/cookbook/read-ics-file/).


<!-- https://recursiveai.org/onemail/cookbook/read-ics-file/ -->

# Read an ICS calendar file in C#

Parse the file with `ContentDocument.Parse`, take the calendar, and call `CalendarRecurrence.Expand` for every occurrence in a date range. Repeat rules, exceptions, moved instances and time zones are applied for you. The program below reads a small calendar with a weekly meeting that crosses the US daylight saving change.

```shell
dotnet new console -o ReadIcs && cd ReadIcs
dotnet add package RecursiveAI.OneMail.Calendar --version 0.3.0-preview.2
```

Save this as `team.ics`: a standup on Mondays and Wednesdays in New York time, eight times, minus March 10, with March 15 moved to the afternoon, and an all-day launch.

```text
BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Example//Team calendar//EN
BEGIN:VEVENT
UID:standup@example.com
DTSTAMP:20270101T000000Z
SUMMARY:Team standup
LOCATION:Room 4
DTSTART;TZID=America/New_York:20270301T093000
DTEND;TZID=America/New_York:20270301T094500
RRULE:FREQ=WEEKLY;BYDAY=MO,WE;COUNT=8
EXDATE;TZID=America/New_York:20270310T093000
END:VEVENT
BEGIN:VEVENT
UID:standup@example.com
DTSTAMP:20270101T000000Z
RECURRENCE-ID;TZID=America/New_York:20270315T093000
SUMMARY:Team standup (moved)
DTSTART;TZID=America/New_York:20270315T140000
DTEND;TZID=America/New_York:20270315T141500
END:VEVENT
BEGIN:VEVENT
UID:launch@example.com
DTSTAMP:20270101T000000Z
SUMMARY:Launch day
DTSTART;VALUE=DATE:20270322
END:VEVENT
END:VCALENDAR
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneMail.Calendar;
using RecursiveAI.OneMail.Calendar.Ics;
using RecursiveAI.OneMail.Calendar.Recurrence;

ContentDocument doc = ContentDocument.Parse(File.ReadAllBytes("team.ics"));
IcsComponent calendar = IcsDocument.Calendars(doc).First();

// The events as written: one entry per VEVENT
foreach (IcsComponent ev in calendar.Events)
    Console.WriteLine($"{ev.Summary,-21} {ev.Start,-16} {ev.GetText("RRULE") ?? "-"}");

// Every occurrence: RRULE, EXDATE and moved instances applied, time zones resolved
Console.WriteLine();
var window = new OccurrenceWindow(new DateTime(2027, 3, 1), new DateTime(2027, 4, 1));
foreach (Occurrence o in CalendarRecurrence.Expand(calendar, window))
{
    string utc = o.StartUtc is { } u ? $"{u:yyyy-MM-dd HH:mm} UTC" : "all day";
    string format = o.Start.IsDate ? "ddd yyyy-MM-dd" : "ddd yyyy-MM-dd HH:mm";
    string local = o.Start.Value.ToString(format);
    Console.WriteLine($"{local,-20}  {utc,-20}  {o.Component.Summary}");
}
```

```text
Team standup          20270301T093000  FREQ=WEEKLY;BYDAY=MO,WE;COUNT=8
Team standup (moved)  20270315T140000  -
Launch day            20270322         -

Mon 2027-03-01 09:30  2027-03-01 14:30 UTC  Team standup
Wed 2027-03-03 09:30  2027-03-03 14:30 UTC  Team standup
Mon 2027-03-08 09:30  2027-03-08 14:30 UTC  Team standup
Mon 2027-03-15 14:00  2027-03-15 18:00 UTC  Team standup (moved)
Wed 2027-03-17 09:30  2027-03-17 13:30 UTC  Team standup
Mon 2027-03-22        all day               Launch day
Mon 2027-03-22 09:30  2027-03-22 13:30 UTC  Team standup
Wed 2027-03-24 09:30  2027-03-24 13:30 UTC  Team standup
```

The three `VEVENT`s become eight occurrences: the rule’s eight dates, minus the excluded March 10, with March 15 replaced by its moved copy. The standup stays at 9:30 in New York while its UTC time moves from 14:30 to 13:30 after daylight saving starts on March 14. All-day events have no UTC instant.

## What you get

- **The typed view.** `IcsComponent` has `Summary`, `Start`, `End`, `Location`, `Uid` and the rest. `GetText` reads any property.
- **Time zones.** A zone resolves from a `VTIMEZONE` in the file, the IANA database, or a Windows zone name.
- **Lossless.** An untouched file writes back byte for byte (`doc.Write(stream)`), and only the lines you edit are rewritten.
- **No exceptions on bad input.** Repairs and guesses are collected in an `ExpansionReport` you can pass to `Expand`.

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **Mind the window.** A rule with no `COUNT` or `UNTIL` never ends. Expansion stops at the window’s end, and `ExpansionOptions` caps the count.
- **RSCALE is refused.** Non-Gregorian rules (RFC 7529) aren’t expanded.
- **Windows.** IANA names resolve through the Windows zone each maps to, so US dates before 2007 follow today’s rules there.
- **Outlook appointments** in MSG and PST files aren’t ICS: [convert them to ICS](https://recursiveai.org/onemail/cookbook/convert-msg-to-ics/) first.

Every rule and choice: [Recurrence and time zones](https://recursiveai.org/onemail/docs/recurrence/). The parser: [ICS and vCard](https://recursiveai.org/onemail/docs/calendar/).


<!-- https://recursiveai.org/onemail/cookbook/read-vcf-file/ -->

# Read a vCard (.vcf) file in C#

Parse the file with `ContentDocument.Parse` and read each card through `VCardDocument.Cards`: names, emails, phones, addresses and dates come back typed and decoded. vCard 2.1, 3.0 and 4.0 all read the same way. The program below reads three contacts, one in each version.

```shell
dotnet new console -o ReadVcf && cd ReadVcf
dotnet add package RecursiveAI.OneMail.Calendar --version 0.3.0-preview.2
```

Save this as `contacts.vcf`. The 2.1 card is quoted-printable UTF-8, as older phones and Outlook export it.

```text
BEGIN:VCARD
VERSION:3.0
FN:Ana Ruiz
N:Ruiz;Ana;;;
ORG:Example Corp;Sales
TITLE:Account manager
EMAIL;TYPE=INTERNET,WORK:ana.ruiz@example.com
TEL;TYPE=WORK,VOICE:+1 555 0100
TEL;TYPE=CELL:+1 555 0199
ADR;TYPE=WORK:;;100 Main St;Springfield;IL;62701;USA
BDAY:1988-04-12
END:VCARD
BEGIN:VCARD
VERSION:2.1
N;CHARSET=UTF-8;ENCODING=QUOTED-PRINTABLE:M=C3=BCller;J=C3=BCrgen
FN;CHARSET=UTF-8;ENCODING=QUOTED-PRINTABLE:J=C3=BCrgen M=C3=BCller
EMAIL;PREF;INTERNET:juergen@example.de
TEL;HOME:+49 30 1234567
END:VCARD
BEGIN:VCARD
VERSION:4.0
FN:Sam Lee
N:Lee;Sam;;;
EMAIL;TYPE=home;PREF=1:sam@example.org
TEL;VALUE=uri;TYPE="voice,cell":tel:+1-555-0142
BDAY:--0615
END:VCARD
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneMail.Calendar;
using RecursiveAI.OneMail.Calendar.VCard;

// vCard 2.1, 3.0 and 4.0 in one file; quoted-printable and CHARSET are decoded
ContentDocument doc = ContentDocument.Parse(File.ReadAllBytes("contacts.vcf"));

foreach (VCardView card in VCardDocument.Cards(doc))
{
    Console.WriteLine($"{card.FormattedName} (vCard {card.VersionText})");
    if (card.Name is { } n)
        Console.WriteLine($"  name:  {Join(n.Given)} / {Join(n.Family)}");
    if (card.Organization.Count > 0)
    {
        string org = string.Join(", ", card.Organization);
        Console.WriteLine($"  org:   {org}; {card.Title}");
    }
    foreach (VCardTyped email in card.Emails)
        Console.WriteLine($"  email: {email.Value} [{string.Join(",", email.Types)}]");
    foreach (VCardTyped phone in card.Phones)
        Console.WriteLine($"  phone: {phone.Value} [{string.Join(",", phone.Types)}]");
    foreach (VCardAddress adr in card.Addresses)
        Console.WriteLine($"  addr:  {Join(adr.Street)}, {Join(adr.Locality)}");
    // vCard 4.0 allows a birthday without a year
    if (card.Birthday is { } b)
    {
        string year = b.Year?.ToString() ?? "-";
        Console.WriteLine($"  bday:  {year}-{b.Month:00}-{b.Day:00}");
    }
}

static string Join(IReadOnlyList<string> parts) => string.Join(" ", parts);
```

```text
Ana Ruiz (vCard 3.0)
  name:  Ana / Ruiz
  org:   Example Corp, Sales; Account manager
  email: ana.ruiz@example.com [INTERNET,WORK]
  phone: +1 555 0100 [WORK,VOICE]
  phone: +1 555 0199 [CELL]
  addr:  100 Main St, Springfield
  bday:  1988-04-12
Jürgen Müller (vCard 2.1)
  name:  Jürgen / Müller
  email: juergen@example.de [PREF,INTERNET]
  phone: +49 30 1234567 [HOME]
Sam Lee (vCard 4.0)
  name:  Sam / Lee
  email: sam@example.org [home]
  phone: tel:+1-555-0142 [voice,cell]
  bday:  --06-15
```

The quoted-printable name decodes to “Jürgen Müller”. Type parameters are kept as written in each version (`INTERNET,WORK`, `PREF`, `voice,cell`). The 4.0 birthday has no year, which vCard 4.0 allows.

## What you get

- **Every field as a list.** `VCardName` and `VCardAddress` hold lists, because a vCard field can have several values.
- **The rest of RFC 6350:** `Photos` (inline or by URL), `Impps`, `Related`, `Categories`, `Note`, `Uid` and more.
- **Lossless.** An untouched file writes back byte for byte with `doc.Write(stream)`.
- **Outlook contacts.** `RecursiveAI.OneMail` reads contacts from MSG and PST files as `MapiContact`, and converts them to and from vCard with a loss report.

## Limits

- **A preview.** 0.3.0-preview.2: the API can change before 1.0.
- **Unchecked against Outlook.** The project lists this gap: “no Outlook-produced contact PST or MSG with the full property set, and no Outlook import/export of vCards to compare against”.
- **Phone and email slots.** Outlook’s fixed slots (business phone, mobile, email 1 to 3) map to vCard only approximately. The conversion reports each such change (`PhoneSlotChanged`, `EmailSlotChanged`).

Everything about contacts: [Contacts](https://recursiveai.org/onemail/docs/contacts/). The parser: [ICS and vCard](https://recursiveai.org/onemail/docs/calendar/).


<!-- https://recursiveai.org/onemail/cookbook/inspect-email-cli/ -->

# Inspect email files from the command line

Install the `onemail` .NET tool and run `onemail inspect` on files or folders. Each message gets one line with its date, sender, subject, attachments and defects. Add `--json` for machine-readable output. It reads every format the library reads: `.eml`, `.msg`, `.pst`, `.ost`, mbox, Maildir, `winmail.dat`, Outlook Express `.dbx` and Outlook for Mac `.olm`. `onemail convert` writes them in another format.

```shell
dotnet tool install -g RecursiveAI.OneMail.Cli --version 0.3.0-preview.2
onemail --version
```

```text
0.3.0-preview.2
```

The tool needs the .NET 10 runtime. The examples use public test files from Apache POI and Apache Tika:

```shell
curl -LO https://raw.githubusercontent.com/apache/poi/REL_5_4_1/test-data/hsmf/attachment_test_msg.msg
curl -LO https://raw.githubusercontent.com/apache/poi/REL_5_4_1/test-data/hmef/quick-winmail.dat
curl -LO https://raw.githubusercontent.com/apache/tika/4.2.0/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-mail-module/src/test/resources/test-documents/complex.mbox
curl -LO https://raw.githubusercontent.com/apache/tika/4.2.0/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-mail-module/src/test/resources/test-documents/testRFC822_multipart_attachments.eml
```

## One file

```shell
onemail inspect attachment_test_msg.msg
```

```text
attachment_test_msg.msg: Msg
  #0  2009-04-22T14:36:33Z  from: Nicolas1 23456 <nicolas1.23456@free.fr>  subject: test pièce jointe 1  attachments: 2 (test-unicode.doc, pj1.txt)  defects: 0
inspected 1 files/stores, 1 items, 2 attachments, 0 defects (0 ambiguities), 0 not readable
```

The last line is a summary, written to standard error.

## A folder, with attachment sizes

`--sizes` adds each attachment’s decoded size in bytes. A folder is searched for every file it can read:

```shell
onemail inspect --sizes .
```

```text
./attachment_test_msg.msg: Msg
  #0  2009-04-22T14:36:33Z  from: Nicolas1 23456 <nicolas1.23456@free.fr>  subject: test pièce jointe 1  attachments: 2 (test-unicode.doc 24064B, pj1.txt 89B)  defects: 0
./complex.mbox: Mbox
  #0  2009-06-01T04:26:31Z  from: Jothi Padmanabhan <jothipn@yahoo-inc.com>  subject: Re: question about when shuffle/sort start working  attachments: 0  defects: 0
  #1  2009-06-01T05:30:49Z  from: Jianmin Woo <jianmin_woo@yahoo.com>  subject: Re: question about when shuffle/sort start working  attachments: 0  defects: 0
  #2  2009-06-01T06:03:13Z  from: Jothi Padmanabhan <jothipn@yahoo-inc.com>  subject: Re: question about when shuffle/sort start working  attachments: 0  defects: 0
./quick-winmail.dat: Tnef
  #0  -  from: -  subject: -  attachments: 5 (quick.doc 19968B, quick.html 428B, quick.pdf 18638B, quick.txt 235B, quick.xml 143B)  defects: 1
./testRFC822_multipart_attachments.eml: Eml
  #0  -  from: Bugs Bunny <Bugs.Bunny@acme.com>  subject: EML 1  attachments: 2 (Test TxtA.txt 33B, Test TxtB.txt 33B)  defects: 0
inspected 4 files/stores, 6 items, 9 attachments, 1 defects (0 ambiguities), 0 not readable
```

## Defects

`--defects` lists each repair under its message, with the byte offset where it was found:

```shell
onemail inspect --defects quick-winmail.dat
```

```text
quick-winmail.dat: Tnef
  #0  -  from: -  subject: -  attachments: 5 (quick.doc, quick.html, quick.pdf, quick.txt, quick.xml)  defects: 1
      mapi:LowerLayerDefect @40
inspected 1 files/stores, 1 items, 5 attachments, 1 defects (0 ambiguities), 0 not readable
```

## JSON

`--json` writes one JSON object per line: one for each file or store, then one for each message.

```shell
onemail inspect --json attachment_test_msg.msg
```

```text
{"type":"container","path":"attachment_test_msg.msg","format":"Msg"}
{"type":"item","index":0,"kind":"Message","provenance":{"format":"Msg","location":"attachment_test_msg.msg","folder":null,"offset":0,"length":52224},"subject":"test pi\u00E8ce jointe 1","sender":[],"from":[{"name":"Nicolas1 23456","addr":"nicolas1.23456@free.fr"}],"to":[{"name":"\u0027nicolas1.23456@free.fr\u0027","addr":"nicolas1.23456@free.fr"}],"cc":[],"bcc":[],"replyTo":[],"date":{"utc":"2009-04-22T14:36:33Z","offsetMinutes":null,"source":"ClientSubmit"},"messageId":null,"inReplyTo":[],"references":[],"conversationTopic":"test pi\u00E8ce jointe 1","attachments":[{"name":"test-unicode.doc","type":"application/msword","contentId":null,"inline":false,"embedded":false},{"name":"pj1.txt","type":"text/plain","contentId":null,"inline":false,"embedded":false}],"defects":[]}
```

## Convert

`onemail convert` writes a file or a whole store in another format and lists what didn’t carry over. `--to` takes `eml`, `mbox`, `maildir`, `emlx`, `ics`, `vcf`, `jcal`, `jcard`, `xcal`, `xcard` or `tnef`:

```shell
onemail convert --to eml attachment_test_msg.msg eml
```

```text
read 1 items, wrote 1 as Eml to eml, skipped 0, 1 with something that did not carry over
  Approximated DateSubSecondDropped (mapi) in 1 items, for example PidTagClientSubmitTime
  Approximated DefaultValueNotWritten (mapi) in 1 items, for example PidTagSensitivity
  Approximated HtmlFromRtf (mapi) in 1 items, for example PidTagRtfCompressed
  Dropped NoInternetMapping (mapi) in 1 items, for example PidTagCreationTime
  Dropped FlagsNotCarried (store) in 1 items, for example flags
```

The message is now `eml/000001-test-pi-ce-jointe-1.eml`. `onemail export` does the same for a whole store into a directory, and `--json` makes both machine-readable. The library call behind them: [Convert a PST file to mbox](https://recursiveai.org/onemail/cookbook/convert-pst-to-mbox/).

## Limits

- **A preview.** 0.3.0-preview.2: the commands and their output can change before 1.0.
- **No extract command.** To save attachments, use the library, as in [Extract attachments from email](https://recursiveai.org/onemail/cookbook/extract-attachments/).
- **Not for calendar files.** `inspect` counts `.ics` and `.vcf` files as not readable; read them with [RecursiveAI.OneMail.Calendar](https://recursiveai.org/onemail/cookbook/read-ics-file/).

The tool’s options: [Mail.Open and the read view](https://recursiveai.org/onemail/docs/shared-view/).