Recursive AI

Ambiguity table (version 1)

All onemail docs

Real mail is full of messages that the RFCs call invalid or leave open, and the established libraries each pick their own answer. OneMail picks one documented answer per case, reports it as a typed defect, and keeps the original bytes so nothing is lost whichever answer a caller prefers. This file lists each case, what three released libraries do with it, and the resolution OneMail takes.

How the table was made

  • Each case is a small invented message in testdata/differential (generated by tools/corpus/make_differential.py; cases.tsv there describes every file).
  • Each message was run through MimeKit 4.18.2, Python 3.13 email (policy.default, message_from_bytes) and Go 1.26 net/mail + mime/multipart, through the drivers in tools/oracles. The columns below are what those runs showed. Where the three agree, OneMail does the same.
  • Where they differ, the rule is: prefer the answer two of the three give; otherwise prefer the answer that loses no bytes and follows the RFC text most closely. No mail client has been observed for any entry yet, so every resolution that is not "all three agree" is provisional: it is a default, to be revisited when a client's behaviour is observed. Provisional entries are marked prov.
  • "Defect" is the typed defect the parser raises; every repair and every guess carries the byte offset. Names are the intended catalogue names; Ambiguous* marks a choice between valid readings, the others mark a repair.
  • Compared forms, normalisation and the difference classes are in tools/oracles/README.md.

Abbreviations: MK MimeKit, Py Python, Go Go standard library.

Boundaries and multipart structure

IDProbe (testdata/differential/…)MKPyGoResolutionDefectBasis
B1bnd-02 a content line --abcdef when the boundary is abccontentcontentcontentA delimiter line is -- + boundary + optional trailing whitespace; a longer line is content.noneall agree
B2bnd-03 trailing spaces and tabs after a delimiterdelimiterdelimiterdelimiterTrailing space and tab are padding (RFC 2046 5.1.1).noneall agree
B3bnd-04 --abc xyz: text after the boundarycontentcontentcontentNot a delimiter; the line stays in the current part.AmbiguousBoundaryLikeLineall agree
B4bnd-14 delimiter in another case (--ABC)contentcontentcontentBoundaries are case-sensitive.noneall agree
B5bnd-07, bnd-08 one boundary is a prefix of the othernested correctlysamesameLongest-exact-match wins: a line closes or starts a part only if it equals a boundary in scope.noneall agree
B6bnd-06 text after the closing delimiter, then another --abcepilogueepilogueepilogueEverything after the closing delimiter is epilogue, including later delimiter-looking lines.EpilogueHasDelimiterall agree
B7bnd-05 closing delimiter missinglast part keeps its final line break (5 bytes)strips it (3)strips it (3)The last part runs to end of input and keeps every byte; no line break is assumed to belong to a delimiter that is not there.MissingFinalBoundaryprov. (MK; keeps all bytes)
B8bnd-09 multipart with no boundarymultipart, no parts, body not exposedleaf, body keptleaf, body keptTreated as a leaf of the declared type with the body kept; no parts.MultipartWithoutBoundary2 of 3
B9bnd-10 boundary=""no partsleafleafSame as B8.MultipartWithoutBoundary2 of 3
B10bnd-25 multipart whose body has no delimiter at allno partsleaf, body keptno partsNo parts; the whole body is the preamble, bytes preserved.MissingStartBoundary2 of 3
B11bnd-11 boundary parameter that ends in a spacenone foundfinds a part2 partsThe boundary is matched literally and also with trailing space trimmed.BoundaryHasTrailingSpaceprov.
B12bnd-12 unquoted boundary=a/b=caccepted wholecut at =media type rejectedThe unquoted value runs to ; or end of value.none (not raised)prov. (MK)
B13bnd-13 two boundary parametersfirstfirstmedia type rejected (leaf)First occurrence wins.DuplicateParameter2 of 3
B14bnd-15 multipart with only --abc--0 partsleaf, empty0 parts0 parts.none2 of 3
B15bnd-16 first delimiter directly after the headers, no blank lineheader block never ends, 0 parts1 partheader errorThe first line that is not a header field or continuation ends the header block and starts the body (see H5).MissingHeaderBodySeparatorprov. (Py)
B16bnd-17 LF-only delimiters inside a CRLF message2 parts2 parts1 partEach line may end in LF or CRLF independently.MixedLineEndings2 of 3
B17bnd-18 part with headers and no blank line before the next delimiter2 parts2 parts0 partsPart with headers and an empty body.MissingHeaderBodySeparator2 of 3
B18bnd-23 delimiters separated by bare CRno parts1 partno partsA bare CR is not a line break (see L1).BareCarriageReturnInHeader2 of 3
B19bnd-26 inner multipart reuses the outer boundarynestedouter swallows innernestedThe innermost open multipart is tried first, so the shared line closes the inner part.AmbiguousDuplicateBoundaryprov. (MK, Go)
B20bnd-19, bnd-20, bnd-21, bnd-22, bnd-24 empty part headers; blank line before delimiter; delimiter at end of file without line break; whitespace-only preamble and epilogue; 80-character boundarysamesamesameThe line break before a delimiter belongs to the delimiter; a part can have no headers (defaults apply); the boundary length limit is not enforced; whitespace-only preamble/epilogue counts as present but empty.BoundaryTooLong for over 70all agree

Line endings and raw octets

IDProbeMKPyGoResolutionDefectBasis
L1eol-02 whole message with CR-only line ends; eol-05 CR inside a header value; eol-09 CR CR LFCR-only: one long header; CR in value dropped from the decoded valueCR is a line breakCR kept in the valueA line ends at LF or CRLF only. A lone CR is data: it stays in the raw value and is dropped from the decoded value.BareCarriageReturnInHeaderprov. (MK, Go)
L2eol-01, eol-03, eol-04, eol-06, eol-07, eol-08 LF-only; mixed CRLF/LF; no final newline; headers end at EOFsamesamesameAccept any mix of LF and CRLF; a body's bytes are never changed.MixedLineEndings when mixedall agree
L3raw-01, raw-09 NUL, form feed and vertical tab in a header valuekeptkeptkeptKept in raw and decoded values.ControlCharacterInHeaderall agree
L4raw-02 NUL in a header nameheaderbody starts hereheaderAccepted as a header name; the raw bytes are kept.InvalidHeaderName2 of 3
L5raw-03, raw-07, raw-08 NUL in a body; 3000-character header; 20000-character body linekeptkeptkeptNo length limits below the configured resource limits; the defect LineTooLong marks lines over 998.LineTooLongall agree
L6raw-04, raw-05, raw-11, adr-24 raw 8-bit octets in headersUTF-8, else Latin-1U+FFFD for non-ASCIIinvalid bytes become U+FFFD; one address list failsA raw 8-bit header value is decoded as UTF-8 if valid, otherwise as Windows-1252 (RFC 6532 for the UTF-8 case).AmbiguousEightBitHeaderprov. (MK)
L7raw-10 UTF-8 byte-order mark before the first headerheader foundbody starts herethe BOM stays in the first name, which then goes unrecognisedThe BOM is ignored when the name is read (the raw bytes keep it), so the first field is found.InvalidHeaderNameprov. (MK)
L8raw-06, cte-14, ct-14 8-bit body declared 7bit or without charsetbytes keptbytes keptbytes keptBody bytes are never altered; charset choice is the encodings layer's.EightBitIn7BitBodyall agree

Header block

IDProbeMKPyGoResolutionDefectBasis
H1hdr-01, hdr-02, hdr-03 duplicate Content-Type, MIME-Version, Content-Transfer-EncodingfirstfirstfirstThe first occurrence is the one in force; all stay in the tree.DuplicateHeaderall agree
H2hdr-04, adr-25 duplicate From, To, Subject, DateFrom/To lists merge; all Subjects keptfirstfirstThe typed accessors use the first occurrence; every header stays in the tree.DuplicateHeader2 of 3
H3hdr-07 whitespace before the colon (Subject : x)headerbody starts hereheaderAccepted; the name is trimmed (RFC 5322 4.2 obsolete syntax).WhitespaceBeforeHeaderColon2 of 3
H4hdr-14 space inside a header nameheaderbody starts hereheaderAccepted as written; the name is kept (Latin-1) and InvalidHeaderName is reported.InvalidHeaderName2 of 3
H5hdr-05, hdr-06, hdr-09 a line without a colon in or after the headersline skipped (or parse error on the first line)body starts at the lineerrorThe first non-header line ends the header block and starts the body, so no byte is dropped.MissingHeaderBodySeparatorprov. (Py)
H6hdr-10 body line that looks like a header (Note: x) with no blank lineheaderheadererrorIt is a header.MissingHeaderBodySeparatorall agree where parsed
H7hdr-08 continuation line before the first headerparse errorignorederrorA line that starts with white space where a field should begin ends the header block and starts the body.HeaderContinuationWithoutHeaderprov.
H8hdr-11 From envelope line firstskippedskippedskippedKept as the entity's envelope line (its own reader event), not a header; the header block starts after it.noneall agree
H9hdr-12 blank lines before the first headerempty header blockempty header blockempty header blockAn empty header block; everything after is the body.NoHeadersall agree
H10hdr-16, hdr-15, hdr-21 folding with tabs, runs of spaces and blank continuation lineskeeps white spacekeeps white spacecollapses to one spaceUnfolding removes the line break only; interior white space stays.none2 of 3
H11hdr-25 trailing white space on a valuetrimmedkept in the decoded valuetrimmedTrimmed in the decoded value, kept in raw.none2 of 3
H12hdr-18 empty file (hdr-19, hdr-20: blank line only and headers only agree)errorempty messageerrorAn empty message; the parser never throws.EmptyMessageprov. (Py)
H13hdr-13, hdr-17, hdr-22, hdr-23, hdr-24, hdr-26 empty values, no MIME-Version, colons in values, odd case, 2000 headers, trace ordersamesamesameNames compare case-insensitively; a missing MIME-Version does not stop MIME parsing; order is preserved.MissingMimeVersionall agree
H14nest-13 message cut inside a header nameerrorbodyerrorThe partial line is kept as the start of the body.TruncatedHeaderprov. (Py)

Content-Type

IDProbeMKPyGoResolutionDefectBasis
C1ct-02, ct-03, ct-04, ct-09 text without subtype, text/, garbage, empty valueapplication/octet-streamtext/plain (text/ kept literal for ct-03)text/plain or textA value that is not type/subtype is treated as absent: text/plain; charset=us-ascii, raw header kept.InvalidContentType2 of 3
C2ct-05 parameter without =droppedcharset=""droppedDropped.InvalidParameter2 of 3
C3ct-07 comments in the valueone comment stays in the valuecomment stays in the typeremovedComments are removed (RFC 2045 5.1).noneprov. (Go)
C4ct-01, ct-06, ct-08, ct-10, ct-11, ct-12 odd case and spacing, unterminated quote, duplicate charset, ;;, unknown charset, raw UTF-8 in a parametersamesamesameLower-case types and parameter names; first duplicate wins; an unterminated quote runs to the end of the value; extra semicolons are ignored; unknown charsets are kept as written.UnterminatedQuote, DuplicateParameterall agree
C5ct-13 top-level message/rfc822nested messagenested messagenested messageParsed as a message.noneall agree

Content-Transfer-Encoding

IDProbeMKPyGoResolutionDefectBasis
E1cte-01, cte-04, cte-19, cte-17, cte-12, cte-10, cte-11 unknown, odd-case, empty, binary; QP lower-case hex, soft breaks, invalid escapessamesamesameUnknown, empty, 7bit, 8bit, binary: identity. Names are case-insensitive and trimmed. QP: invalid escapes pass through as written.UnknownTransferEncoding, InvalidQuotedPrintableall agree
E2cte-02, cte-03 composite type declared base64 or 8bitparts parsedparts parseddecodes first, no partsThe declared encoding is ignored for multipart/*: parts are parsed from the raw body.AmbiguousEncodedComposite2 of 3
E3cte-05 base64 with characters outside the alphabet9 bytes10 bytes3 bytesCharacters outside the alphabet are skipped, the rest is decoded as one stream.InvalidBase64prov. (Py)
E4cte-06 base64 without paddingdrops the last partial group (9)decodes it (11)drops it (9)The final partial group is decoded.MissingBase64Paddingprov. (Py; keeps data)
E5cte-07, cte-20 white space and line breaks inside base64skippedskippedstops at white spaceWhite space is skipped anywhere.none2 of 3
E6cte-08 data after = paddingcontinuescontinuesstopsDecoding continues after padding; each padded run is decoded on its own.DataAfterBase64Padding2 of 3
E7cte-13 trailing spaces before a QP line endkeptkeptstrippedKept (they are data in transit; RFC 2045 6.7 rule 3 only forbids producing them).none2 of 3
E8cte-15 message/rfc822 declared base64not decodeddecoded, parseddecoded, parsedStays a leaf whose body is the encoded bytes (decoding it first would break the byte-for-byte tree).AmbiguousEncodedCompositeprov. (MK)
E9cte-16 uuencodedecodeddecodednot decodedDecoded (x-uuencode, uuencode, x-uue, uue).none2 of 3
E10cte-18 comment after the name (base64 (note))decodeddecodednot decodedComments are removed first.none2 of 3

Parameters, encoded words and file names

IDProbeMKPyGoResolutionDefectBasis
N1enc-01, enc-30 RFC 2047 encoded word inside a quoted or token parameter valuedecodeddecoded (quoted only)literalDecoded, although RFC 2047 5 forbids it in parameters; every mail client does.AmbiguousEncodedWordInQuotedString2 of 3
N2enc-04 filename= and filename*= togetherextendedplainextendedThe extended (RFC 2231) value wins.AmbiguousParameterForms2 of 3
N3enc-02, enc-03, enc-09, enc-05 RFC 2231 charset, continuations, out-of-order continuationssamesamesameContinuations are joined by index before charset decoding (RFC 2231 3).noneall agree
N4enc-06 unknown charset in RFC 2231decoded as Latin-1U+FFFDparameter droppedFall back to Windows-1252.AmbiguousUnknownCharsetprov.
N5enc-07 filename*= without the charset quote marksLatin-1 mojibakeUTF-8parameter droppedPercent-decode, then UTF-8 if valid, else Windows-1252.MalformedExtendedParameterprov. (Py)
N6enc-08 continuations starting at *1joineddroppeddroppedJoined in index order.AmbiguousParameterSegmentsprov. (MK)
N7enc-14 two filename parametersfirstfirstparameter droppedFirst wins.DuplicateParameter2 of 3
N8enc-15 unquoted file name with a spacewhole valuecut at the spacerejectedThe value runs to ; or end of value.none (not raised)prov. (MK)
N9enc-10, enc-11 name versus filenamedisposition filename, else type namesamesameDisposition filename first, then Content-Type name.noneall agree
N10enc-12, enc-13 path separators, .., backslash escapesas writtenas writtenas writtenThe name is kept as written; making it safe to write to disk is the caller's or the export layer's job.UnsafeFilenameall agree
N11enc-16–enc-20, enc-22–enc-24, enc-28, enc-29 encoded words: Q and B, adjacent, mixed with text, unknown charset, over-long, split by a fold, empty, in Comments/KeywordssamesamesameRFC 2047 decoding; white space between adjacent encoded words is dropped.EncodedWordTooLong, AmbiguousSplitMultibyteall agree
N12enc-21 encoded word with invalid base64word removedword replaced by a spacekept literallyLeft in the text as written (RFC 2047 6.3 allows displaying it as ordinary text).InvalidEncodedWordBase64, InvalidEncodedWordQuotedPrintableprov. (Go)
N13enc-25 encoded word in a quoted display namedecodeddecodedliteralDecoded.AmbiguousEncodedWordInQuotedString2 of 3
N14enc-27 ISO-2022-JP encoded wordescape sequences leftdecodednot decodedDecoded.noneprov. (Py)

Addresses

IDProbeMKPyGoResolutionDefectBasis
A1adr-01, adr-02, adr-27 group syntax, empty groupflattenedflattenedflattenedGroups are kept as groups in the model; the shared view flattens members.noneall agree
A2adr-03 unquoted comma in a display nameone mailbox, name Doe, Johntwo: Doe (no domain) and Johnlist rejectedOne mailbox named Doe, John: a comma before < stays in the phrase.AmbiguousUnquotedCommaInPhraseprov. (MK)
A3adr-04 bare address after plain wordssecond mailbox droppedsecond mailbox with a wrong local partlist rejectedThe words become the display name and the address is kept.MissingAngleBracketsprov.
A4adr-05 name given as a trailing commentname from the commentno namename from the commentThe comment becomes the display name when there is none.none2 of 3
A5adr-06, adr-07, adr-20 nested comments, obsolete source route, spaces around @ and .samesamelist rejectedComments are stripped, the route is dropped, CFWS inside the address is removed.ObsoleteRoute2 of 2 that parse
A6adr-08 quoted local partquotes keptquotes keptunquotedThe address keeps its quoting as written; an unquoted form is available alongside.none2 of 3
A7adr-09 empty address <>mailbox droppedkept, address <>list rejectedA mailbox with an empty address and its name is kept.EmptyAddressprov.
A8adr-10 local part without a domainkeptkeptrejectedKept as written: john and mary@ are two mailboxes, each with MissingDomain (dropping a recipient loses data).MissingDomainprov.
A9adr-11, adr-13, adr-14, adr-16, adr-21, adr-22, adr-26, adr-28 IP literals; empty list elements; unclosed <; dotted names; 50 entries; escaped quoted names; several From; empty Cc/Bccsamesamemostly sameAccepted as written; empty elements are skipped; an unclosed < is closed at the end of the value.UnclosedAngleall agree where parsed
A10adr-12, adr-24 raw UTF-8 or Latin-1 in an addressname decoded; local part with UTF-8 droppedU+FFFDrejectedRFC 6532: UTF-8 in local part and domain is accepted; names follow L6.AmbiguousEightBitHeaderprov.
A11adr-15 unclosed quoted namename John, address keptwhole value as addressrejectedRecover as MimeKit does: the open quote ends at the <.UnterminatedQuoteprov. (MK)
A12adr-17 @ inside an unquoted display namename keptwrong addressrejectedThe address is the one in angle brackets.noneprov. (MK)
A13adr-18 two address-specs in one mailbox (<a@x><b@y>)bothfirst onlyrejectedBoth are kept as two mailboxes.AmbiguousAdjacentAngleAddrsprov. (MK)
A14adr-19 semicolon as a separatorfirst onlyfirst onlyrejectedOnly the first address is parsed; the remainder is kept as unparsed text in the defect.UnparsedAddressText2 of 2 that parse
A15adr-23 leading, trailing and doubled dots in a local part (.a@x, b.@x, c..d@x)address droppedkeptrejectedKept as written. These are common in old corporate mail, and dropping them loses recipients.ObsoleteLocalPartprov. (Py; in about 9% of messages in a 517,000-message public corpus, MimeKit drops such recipients)
A16gateway-rewritten addresses seen in the Enron corpus: <"a"@b.com@gw>, munn"."mary@gw, "d@x".@gw@gwMimeKit keeps the part up to the second @, or drops the address; Go rejects the listPython gives <> or a mangled addressKept as written, with InvalidAddressSyntax.InvalidAddressSyntaxprov. (no common reading)

Dates

All forms are read into an instant plus the written zone; the source text is kept with the value.

IDProbeMKPyGoResolutionDefectBasis
D1date-01–date-13, date-18, date-24, date-26, date-28, date-33 standard, no seconds, no weekday, wrong weekday, EST/GMT/UT, comment zone, lower-case month, extra white space, ±14:00samesamesameAccept; the weekday is ignored; named zones follow RFC 5322 4.3 (EST −5, PDT −7 …); with two Date headers the first is used.WeekdayMismatchall agree
D2date-34–date-37, date-03–date-05 two-digit years00–68 → 20xx; 69 → 2069; 70+ → 19xx00–68 → 20xx; 69–99 → 19xxsame as Py00–68 → 2000–2068, 69–99 → 1969–1999. All three libraries read 50 as 2050, which differs from RFC 5322 4.3 (50–99 → 19xx); OneMail follows the libraries.AmbiguousTwoDigitYear2 of 3 (69)
D3date-14 three-digit yearrejectedyear 101rejectedAdd 1900, as RFC 5322 4.3 says (101 → 2001).ThreeDigitYearprov. (RFC only)
D4date-10, date-12, date-21, date-25 -0000, military letters, GMT+2, missing zoneUTCUTC, zone unknown for -0000 and noneZ and none rejected; GMT+2 read as +2UTC with the zone marked unknown (RFC 5322 4.3: military zones are unreliable and read as -0000).UnknownTimeZone2 of 3
D5date-16 seconds = 6059rejectedrejectedAccept as a leap second, instant clamped to :59.LeapSecondprov. (MK)
D6date-17 zone +2400UTCrejected+24:00Out-of-range zones are read as unknown.InvalidTimeZoneprov.
D7date-19 2001-05-14 23:39:00parsed as UTCrejectedrejectedRejected (not RFC 5322); the header stays as text.InvalidDate2 of 3
D8date-22 hour 24wrong date (00:00 on the 14th)rejectedrejectedRejected.InvalidDate2 of 3
D9date-27 year 1899rejectedacceptedacceptedAccepted.none2 of 3
D10date-29 PDT−7−7rejectedAccepted (obsolete zone, RFC 5322 4.3).none2 of 3
D11date-30 fractional secondswrong time (00:00)rejectedacceptedFraction dropped.FractionalSecondsprov. (Go)
D12date-15, date-20, date-23, date-31, date-32 garbage, ISO with T…Z, 31 April, empty, day 0rejectedrejectedrejectedRejected; the value reads as absent and the header stays as text.InvalidDateall agree
D13date-38 four-digit year with leading zeros (0002), which a client that once wrote a two-digit year can produce20022002year 2Read as a two-digit year (rule D2).AmbiguousTwoDigitYear2 of 3

Nested messages and structure

IDProbeMKPyGoResolutionDefectBasis
M1nest-01, nest-02, nest-05, nest-17 message/rfc822 plain, multipart inside, 8bit, 30 levelsnestednestednestedParsed recursively, bounded by the nesting limit.noneall agree
M2nest-03, nest-04 message/rfc822 that is empty or has no headersmessage part without inner message, or body in an empty inner messageinner message with empty bodyno inner messageAn inner message exists whenever the part is message/rfc822; text without headers becomes its body.NoHeadersprov.
M3nest-06, nest-07 multipart/digest children without a typemessage/rfc822message/rfc822text/plainmessage/rfc822 (RFC 2046 5.1.5).none2 of 3
M4nest-08, nest-09 message/delivery-status, message/partialbytesgroups parsed as headersbytesBytes are kept; the status groups are parsed on demand.none2 of 3
M5nest-12 cut inside a part's headerpartial partpartial partno partThe part exists with its partial header.TruncatedHeader2 of 3
M6nest-14, nest-15, nest-16 10 and 200 nested multiparts, 3000 partsall parsedall parsedall parsedNone of the three limits depth (200) or part count (3000). OneMail's limits are on by default and are listed in the resource-limit notes.DepthLimitExceeded, EntityLimitExceededno oracle limit
M7nest-10, nest-11, nest-18, nest-19, nest-20, nest-21 alternative/related, signed, QP HTML, inline and unknown dispositions, a lying Content-Length, mbox-like trailer linessamesamesameStructure as written; Content-Length is advisory and ignored; an unknown disposition is kept as written.noneall agree

Defect names in this table that are not raised, and why

The decoders and the tree keep what they repair without a defect where the repair is the specified reading, or report it by another route.

Name in the tableWhere it is reported instead
InvalidBase64, MissingBase64Padding, DataAfterBase64Padding, InvalidQuotedPrintable (E1, E3, E4, E6)Body decoding is lazy, so the decoders report TransferIssues flags on the decoding stream, not tree defects.
DuplicateHeader (H1, H2)Not raised: all copies stay in the tree; only a repeated Content-Type or Content-Transfer-Encoding is a defect (AmbiguousDuplicateContentType, AmbiguousDuplicateTransferEncoding).
NoHeaders, EmptyMessage, TruncatedHeader (H9, H12, H14, M2)Not raised: an empty or cut-off message is a valid empty tree; MissingHeaderBodySeparator marks a header block that ran into the end of input.
MissingMimeVersion (H13)Not raised: a missing MIME-Version does not change parsing.
LineTooLong, EightBitIn7BitBody (L5, L8)Not raised: body bytes are never judged.
EnvelopeLine (H8)Not a defect: the reader emits an EnvelopeLine event.
UnsafeFilename (N10)Not raised: a file name is kept as written.
EncodedMessagePart (E8)Reported as AmbiguousEncodedComposite.
UnquotedParameterSpecials (B12, N8)Not raised: an unquoted value runs to ;.

Differences the differential run reports, and how each is explained

tools/oracles/explained.tsv maps every difference between OneMail and each oracle to an entry above, to a documented normalisation of the comparison (NORM-*) or to a named limitation of that oracle (LIMIT-*). A difference with no line fails the run.

Calendar and contacts (ICS, vCard)

Observed black-box on Python icalendar 7.3.0 (Py-ical) and vobject 0.9.9 (Py-vobj), over the public corpus and our probes (tools/calendar/oracles). Ical.Net 5.2.3 and FolkerKinzel.VCards 8.2.0 expose typed models only, so they settle value questions, not line syntax. The tree always keeps every byte (untouched input writes back identical); the rows below govern the decoded view and the defects. Every row is our default, marked provisional where fewer than two oracles agree.

IDCasePy-icalPy-vobjResolutionDefectBasis
K1Line ends: LF only, lone CRaccepts LFaccepts LFCRLF, LF and lone CR all end a line. Reported once per file, at the first.BareLineFeed, BareCarriageReturnall agree for LF; CR prov.
K2Fold with two whitespace characters after the breakone removed (ab cd)one removedExactly one space or tab is removed (RFC 5545 §3.1); the rest is content.noneall agree
K3Fold inside a multi-byte UTF-8 sequencen/an/aLines are unfolded as bytes before decoding, so the character is whole.noneRFC 5545 §3.1
K4Line with no colonskippedparse errorKept in place as a malformed line (written back as read); not a property.MissingColonprov. (lossless)
K5Space before the colon (SUMMARY :x)acceptsparse errorAccepted; the name is trimmed.WhitespaceAroundDelimiterprov. (1 of 2)
K6Lower-case names and begin:vcalendaracceptsacceptsNames and component names compare ASCII case-insensitively; case is kept as written.noneall agree
K7END: that does not match the open componenterrorerrorCloses up to the matching open component; the components in between are left unclosed. With no match, the line is kept as malformed.MissingEnd, UnmatchedEndprov. (both reject; ours must not)
K8RFC 6868 caret sequences in a parameter value (^n, ^^, ^')decodedliteralDecoded.AmbiguousCaretEncodingprov. (1 of 2)
K9The same parameter twice (ROLE=A,B;ROLE=C)last winsmergedBoth are kept in order as written; readers that want one value take the first.nonelossless
K10Unknown text escape (\q, \", \:)\q and \" kept, \: becomes :\q and \: kept, \" becomes "The backslash and the character stay as written. Only \\ \; \, \n \N are resolved (vCard 2.1: \\ \;).noneprov.; \q agrees
K11Escapes in X- propertiesrawunescapedUnescaped like any TEXT value (RFC 5545 §3.8.8.2 gives X- properties the TEXT type).noneprov. (1 of 2)
K12Invalid UTF-8 textU+FFFDU+FFFDThe line is read as Windows-1252, the reading clients use for legacy files, unless CHARSET names a charset we can decode. The source bytes are untouched.AmbiguousNonUtf8Textprov. (our default)
K13vCard 2.1 CHARSET we cannot decoden/an/aRead as UTF-8, or Windows-1252 when that is invalid.UnknownCharsetour default
K14vCard 2.1 quoted-printable soft break (= at line end)n/ajoinedJoined whenever the line is marked QUOTED-PRINTABLE, in vCard and ICS alike.QuotedPrintableSoftBreakprov.
K15vCard 2.1 base64 data on unindented lines, ended by an empty linen/ajoinedJoined when the line carries ENCODING=BASE64/B/bare BASE64 and the next line holds only base64 characters. The ending empty line belongs to the property.Base64ContinuationWithoutFoldprov.
K16vCard 2.1 bare parameters (TEL;WORK;VOICE)n/adropped on some inputsKept as bare parameters; the typed view lists them as types together with TYPE= values.nonelossless
K17Comma lists with an empty element (ADR street 5,)n/aelement droppedThe empty element is dropped in the typed view; the raw value keeps it.noneprov.
K18Properties inside VCALENDAR after BEGIN:VCARD junk, stray lines around componentsn/an/aKept as items of the document root, in order.MissingColon, UnmatchedEndlossless
K19A date-time with a TZID and a Z suffixUTCUTCUTC wins; the TZID stays a parameter.noneall agree
K20Nesting beyond 32 levelsunlimitedunlimitedDeeper BEGIN/END lines are kept as ordinary properties.DepthLimitExceededlimit
CV1Unknown escape in a typed vCard 3.0/4.0 value (\:, \", \q)vobject drops the backslash of \" only; FolkerKinzel drops that of \: onlyThe typed view (not the raw value) drops the backslash and keeps the character, for every character after \\ \; \, \n \N. vCard 2.1 unescapes \\ and \; only.noneprov. (oracles disagree)
CV2TEL;TYPE=VOICE,MSG,WORK imported to a contactn/an/aEvery matching rule fires: Business and Other (MS-OXVCARD 2.1.3.4.1 example, literally). A fax alone fills Primary Fax, pref fills Primary, no type fills Other.nonespec example; no client observed
CV3ADR with no recognised typen/an/aBusiness and Other; postal, dom, intl, parcel fill Other; a fourth address is dropped.AdditionalValueNotMappedspec example; prov.
CV4A second or later TEL/EMAIL of a kind with one slotn/an/aThe second work (home) number goes to Business2 (Home2); later ones are reported, not stored. The first three emails fill the slots in card order.AdditionalValueNotMapped, and on export PhoneSlotChanged, EmailSlotChangedspec; prov.
CV5BDAY as a partial date or with a timen/an/aPidTagBirthday is the date at 11:59 UTC, PidLidBirthdayLocal the date at 0:00 (Outlook's own pair); the time of day is dropped; a date without a year is not stored.TimeOfDayDropped, PartialDatespec 2.1.3.2.5; prov.
CV6An empty text property in a contact (PidTagDepartmentName = "")n/an/aTreated as absent: no vCard property is written and the report says so.EmptyValueNotWrittenagent default
CV7CR CR LF in a vCardtolerated by vobjectn/aTwo line ends (K1); a folded value that contains it is cut, as the reader does for any line end. A reader-level decision is open.noneprov.

JSON and XML forms (jCal, jCard, xCal, xCard)

Observed black-box on ical.js 2.2.1, Python icalendar 7.3.0, biweekly 0.6.8 and ez-vcard 0.12.2 (tools/calendar/oracles/forms); the rows say how we resolve what the RFCs leave open or what the oracles do differently. Reasons and the full class list: jcal-xcal.md and explained.tsv.

IDCaseOraclesResolutionDefect or lossBasis
J1Value that does not fit its type (DTSTAMP:19970901T1300Z, TRIGGER:19980403T120000)ical.js, biweekly print a guessed, often garbled typed valueCarried as unknown with the raw text; the declared VALUE stays as a parameter, so the text writes back as readValueNotRepresentablelossless
J2DTSTART:19971102 with no VALUE=DATEical.js, biweekly: date-time with a garbled timeA date (RFC 7265 Appendix B.1 shows it)AmbiguousValueTypeRFC example
J3ENCODING=BASE64 without VALUE=BINARY on ATTACH or an extension property; VALUE=BINARY without ENCODINGn/aBoth are binary; ENCODING=BASE64 is added in the second caseAmbiguousValueTypeRFC 5545 §3.3.1
J4TZ:-0500ical.js textutc-offset (RFC 7095 Appendix B.1); VALUE is not written backnoneRFC example
J5Parameter with no value (ATTENDEE;MAILTO:)ical.js, icalendar drop itOne empty value, in JSON and XMLnonelossless
J6Parameter given twiceical.js, icalendar keep the lastValues merged in orderDuplicateParameterMergedlossless
J7Same JSON object key twicen/aBoth keptDuplicateKeylossless
J8TYPE, PID, SORT-AS values, also a quoted "a,b"ez-vcard keeps one stringSplit into arrays (RFC 7095 §3.3.1.1)noneRFC
J9Inline vCard base64 (PHOTO, KEY)ical.js keeps binary and ENCODINGdata: URI; media type from TYPE, else application/octet-streamInlineBinaryAsDataUriRFC 7095 §3.5, RFC 2397
J10Backslash escapes in a vCard URI value (URL:http\://x)ical.js and ez-vcard unescapeUnescaped (\\ \, \; \:); a backslash cannot occur in a URInone2 of 2
J11vCard 2.1/3.0 data in xCardez-vcard writes 4.0 everywhere4.0 forms for GEO, TYPE=pref; VERSION dropped. jCard keeps the source version and type namesVersionNotCarried, VersionSpecificFormRFC 6351
J12Group prefix on iCalendar dataical.js keeps GROUP.NAME as the nameSplit into the group parameterGroupNotCarried in XMLRFC 7095 §3.3.1.2
J13Property, parameter, component or type name that is not an XML namebiweekly, ez-vcard drop or renameLeft out of xCal/xCard (JSON carries it)PropertyNotCarried, ParameterNotCarried, ComponentNotCarried, TypeNameNotCarriedXML 1.0
J14Character XML cannot hold (U+000C, NUL, lone surrogate)n/aU+FFFDInvalidXmlCharacterXML 1.0
J15vCard 2.1 quoted-printable =0D=0Aez-vcard keeps CR LF in LABELLine feed in the form; the text writer uses quoted-printable again and never folds a line after =noneRFC 6350 §3.4
J16Components outside a VCALENDAR (bare VEVENT examples)biweekly writes calendars onlyCarried at the top level of the formnonelossless
J17Durations, UTC offsets, TZID values, repeated FREEBUSY periods, URI scheme case, local times with TZIDicalendar, biweekly normalise (P7W to P49D, -00:00 to +00:00, local to UTC, MAILTO: to mailto:)Kept as writtennonelossless

Recurrence and time zones

Ours against recurring-ical-events 3.8.2 on icalendar 7.3.0 (rec), python-dateutil 2.9.0.post0 rrule through our glue (du) and Ical.Net 5.2.3 (ical), black-box, over every recurring component of the public calendar corpus plus our generated fixtures: 2,384 components, each in a fixed window (tools/calendar/run_expand_diff.sh; method in docs/recurrence.md). A component is identical when the sorted start lists are equal, differing otherwise, not expanded when the oracle errors or times out (330 libical components have an invalid DTSTART that every oracle rejects). Counts are components. Each difference gets exactly one id, the first cause that applies; the id is the entry below. Basis as in the other sections: all agree, 2 of 3, or provisional (our default).

OracleComparedIdenticalDifferingExplainedUnexplainedNot expanded
rec2,3722,233139139012 (2 broken property, 8 timeout, 2 value error)
du2,3782,23814014006 (1 broken property, 3 type error, 2 value error)
ical2,1881,4097797790196 (130 timeout, 33 argument, 30 out of range, 3 other)
IDCaseOraclesResolutionDefectDifferences (rec / du / ical)Basis
R1BYWEEKNO with no BYDAY, BYMONTHDAY or BYYEARDAYrec and du: every day of the week; Ical.Net: the weekday of DTSTARTThe weekday of DTSTART (RFC 5545 §3.3.10: a finer part that is absent takes DTSTART's value).none32 / 32 / 0prov. (RFC; Ical.Net agrees)
R2A part the §3.3.10 table marks N/A for the frequency (BYWEEKNO with MONTHLY, BYYEARDAY with DAILY/WEEKLY/MONTHLY, BYMONTHDAY with WEEKLY, BYSETPOS alone, ordinal BYDAY where §3.3.10 forbids it)the three disagree with each other and with us on some; 3 of the Python differences are one occurrence at the COUNT boundary of WEEKLY;BYMONTHDAY;BYSETPOS;COUNTThe part filters, never expands.none102 / 102 / 449prov. (the table gives no result)
R3DTSTART does not fit its own rule (§3.8.5.3: undefined)rec and du include DTSTART; Ical.Net omits itDTSTART is always the first instance and counts toward COUNT.AmbiguousUnsynchronizedStart0 / 0 / 75 (rec and du give our list on all 94 such components, apart from those with an R1 or R2 part)2 of 3
R4UNTIL of another value type than DTSTART (DATE vs date-time, UTC vs floating)the oracles differ with each otherA UTC UNTIL is an instant for zoned and UTC series and a wall-clock value for floating ones; a DATE UNTIL covers its whole day.AmbiguousUntilType2 / 1 / 1prov. (§3.3.10 requires equal types)
R5Wall times in a clock change: the skipped hour and the repeated hourrec and du: skipped shifted, repeated first; Ical.Net differsSkipped: shifted by the offset before the gap (RFC 5545 §3.3.5), or dropped with SkippedTimePolicy.Skip. Repeated: the first.AmbiguousSkippedLocalTime, AmbiguousRepeatedLocalTime0 / 0 / 57 (rec and du give our list on all 85 components that hit a clock change; ical on 26 of 83)2 of 3
R6EXDATE as a DATE on a date-time seriesnot in the corpusThe whole day is removed.AmbiguousExdateType0 / 0 / 0 (unit test only)prov. (clients treat a date as a day)
R7WEEKLY with BYSETPOS in the first, partial weekrec and du position inside the part of the week after DTSTARTPositions count in the whole week (§3.3.10), so an earlier instant of the first week takes a position and is then dropped as before DTSTART.none1 / 1 / 0prov. (RFC)
Z1A TZID named both by a VTIMEZONE in the file and by the runtime's databasenot compared (the oracles resolve zones in their own way; the zone fixtures use one or the other)The VTIMEZONE wins.nonen/aprov.
Z2A vendor-prefixed TZID (/softwarestudio.org/Olson_20011030_5/America/New_York)not comparedThe longest tail of up to three segments that names a zone.AmbiguousZoneNamen/aprov.
Z3A Windows zone name with no VTIMEZONEnot compared; instants checked for 5 names (W. Europe, Eastern, AUS Eastern, New Zealand, Pacific)The CLDR default-territory IANA zone.AmbiguousWindowsZonen/aprov. (CLDR)
Z4A TZID that resolves nowherenot comparedThe values stay floating; no instant is claimed.UnknownTimeZonen/aprov.
Z5An IANA TZID on a host whose runtime knows only Windows zone ids (Windows without ICU, or with InvariantGlobalization)not comparedThe runtime lookup is repeated under the Windows zone CLDR assigns the IANA name (default-territory row first, then any territory). The zone keeps its IANA Id and ZoneSource.Iana. Windows zone data has fewer historic rules than IANA (US dates before 2007 follow the current rules), so such instants can differ from the same file read on Linux or macOSnonen/aour default (CLDR)

Limits of one oracle (not ambiguities; no entry in the resolution column):

IDOracleBehaviourDifferences
L-oracle-ignores-exrulerec, duEXRULE is not applied1 / 1 / n/a
L-du-glue-has-no-overridesduour glue expands the master only, so RECURRENCE-ID components are not applied0 / 3 / n/a
L-ical-duplicate-instantsicalan instant produced by several BYxxx combinations is returned several times; the set of instants is oursn/a / n/a / 10
L-ical-window-wall-clockicalthe window start is compared with the wall time of a zoned instant, so in zones east of UTC an instant before the window gets through; without those instants the lists are equaln/a / n/a / 102
L-ical-weekno-intervalicalYEARLY with BYWEEKNO and INTERVAL above 1 lands off the interval grid (2026, 2029, 2031 for INTERVAL=2)n/a / n/a / 8
L-outvoted-by-the-other-twoanythe oracle differs and at least one other oracle gives our list1 / 0 / 77

The AmbiguousUntilType and AmbiguousExdateType resolutions are the ones most likely to change when Outlook is observed (the Outlook oracle, before Outlook writing). Counts come from one run of the 2,384 components; tools/calendar/run_expand_diff.sh repeats it. Zone behaviour is checked in the DST subset instead: 108 dst.ics components in America/New_York, Europe/Berlin, Australia/Sydney and Pacific/Auckland (both transitions each) and 9 with Windows names give our list on 108/108 and 9/9 for rec and du.

Mailbox files (mbox, Maildir, EMLX)

Black-box runs of Python 3.13.17 mailbox (mbox and Maildir) and MimeKit 4.18.2's mbox mode (MimeParser with MimeFormat.Mbox; tools/oracles/mailbox-driver) over small probes (tools/mailbox/gen_mboxes.py, gen_stores.py) and the dovecot-crlf file (3,240 From lines). Go has no mbox reader. Neither oracle is a mail client, so every row that is not "agree" is provisional. The reader's behaviour is in mailboxes.md. MK = MimeKit, Py = Python.

IDCasePyMKResolutionDefectBasis
S1A line starting From that is not an envelope line (From the desk of x, From maildir-storage.c, line 325:), unquoted in the bodysplitssplits, then often fails on the next "message"Only a line with a sender and a date after From (RFC 4155) starts a message; the rest is text. MboxFromLineMode.Any splits like both oracles. dovecot-crlf has three such lines: Py 3,240 messages, OneMail 3,237, MK raises at message 556AmbiguousFromLineprov.; real-world file
S2A Content-Length header (mboxcl/mboxcl2, or in any file)ignoredignoredUsed when the file's variant is unknown, mboxcl or mboxcl2 and the length ends exactly at an envelope line or the end of the file (a blank line may sit between). Otherwise ignored. mboxo and mboxrd ignore it. Two different values: the firstContentLengthMismatch, AmbiguousContentLengthprov.
S3A CRLF file: the empty line after a messagekeeps the CR LF in the messagethe message runs to the next envelope lineThe empty line (LF or CRLF) before the next envelope line or the end of the file is the separator, not message textnoneprov.
S4A body that the oracle's split leaves unparseablen/araises FormatExceptionThe tree is OneMail's job, not the splitter's: a message is never rejectednoneoracle limit
S5A line that is only From (no sender or date)splitsdoes not splitNot an envelope line in strict mode; a delimiter in Any modeAmbiguousFromLine1 of 2
S6mboxcl2 (unquoted bodies, Content-Length only)splits inside messages that hold envelope-looking linessameRead correctly through Content-Length; OneMail writes it only when asked for the variantnoneoracle limit
S7Bytes before the first envelope lineskippedskippedKept as the store's preamble, so entries plus preamble tile the filePreambleBeforeFirstMessageall agree
S8An envelope line with no empty line before itsplitssplitsSplits; the previous message keeps its last lineMissingSeparatorLineall agree
S9Unquoting a stored body when the variant is not knownn/an/amboxrd's rule (every >+From loses one >); an entry that a Content-Length delimited is returned as stored, because mboxcl and mboxcl2 differnoneour default
S10Maildir info flags not in ASCII order or repeated (:2,SF)read as writtenn/aRead as written, reported; written back sortedUnsortedFlagsour default
S11Maildir file in newsubdir new, no flagsn/aFlags from the name, plus Recentnoneall agree
S12Mail's flags word and date-sent/date-received reals in EMLX plistsn/an/aBits 0, 1, 2, 4, 6, 8 map to seen, deleted, answered, flagged, draft, forwarded; the raw word is always kept. A real is Unix seconds from 1e9, otherwise seconds since 2001-01-01. No real Mail store has been inspected, so this is a default to check against onenoneour default
S13Courier courierimapkeywords/:listn/an/aKeyword names, an empty line, then unique:n n rows with zero-based positions. Written from Courier's published description of the directory; the line format inside the files is not in that document, so this is unchecked against CourierMalformedControlFileour default
S14DBX: the length a message record states (field 0x11) differs from the chain'sn/an/aundbx 0.21 writes the chain's bytes whatever the record states (3 records: stated 10 vs 3,024 bytes, 99,999 vs 63, 0 vs 723). The chain wins; the notes warn the field "can be wrong"AmbiguousDbxDeclaredLengthundbx 0.21 on generated stores, and on a real store (NIST CFReDS Hacking Case): 33 of the 34 news posts that have text state a length 5 to 380 bytes different from the chain's, and undbx's file equals the chain in all 34
S15DBX: the character set of record strings (subject, sender, folder names)n/an/awindows-1252. The notes name none; the strings are in the system ANSI code page of the machine that wrote them, which the file does not recordnoneour default; not observable through undbx
S16DBX: a message file in the store folder that Folders.dbx does not namen/an/aRead, as a folder named after the fileDbxUnlistedFileundbx 0.21 also extracts every file in the directory; our default for the folder name
S17DBX: a message record with no text chain (a header that was never downloaded)n/an/aAn empty message in the folder, flagged. undbx 0.21 writes a 0-byte file for it (its -i option skips empty messages)AmbiguousDbxNoMessageTextundbx 0.21 on a generated store; the real store above has 2,019 header-only records among 2,053 (news headers), and undbx writes each as a 0-byte file
S18DBX: field 0x09 of a folder recordn/an/aRead as the special-folder kind (1 Inbox, 2 Outbox, 3 Sent, 4 Deleted, 5 Drafts). The notes describe the field twice (a per-folder index and the kind)nonea real Folders.dbx (35,527 records) has the kinds on Inbox 1, Outbox 2, Sent Items 3, Deleted Items 4, Drafts 5 and absent on the rest

| S19 | The character before the Maildir info section (unique:2,FS or unique;2,FS) | Py mailbox writes and reads : only | n/a | Read: the first :, else the first ; that is followed by 2,. Written: MaildirStoreOptions.InfoSeparator, default ; on Windows (where a file name cannot hold :) and : elsewhere; isync's InfoDelimiter documents the same default. No Windows mail client has been observed | none | our default (isync documentation) |

Reports, authentication results and MHTML

Probes are testdata/reports/… (cases.tsv describes each); MK is MimeKit 4.18.2's public API, Py is Python 3.13 email (plus urllib.parse.urljoin for MH6–MH8). Go has no report or MHTML support and is not compared. All entries are provisional (no client observed). Method: reports-and-mhtml.md.

IDProbeMKPyResolutionDefectBasis
RP1dsn-11, dsn-12, dsn-32 extra blank lines between or around blocksskippedeach is an empty blockRuns of blank lines are skipped; one defect per run.AmbiguousReportBlankLinesprov. (MK, RFC)
RP2MDN/ARF body with a leading blank line, or several blocksempty first blockempty first blockThe first block is empty; further blocks are kept as extra blocks.AmbiguousReportExtraBlocksMK and Py agree
RP3dsn-13 a line without a colon in a blockkept as a field named by the lineblock ends thereSkipped, reported.ReportMalformedLineprov. (no two agree)
RP4dsn-34 a name with a spacekept as a fieldblock ends thereKept under its text before the colon.InvalidHeaderNameprov.
RP5dsn-18 whitespace before the colontrimmedfield lostTrimmed, reported.WhitespaceBeforeHeaderColonprov. (MK, RFC 5322 obs-)
RP6dsn-14 a fold before any fieldignoredn/aSkipped, reported.ReportMalformedLineprov.
RP7dsn-26 a lone CR in a valuekeptline split at CRStays data.BareCarriageReturnInHeaderMK, same as the header reader
RP8dsn-21 an empty delivery-status bodyno reportone empty blockA report with no groups.ReportMissingGroupprov.
RP9a whitespace-only line inside a blockfoldfoldA fold, not a block end.noneall agree
RP10repeated single-use field (dsn-15 Action twice)both kept in ordern/aBoth stay in the block; the typed accessor reads the first.AmbiguousReportDuplicateFieldprov. (lossless)
AR1auth-26, auth-08 first token is name=… (no authserv-id)no idn/aNo authserv-id; the token starts the first result.AuthenticationResultsMissingIdMK
AR2auth-08 a property with no ptype (action=none)droppedn/aKept with an empty ptype.AmbiguousAuthenticationResultsPropertyprov. (lossless)
AR3auth-11,12,14,15,16,18,19,27 broken fields (bad version, empty clause, garbage, none beside results, unterminated quote or comment)the whole field failsn/aThe clauses that can be read are kept, each repair is a defect.AuthenticationResultsInvalidVersion, InvalidAuthenticationResults, AmbiguousAuthenticationResultsNone, UnterminatedQuotelossless
AR4auth-10 quoted authserv-id, property value or reason (valid per RFC 8601 §2.2)the field failsn/aParsed.noneRFC
RS1spf-06,07,10 unknown result, words that are not pairs, bad client-ipno oracleno oracleUnknown result kept as SpfResult.Unknown with the text; bad pairs skipped.InvalidReceivedSpfprov. (RFC 7208 §9.1 only)
MH1mht-01 cid: references, with brackets, %40, or caseno matchn/a (no MHTML)Matches Content-ID only, brackets stripped, percent-decoded, exact then ASCII case-insensitive.noneRFC 2557 §8.3
MH2mht-02,06,08,09,10,11,15,19 base from the part's or an enclosing entity's Content-Locationbase only from the multipart's Content-Basen/aBase order of RFC 2557 §5 as written in the method note.noneRFC
MH3mht-03 dot segments, no baseno matchn/aRFC 3986 §5.2.4 on thismessage:/.noneRFC
MH4mht-07 percent escapes, space, case of %hexexact onlyn/aExact first, then normalised.noneprov.
MH5mht-17 mid:no matchn/aNeeds a matching Message-ID and Content-ID.noneRFC 2392
MH6urljoin with base thismessage:/…n/areturns the reference unchanged (scheme not hierarchical for it)Resolved per RFC 3986.noneRFC
MH7urljoin of http:g against http: basen/anon-strict: http://a/b/c/gStrict: http:g (RFC 3986 §5.2.2).noneRFC
MH8urljoin with an upper-case scheme or host in the referencen/alower-cases the schemeLeft as written; Normalize lower-cases when matching.nonelossless
MH9two parts with the same location or content id (mht-06)firstn/aFirst wins.AmbiguousMhtmlDuplicateLocation, AmbiguousMhtmlDuplicateContentIdprov.
MH10start names no part (mht-05)n/an/aThe first part is the root.MhtmlStartNotFoundRFC 2387

Documented oracle limits (not ambiguities; in explained.tsv as LIMIT-*): Python does not read message/global-delivery-status or transfer-encoded report parts as reports.

The shared read view (bodies and attachments)

Checked against Python 3.13 email with policy.default (get_body/iter_attachments) over the probes, the Apache James test messages, the dovecot mbox and a sample of the CMU Enron maildir; tools/oracles/viewcheck.sh reproduces the run. The view's choices below are provisional: no mail client has been observed for them yet. The view raises no defects of its own; it chooses between readings of a valid tree. The tree always keeps every part (docs/shared-view.md).

IDCasePython emailResolution
V1Which parts are attachmentsImmediate children of the top multipart, except the first text/plain, text/html, multipart/related and multipart/alternative; a nested multipart is one attachment; a message that is not multipart has noneA leaf part or message/* part that is not the chosen text or HTML body, found by walking every multipart, in document order. A message that is a single leaf part (not text-body) is its own attachment.
V2Body inside multipart/signed or a nested multipart/mixedget_body does not look inside multipart/signed; the nested part's text is listed as an attachmentThe body is searched in the signed content (the first child) and in nested mixed parts, in order; the signature part is an attachment.
V3Body in multipart/alternative with several text partsThe last alternative that matchesThe last alternative that matches (RFC 2046 5.1.4). The other text renderings are not attachments unless marked Content-Disposition: attachment.
V4Inline partsParts of a multipart/related are not listedListed, with IsInline set when the part is not marked attachment and has a Content-ID, so callers can resolve cid: references.
V5A text part with Content-Disposition: attachmentAn attachmentAn attachment; never the body. A text part with a file name but no disposition is still a body when it is the first match (the same as Python).
V6Text body without a usable charsetus-ascii, bytes above 127 replacedUTF-8 when the bytes are valid UTF-8, otherwise Windows-1252 (N5's rule).
V7Address groups in From/To/…Group name keptFlattened to the members, the group name dropped (A1). The tree keeps the group.
V8Attachment file nameget_filename(): disposition filename, else type name, with the parameter rules of N1–N8The same order (N9). Names are kept as written, not made safe for a file system (N10); no name is null, never an empty string.
V9A message with a TNEF part (winmail.dat)Python: winmail.dat is one attachment. MimeKit (black box, MimeKitTnef): MimeMessage.Attachments lists winmail.dat; TnefPart.ExtractAttachments returns the body (an unnamed RTF, HTML or text entry) and the attachmentsA part that qualifies under MS-OXCMAIL 2.2.3.8 (the conversion's rule, conversion.md) is not listed; the attachments of its TNEF take its place in document order, RtfBody and the bodies the MIME side lacks come from it, and the part stays reachable (MimeMailItem.TnefParts, the tree). A part that does not qualify stays an attachment. Outlook shows a TNEF message this way (2.2.3.8); the attachments equal MimeKit's extraction, tnefparse's, tnef's and ytnef's on 64 wrapped messages (252 checks, 0 unexplained). Prov.: no client observed.

MAPI and MSG

Observed on the Apache Tika and POI MSG files (55) and our generated ones, against extract-msg 0.55.0 as a black-box oracle. No Outlook has been observed yet, so each resolution is provisional.

IDCaseOther readerResolutionDefectBasis
X1Same property tag twice in one property streamn/aBoth kept in the bag; lookups return the lastAmbiguousDuplicatePropertyprov.
X2Value stream with no property entry (POI fuzz-minimised file)ignores itRead as a property; the stream is the dataAmbiguousOrphanValueStreamprov., lossless
X3String type differs from the store support maskfollows the entry's typeThe entry's own type decidesAmbiguousStringTypeMS-OXMSG 2.4.2
X48-bit strings, no PidTagMessageCodepagefixed Latin defaultLocale's ANSI page, then Internet code page, then enclosing message, then 1252AmbiguousCodePage36 corpus strings differ, all in files with only a locale or Internet code page
X5String size field: stream length or stream plus terminatorn/aBoth acceptednoneANSI files store the NUL in the stream

PST/OST messaging and LTP

Observed on the Tika testPST files, the CFReDS OST and our generated files against pffexport 20180714 as a black-box oracle. No Outlook has been observed yet, so each resolution is provisional. L-entries are the lists-tables-properties choices of docs/pst-ltp.md.

IDCaseOther readerResolutionDefectBasis
P1Message has attachment subnodes but no attachment tableno attachmentsThe attachment subnodes found are usedAmbiguousAttachmentTableMissingprov., lossless
P2Embedded-message object property holds a bare subnode NID, not the (NID, size) structuren/aThe NID is usedAmbiguousObjectReferenceMS-PST 2.3.3.5 states the structure; prov.
P3Table row identifier differs from the NID the object is stored undertrusts the nodeThe node decides; the mismatch is reportedRowIdMismatchprov.
L1Rows per block of a subnode row matrixn/aTaken from the first block's own size, which equals the formula for 8 KiB blocksnoneprov.; real files agree
L2BTH key that does not increasen/aSkipped, so a node reached twice gives no duplicatesnoneprov.
L3Property or cell of an unknown typen/aRead as an HNID in a PC; kept inline when one, two or eight bytesnoneprov.
L4Positions in a heap's fill-level bitmap blocks (HNBITMAPHDR)n/aAllocations may not start before the block's header (12, 66 or 136... bytes, 2 elsewhere)noneexercised on our files only

RTF bodies (compressed RTF, de-encapsulation)

Probes are the 45 RTF bodies of the public Tika and POI .msg files, the specification examples (testdata/rtf), and 3,000 damaged streams. Oracles: compressed-rtf 1.0.7 (decompress, compress) and RTFDE 0.1.2.2 (de-encapsulation), run as released packages; the message's own PidTagHtml/PidTagBody is a third reference. Where the specification is explicit the entry says so. Entries without an observed Outlook-side result are provisional. Method: rtf.md.

IDCaseOracle / own bodyResolutionDefectBasis
RT1\par and \line in the extracted textRTFDE writes LF; own plain-text bodies use CRLF (4 of 4)CRLF (MS-OXRTFEX 2.2.3.2 and 2.2.3.3)nonespec; seen in 31 HTML and 4 text results vs RTFDE
RT2A literal no-break space (\'a0) in visible textRTFDE writes &nbsp;The character U+00A0. A string has no code page to lose it innoneprov.; 4 files vs RTFDE
RT3The message's own PidTagHtml carries <meta http-equiv=Content-Type …> that the RTF's HTML lacksown bodyNot part of the encapsulated HTML; nothing to resolve. Callers who want the stored body read PidTagHtmlnone5 of 6 own HTML bodies
RT4Own PidTagHtml is a re-serialised copy of the RTF's HTML (tag case, dropped </tbody>)own bodyThe encapsulated HTML is returned as written; same text (checked by words)none1 of 6 (bug66335); RTFDE agrees with OneMail on it
RT5\_spec table: soft hyphen in \htmltag; RTF 1.9.1: non-breaking hyphenU+00AD inside \htmltag, U+2011 elsewherenonespec
RT6\htmlrtf active when an \htmltag group starts; code page of \htmltag textspec: content MUST be copied; default code page MUST be usedCopied whatever the state; decoded with the document's default code page, not the font'snonespec
RT7\uN and \lquote, \~… inside \htmltagspec table gives HTML text (&#x20AC;, &lsquo;); 2.2.3.2 says "Unicode characters"The characters. A .NET string needs no entities, and both denote the same textnoneprov.
RT8How many characters \ucN skipsRTF 1.9.1 says "characters", with \'hh as oneText bytes and \'hh escapes count; control words do not; { or } cancels the rest; \uc is group-scopednoneprov.
RT9Code page of text bytesspec: "the code page of the current font"Font \cpgN, else \fcharsetN (0 = 1252 even under another \ansicpg; 1 = document), else \ansicpg/\ansi/\mac/\pc/\pca, else 1252; before any \f the \deff fontnonespec example uses \fcharset0 under \ansicpg1251 for exactly this; the corpus Cyrillic body decodes to its own body
RT10\htmlrtf in a \fromtext documentspec defines it for HTML onlyIgnoredAmbiguousHtmlRtfInPlainTextprov.
RT11A code page with no decoder (symbol, platform gap)spec silentThe document's default code pageAmbiguousUnknownCodePageprov.
RT12Detection: \fromhtml/\fromhtml0, text before the control word, an eleventh token, CR/LF before the first tokenspec 2.2.3.1 (SHOULD)Only \fromhtml1 or \fromtext within ten group-start/control-word tokens after {\rtf; line breaks are skipped as in RTFNotEncapsulated, NotRtfspec
RT13Destinationsspec: skip standard destinations without visible text, keep \fonttbl; ignorable \* groups other than \htmltag skippedA fixed list of body-less destinations is skipped; an unknown destination that is not ignorable is read as text (it cannot be told from a formatting group)noneprov.
RT14RAWSIZE far above the data (a decompression bomb)compressed-rtf decodes the data and ignores the claimRefused before allocating: above 256 MiB, or above 20 × the input lengthOutputLimitExceeded, ExpansionRatioExceeded123 of 3,000 damaged streams differ for this reason only
RT15COMPSIZE disagrees with the bytes presentcompressed-rtf ignores itContents are bytes 16 to COMPSIZE + 4 when that is within the input, else to the end; the CRC covers exactly thoseCompressedSizeMismatchspec 2.1.3.1.1; prov. for the extra bytes
RT16Data ends before the end markercompressed-rtf returns what it decodedThe decoded bytes, with a defectMissingEndMarkerprov.
RT17RAWSIZE disagrees with the output; MELA contentsspec: MELA CRC not validated, size may be read either wayOutput is what the data holds (growing up to the limit), with a defect; MELA is the contents as storedRawSizeMismatchspec 2.2.3.1
RT18Compressing nothingspec note: Outlook writes 02 00 0D 0001 0C F0 (end marker only) is written; 02 00 0D 00 is read as one NUL bytenonespec

TNEF

Observed on the public Apache POI and Tika TNEF files (6 distinct) and our generated ones, against tnefparse 1.4.0, tnef 1.4.18 and ytnef 2.1.2 as black-box oracles. No Outlook has been observed, so each resolution is provisional unless it follows the specification's own text.

IDCaseOther readerResolutionDefectBasis
T1A message-level legacy attribute appears twicen/aThe last one is used; both stay in the attribute layerAmbiguousDuplicateAttributeprov., lossless
T2A legacy attribute and the encapsulated property differ (e.g. attDateRecd vs PidTagMessageDeliveryTime in winmail-sample1.dat)n/aThe encapsulated property wins; the legacy value is not addedAmbiguousLegacyConflictMS-OXTNEF 2.1.3; seen in 1 corpus file
T3PtypNull value in a property listtnef and ytnef reject it (0 and 4 bytes alike)Takes no bytesnoneMS-OXCDATA 2.11.1; prov.
T4Legacy message class names (IPM.Microsoft Mail.Note)n/aMapped to the modern classAmbiguousMessageClassMS-OXTNEF 2.3.3.4; the spec sample and corpus files use it
T5Wrong checksum on the message class attributen/aIgnored, the data is usedAmbiguousMessageClassChecksumMS-OXTNEF 2.3.3.4
T6An embedded-message or OLE attachment also has attAttachDatan/aThe attribute is ignoredAmbiguousAttachDataIgnoredMS-OXTNEF 2.3.3.7
T7attOemCodepage differs from PidTagMessageCodepagen/aattOemCodepage decides how 8-bit strings are readAmbiguousCodePageMS-OXTNEF 2.3.3.2 order; no corpus file has the conflict, so no client observation
T8attOwner with no meeting classn/aMaps to the PidTagSentRepresenting groupAmbiguousOwnerprov.

MS-OXCMAIL conversion (MAPI to MIME and back)

The rules are in conversion.md. No Outlook has been observed yet; the two oracles are black-box MimeKit 4.18.2 and Python 3.13 reading what we write, and the corpus is the Tika and POI MSG files and the Tika and CFReDS PST/OST files. Each resolution is provisional until Outlook can be run.

IDCaseWhat the spec saysResolutionDefect or report entryBasis
OX1An EX sender or recipient has no SMTP address property2.1.3.1.7: look it up in the address book directoryWithout a directory: the same distinguished name elsewhere in the message, then the matching transport From/Sender field (one mailbox), then a unique display name in the transport headers. Never derived from the distinguished nameAddressUnresolved when nothing resolveson the 45 MSG and 25 PST/OST items, 16 of 28 EX senders and 50 of 50 EX recipients resolve; the 12 senders that do not (all MSG) have no SMTP property and no transport From field; 9 PST senders have no address at all
OX2Transport headers and properties disagree2.2.3.2.28 keeps the received block; 2.1.3.2 builds fields from propertiesThe received field stays when it agrees; the properties win when it does not (a user edit makes the properties newer)TransportHeaderOverridden, NamedPropertyValueSetAside2 fields and 5 named-property values across the 70 corpus items
OX3Default values (PidTagSensitivity 0, false receipt flags)2.1.3.2.6 and 2.1.3.1.5: no fieldNo field; reading back leaves the property absentDefaultValueNotWritten2.1.3.2.6, 2.1.3.1.5; 151 entries over the 70 exported items
OX4Importance: Normal2.1.3.2.5: MAY be omittedWritten whenever the property exists (0, 1, 2)none2.1.3.2.5 (MAY omit); writing it keeps the property on the way back, 0 silent differences in the 70-item property round trip
OX5RTF that encapsulates nothing2.1.3.3: not coveredA text/rtf part when there is no HTML, else not writtenRtfCarriedAsTextRtf, RtfRedundant2.1.3.3 is silent; 5 and 13 entries over the 70 exported items; no client observed
OX6Case of a PS_INTERNET_HEADERS name2.2.3.2.29: "the header name"As the message wrote it. Stores differ (Exchange stores lower case, Outlook as written), so comparisons are case-insensitivenone2.2.3.2.29; the MSG and PST corpora hold both spellings, so the property round trip compares case-insensitively
OX7Missing or invalid Date on import2.2.3.2.2: the current timeNo PidTagClientSubmitTime; a conversion must be reproducible. Export falls back to the delivery timeDateFromDeliveryTimedeviates from 2.2.3.2.2 on purpose (reproducible output); 4 MSG items and 1 Mime4j message hit it
OX8Non-ASCII display name2.1.3.1: RFC 2047The whole name as one run of encoded words (never leaving "@" or "(" in plain text beside them)noneMimeKit and Python read it; a plain-text remainder broke an address in a first attempt
OX9Which MIME part is the body2.2.3.3.2The part the shared view chooses (shared-view.md), a named text part includednone2.2.3.3.2; body hashes equal the view's on 70 of 70 exported items in both oracles
OX10Composite media type on flat attachment datanot coveredapplication/octet-streamCompositeTypeOnFlatDataspec silent; a multipart/signed without a boundary broke the parse in MimeKit and Python; 1 MSG item and 6 Mime4j messages hit it
OX11Bcc2.1.3.1.1.2Not written unless asked; a Bcc field on import is Bcc recipientsBccNotWritten2.1.3.1.1.2; 2 MSG items, 2 Mime4j and 1,272 Enron-sample messages hit it
OX12multipart/signed on import2.2.3.4Class IPM.Note.SMIME.MultipartSigned; content and signature become a body and an attachmentSignatureNotVerifiable2.2.3.4; 213 of 1,430 Dovecot messages; the Outlook form, while signed MIME stays bytes elsewhere
OX13Subject prefix2.2.3.2.6.1: one to three characters other than colon, blank or digit, then colon and blanksAs stated, with an empty PidTagSubjectPrefix when there is nonenone2.2.3.2.6.1; 23 PST items carry a prefix marker the export removes (SubjectPrefixMarkerRemoved), none differ in the oracles
OX14Embedded message namesnot coveredThe attachment's own name only; the subject is not invented as a namenonespec silent; attachment names of all 70 exported items equal the source in both oracles
OX15Reply-To entries2.1.3.1.2: names ignored when the counts differOnly one-off SMTP entries are written; address-book entries need a directoryReplyToUnresolved2.1.3.1.2; 9 entries over the 70 exported items
OX16A TNEF property that a header field also sets (subject, sender, reply-to)2.2.3.8 step 2.1: ignore attributes that set a property already set by the headersPer property, and a group set by any header (subject, sender, sent-representing sender, reply-to) is the headers' whole, so two sources are never mixed in one sender or subject. Ignored values that differ are reportedTnefPropertyIgnored2.2.3.8; 152 of 1,596 properties over 52 wrapped corpus messages (none in the bare shape, which has no headers); prov. for groups
OX17Plain text in MIME, richer body in the TNEF2.2: the plain text MUST NOT be preferred to a richer TNEF bodyA body property is added when the MIME side has none of that kind; the MIME text stays PidTagBody; PidTagNativeBody and PidTagInternetCodepage are the TNEF's (the author's) unless the HTML is the MIME side's, then the richest bodyTnefPropertyIgnored for a differing body of the same kind2.2; prov. for the codepage rule
OX18Message classthe class is a property; header processing gives IPM.NoteThe default IPM.Note gives way to the TNEF's IPM.* class; a class the MIME structure gave (report, S/MIME) staysTnefMessageClassUsedspec silent on defaults; needed for meeting requests carried in TNEF; prov.
OX19Recipients in the headers and in the TNEF2.2.3.8 step 2.1The headers' recipients win; the TNEF table is used only when the headers gave noneTnefRecipientIgnored2.2.3.8; prov.
OX20A TNEF stream ending in CR LF2.2.3.8 step 3: SHOULD ignore the stream (a SHOULD, no client observed)Unpacked, as tnef 1.4.18, ytnef 2.1.2, tnefparse 1.4.0 and MimeKit 4.18.2 all do (4 of 4 oracles on both bug52400 files); reported, not hidden. Prov. until a client is observedAmbiguousTnefTrailingCrLf (view defect), TnefTrailingCrLfIgnored (Internal loss entry)2.2.3.8 step 3; the two bug52400 corpus files; the earlier choice (keep as attachment) was reversed on the oracle evidence
OX21Correlation key vs X-MS-TNEF-Correlator2.2.3.8 step 4: a mismatch means pure MIME; the field is spelled X-MS-TNEF-Correlator (2.1.3.2.26) and X-MS-Tnef-Correlation (2.2.3.8)Both spellings are read; the binary key matches when its bytes (trailing NULs and blanks removed) equal the field as ASCII text or as hexadecimal digits; either one missing is a matchTnefNotUsed2.1.3.2.26, 2.2.3.8; none of the corpus files has a key; prov.
OX22Writing a TNEF message2.1.3.7; the key is "unique"The key of the message's own PidTagTnefCorrelationKey when it has one, else the first 16 bytes of SHA-256 of Message-ID (or subject) and submit time in hex; no top-level recipient table in the TNEF; an empty text/plain when there is no text bodyTnefWriterLimit, TnefPlainTextNotRendered2.1.3.7, 2.1.3.1.1; 45 MSG and 25 PST/OST corpus items through OxcMailTool rt-tnef: 10,496 of 11,142 properties exact, 227 the same text under another string type, the rest in the loss report, 0 silent (pure MIME: 6,453 exact, 4,290 reported)