--- title: "Reading and repair rules" url: https://recursiveai.org/onepdf/docs/repair/ description: "How OnePDF opens a file, and what it does when the file is damaged. Every repair adds a PdfDiagnostic (severity Warning, with object number and offset…" version: "1.11.0" source: https://github.com/recursive-ai-oss/onepdf-csharp/blob/b90019f86b3d08820577502740a10c66befdd989/docs/repair.md --- # Reading and repair rules How OnePDF opens a file, and what it does when the file is damaged. Every repair adds a `PdfDiagnostic` (severity Warning, with object number and offset where known) to `PdfDocument.Diagnostics`. With `PdfOpenOptions.Strict` each of them throws `PdfFormatException` instead. Informational notes (Info) never throw. ## The fallback chain 1. **Trust the cross-reference data.** Find the last `startxref` in the final 8 KiB, read the section it names, then follow `/Prev` back to the first section. Sections are tables (`xref` ... `trailer`) or cross-reference streams (ISO 32000-2 7.5.8). In a hybrid file the table's `/XRefStm` supplies entries for objects the table lists as free (7.5.8.4). Each section is one revision (`PdfDocument.Revisions`, oldest first). 2. **Fix individual objects by scanning.** When an entry's offset does not lead to `n g obj` with the expected number, the file is scanned once (shared by all later lookups) for object headers and the object is read from where it is found. Diagnostic: "the cross-reference offset ... does not point at the object". An object that cannot be found reads as null (severity Error). 3. **Rebuild.** If the xref data cannot be read at all (no `startxref`, no section at the offset, a damaged or truncated table or trailer, a `/Prev` loop, a subsection claiming more entries than the file can hold) or the trailer's `/Root` does not lead to a dictionary, the whole file is scanned and the cross-reference data is replaced by what the scan finds. Diagnostic: "the cross-reference data is unusable (...); rebuilt by scanning the file for objects". Limit violations (`PdfLimitException`) are never repaired: they propagate. ## Rules in detail - **Header.** `%PDF-` is looked for in the first 1024 bytes. Bytes before it are ignored (Info). A missing header is a warning. Offsets in `startxref` and the xref data are tried as absolute file positions, then relative to the header; the first that lands on a section is used for the whole file (Info). - **Table entries.** Entries are read as tokens, not fixed 20-byte lines, so 19-byte and CR-only lines work. An entry of type `n` with offset 0 is free. A first subsection that starts at 1 although its first entry is the free head (`0000000000 65535 f`) is renumbered from 0 (warning). - **Trailer.** The newest trailer is used; `Root`, `Info`, `ID` and `Encrypt` it lacks are taken from older trailers. Each revision keeps its own trailer. - **Stream length** (7.3.8.2). `/Length` (direct, or indirect) is trusted only if `endstream` follows the data after at most 16 white-space bytes. Otherwise the data ends at the next `endstream`, else `endobj`, else the end of the file, less one end-of-line (CRLF, LF or CR). Diagnostic: "the stream's /Length ... is wrong". A `/Length` that is indirect and cannot be resolved takes the same path. The length of an object stream or xref stream is never read from inside an object stream. - **Stream keyword.** `stream` may be followed by spaces before its end-of-line; CR alone is accepted. - **Missing `endobj`.** Any token other than `endobj` after an object's value is a warning; reading continues. - **Unterminated containers.** An array or dictionary cut off by `endobj`, `stream`, `obj`, `xref`, `trailer`, `startxref` or the end of data is closed there (warning). A stray `]` or `>>` is ignored. A dictionary key that is not a name skips that token; a key without a value is dropped. Null values are dropped (a missing key and a null value are the same, 7.3.7). - **Numbers.** Leading signs, a second decimal point, and a minus after digits are tolerated (`--5` is -5, `1.2.3` is 1.2, `5-3` is 5). Integers beyond 64 bits become reals. - **Strings and names.** Unterminated strings run to the end of data. Bad `#xx` escapes in names stay literal. Name bytes that are not valid UTF-8 are read as Latin-1. - **Object streams.** The entry's index is checked against the stream's header; a wrong index is found by number (warning). `/N` and `/First` are validated against the decoded size before anything is allocated. When `/N` or `/First` is missing, negative, too large, or `/First` lies inside the header, the header is salvaged (warning "invalid /N or /First"): the leading run of `number offset` integer pairs is the header (at most `/N` pairs when `/N` is sane, else as many as the data allows), and `/First` is taken to be where that run ends. A header that stops early keeps the pairs read so far and the rest of the stream's objects read as null (warning). An object stream that is missing or not a stream makes its objects read as null (warning). Strict mode throws for each of these. - **References.** A reference to a missing, free or null object resolves to null. A chain of references to references longer than `PdfLimits.MaxRecursionDepth` (or a loop) reads as null (warning). A load that needs itself is a `PdfFormatException`. - **Page tree.** `Pages` (and `PageCount`) walk `/Pages` iteratively once; the inheritable attributes `Resources`, `MediaBox`, `CropBox` and `Rotate` come from the nearest ancestor that has them. A page without a usable `MediaBox` is US Letter (warning); a `CropBox` is clipped to the `MediaBox`; a `Rotate` that is not a multiple of 90 is 0 (warning). Nesting beyond `MaxNestingDepth` is skipped (warning). A node reached twice is skipped (warning); a kid that is not a dictionary is skipped (warning); a node without `/Kids` is a page unless `/Type` is `/Pages`. - **External streams.** A stream with `/F` reads as empty (warning). Nothing outside the document is ever opened. - **Encryption.** `/Encrypt` is read as soon as the cross-reference data is, before the catalog or any object stream is trusted (a file rebuilt by scanning authenticates before its object streams are opened). `PdfOpenOptions.Password` is tried as user then owner password; the empty password is always tried. A wrong or missing password is `PdfPasswordException`; an unsupported or malformed handler is `PdfEncryptionException`. Everything parsed before authentication is dropped from the cache. Strings are decrypted when their object is parsed, stream data when it is read (before filters), under the object number and generation of the containing indirect object. Exempt: the `/Encrypt` dictionary, cross-reference streams, strings inside object streams (the stream is encrypted as a whole) and the trailer `/ID`. A `Crypt` filter names the crypt filter to use (`Identity` leaves the data alone) and is not part of the decode chain. A metadata stream is left in the clear when `/EncryptMetadata` is false. Damaged ciphertext (bad padding, partial block) decrypts as far as it goes with a warning. ## The rebuild scan One pass over the file, byte by byte, no whole-file buffer: - `obj` preceded by ` ` and not by `end`, followed by a delimiter or white space, starts an object. The object is parsed; for a stream the body is skipped using a trusted `/Length` or the next `endstream`, so bytes inside stream data (an embedded PDF, say) are not taken for objects. - The last definition of a number wins. `trailer` followed by a dictionary is a trailer candidate; so is the dictionary of any `/Type /XRef` stream. - The trailer used is the last candidate with a `/Root`; `Root`, `Info`, `ID` and `Encrypt` are each taken from the latest candidate that has them. With no candidate, the last `/Type /Catalog` object is used (warning). - Objects inside object streams are added after the scan. An object stream counts as defined where it sits in the file, so a later direct definition of the same number wins over its contents and a later object stream wins over an earlier definition. An object stream that cannot be read is skipped (warning). - The result is a single revision. More than `PdfLimits.MaxObjects` objects raises `PdfLimitException`. ## Limits checked while reading | Limit | Checked | |---|---| | `MaxObjects` | declared subsection and `/Index` counts before reading entries; headers during the rebuild scan; page-tree nodes | | `MaxNestingDepth` | on entering each array or dictionary level (explicit stack, no recursion) | | `MaxDecodedStreamBytes`, `MaxExpansionRatio` | by the filters while they run; raw stored size before reading it | | `MaxRecursionDepth` | reference-to-reference chains | ## The typed layer Name trees, number trees, outlines, form field trees and action chains are walked iteratively with a visited set, so a node reached twice (a cycle) is skipped with one warning, and with `MaxObjects` / `MaxNestingDepth` bounds. A malformed tree key, a non-dictionary page or annotation entry, a destination that leads nowhere, an unparseable date or an XMP packet that is not well-formed XML never throws: the value reads as null or empty (XMP keeps its raw bytes). DTDs in XMP are refused, never expanded. Stream-backed text (embedded files, XFA, XMP) is size-checked against the limits before it is read; over the limit the item is ignored with a warning, except `PdfEmbeddedFile.ReadContent`, which throws `PdfLimitException` because the caller asked for it.