Conformance: onnx/backend/test

Generated by tools/conformance.sh (it runs OnnxSweep conformance over the frozen ONNX 1.22.0 backend-test data, then tools/conformance_table.py). Nothing here is edited by hand. The method is at the end.

Summary

A gap is an unsupported-op or unsupported-type case: the library says so, loudly, when the session is created or first run. A fail is a case that ran (or threw) and is wrong: a defect. A known-diff is a failing case whose difference is understood and listed in the runner with its reason (Resize align_corners follows ONNX Runtime; the DFT inverse cases' expected data is noisier than their tolerance). A not-exact case, in exact mode only, uses an operator that has no exact kernel yet; its result is not computed.

Operators (node cases)

Opsets lists the opset versions of the cases that pass. exact is the pass count of the same cases in exact mode; not-exact counts those that need an operator without an exact kernel. Gap causes are the most frequent reasons.

Model cases (simple, pytorch-converted, pytorch-operator)

105 further model cases pass in both modes and are not listed.

Operators with no kernel

Operators named by unsupported-op cases, with the number of cases that need them (a case can need several).

Every case that does not pass

One line per case: case (opset), fast status, exact status, reason.

<details><summary>List</summary>

</details>

Method

• Data. The frozen ONNX 1.22.0 onnx/backend/test/data directories node, simple, pytorch-converted and pytorch-operator, fetched by tools/fetch_backend_test.py at a pinned SHA-256 (the data is never committed). ONNX 1.23 and later no longer ship frozen node cases, so cases for operators newer than that release are not in the data; see the gaps above.
• Run. Each case's model.onnx is imported and a Session is created in the mode under test (Fast or Exact, one intra-op thread, default optimization level). Every test_data_set_* is fed in and every output compared with the recorded output.
• Tolerance. Floating-point outputs (float16, bfloat16, float32, float64) pass when each element satisfies |got - want| <= 1e-7 + 1e-3 * |want| or both are NaN or equal (this includes infinities); integer and boolean outputs must be equal; shapes and element types must be equal.
• Status. unsupported-op: no kernel for the operator at that opset, or the operator rejects an attribute value or rank it does not handle. unsupported-type: an element type or value kind the library does not have (string, 4-bit, 8-bit float, map; sequences and optionals of tensors are supported). not-exact: exact mode only, an operator without an exact kernel. fail: anything else that is not a pass.
• Reading the gaps. Cases at opset 6 (the legacy pytorch models and a few operator versions) are below the supported range, which starts at opset 7. The rest are listed by cause in the operator table: element types the tensor layer does not have, the ai.onnx.ml and training domains, random-number operators and training modes, and the newest preview operators. known-diff cases are explained in the runner (ConformanceRunner.KnownDifferences) and count as neither pass nor fail.
• Attribution. A node case counts under its only operator; a case with several operators (the _expanded function bodies) counts under the longest operator name its case name starts with.