Accessibility testing with Playwright in C# · onehtml cookbook

Open the page with Playwright, run axe-core on it with Deque.AxeCore.Playwright, and check the same DOM against WCAG 2.2 AA with onehtml by passing Page.ContentAsync() to its checker. The program below tests a sample sign-up form with four problems.

Terminal

dotnet new console -o PlaywrightA11y && cd PlaywrightA11y
dotnet add package Microsoft.Playwright --version 1.64.1
dotnet add package Deque.AxeCore.Playwright --version 4.14.0
dotnet add package RecursiveAI.OneHtml.Accessibility.Validation --version 1.5.1
curl -O https://recursiveai.org/onehtml/samples/signup.html

Put this in Program.cs and run dotnet run:

using Deque.AxeCore.Commons;
using Deque.AxeCore.Playwright;
using Microsoft.Playwright;
using RecursiveAI.OneHtml;
using RecursiveAI.OneHtml.Accessibility.Validation;
using RecursiveAI.OneHtml.Accessibility.Validation.Catalogue;
 
// Downloads Chromium the first time; does nothing once it is there
Microsoft.Playwright.Program.Main(["install", "chromium"]);
using IPlaywright playwright = await Playwright.CreateAsync();
await using IBrowser browser = await playwright.Chromium.LaunchAsync();
IPage page = await browser.NewPageAsync();
// In your suite, this is your app's URL
await page.GotoAsync(new Uri(Path.GetFullPath("signup.html")).AbsoluteUri);
 
// axe-core runs inside the page
AxeResult axe = await page.RunAxe();
foreach (AxeResultItem v in axe.Violations)
{
    Console.WriteLine($"axe      {v.Id} ({v.Impact}): {v.Nodes.Length} node(s)");
}
 
// onehtml checks the DOM as the browser holds it now, in this process
Document doc = Document.Parse(await page.ContentAsync());
var options = new CheckOptions { Profile = ValidationProfile.Wcag22AA };
CheckReport report = new Checker().Check(doc, options);
foreach (RuleResult failure in report.WithOutcome(Outcome.Failed))
{
    Console.WriteLine($"onehtml  {failure.RuleId} at {failure.Locator}");
    Console.WriteLine($"         {failure.Message}");
}
Console.WriteLine($"Needs review: {report.Count(Outcome.NeedsReview)}, " +
    $"needs render: {report.Count(Outcome.NeedsRender)}");
 
// SARIF 2.1.0 for GitHub code scanning or any SARIF viewer
var export = new ExportOptions { Url = "signup.html", Indented = true };
File.WriteAllText("signup.sarif", report.ToSarifJson(export));

Text

axe      button-name (critical): 1 node(s)
axe      color-contrast (serious): 1 node(s)
axe      image-alt (critical): 1 node(s)
axe      label (critical): 1 node(s)
onehtml  button-name at button
         The button has no accessible name.
onehtml  color-contrast at form > p:nth-of-type(2)
         The text has a contrast ratio of 2.84:1; 4.50:1 is needed.
onehtml  image-alt at img
         The image has no accessible name.
onehtml  label at input[name="email"]
         The form field has no accessible name.
Needs review: 6, needs render: 3

The first run also prints Chromium’s download progress. Both engines find the same four problems: the logo has no alt, the email field has no label, the gray hint text is too light, and the submit button holds only a hidden icon.

axe-core runs in the page and uses the browser’s layout. RunAxe also takes a Locator for part of a page, and AxeRunOptions limits it to tags such as wcag2aa (README). Deque.AxeCore.Playwright 4.14.0 bundles axe-core 4.14 and targets .NET Standard 2.1; it is MIT-licensed and depends on Deque.AxeCore.Commons, which is MPL-2.0 (NuGet).
onehtml runs nothing inside the page, so it can’t interfere with the app under test. Each failure carries the WCAG criteria and a locator unique in the page, and the report exports SARIF, native JSON or axe-compatible JSON (ToAxeJson()). The SARIF line numbers refer to the DOM Playwright serialized, not to your source file.
NeedsRender results are rules that need layout, here target size. The next section supplies Chrome’s layout so they are decided too.

In an NUnit, MSTest or xUnit suite, use Playwright’s test base class (for NUnit, PageTest from Microsoft.Playwright.NUnit) and assert that axe.Violations is empty and report.Count(Outcome.Failed) is 0.

Decide layout rules from Chrome’s DOMSnapshot

Target size needs the boxes the browser drew. Playwright can open a Chrome DevTools Protocol session (Chromium only), and DOMSnapshot.captureSnapshot returns every node’s layout. onehtml maps that snapshot onto its parse of the same DOM:

using System.Text.Json;
using Microsoft.Playwright;
using RecursiveAI.OneHtml;
using RecursiveAI.OneHtml.Accessibility.Rendered;
using RecursiveAI.OneHtml.Accessibility.Validation;
using RecursiveAI.OneHtml.Accessibility.Validation.Catalogue;
 
// The computed styles Chrome adds to each layout box in the snapshot
string[] styles = ["display", "visibility", "position", "float",
    "overflow-x", "overflow-y", "border-top-style", "border-top-width"];
 
using IPlaywright playwright = await Playwright.CreateAsync();
await using IBrowser browser = await playwright.Chromium.LaunchAsync();
IPage page = await browser.NewPageAsync();
await page.SetViewportSizeAsync(1280, 800);
await page.GotoAsync(new Uri(Path.GetFullPath("signup.html")).AbsoluteUri);
 
// Serialize the DOM, then capture Chrome's layout of that same DOM
Document doc = Document.Parse(await page.ContentAsync());
ICDPSession cdp = await page.Context.NewCDPSessionAsync(page);
JsonElement? capture = await cdp.SendAsync("DOMSnapshot.captureSnapshot", new()
{
    ["computedStyles"] = styles,
    ["includePaintOrder"] = true,
    ["includeDOMRects"] = true,
    ["includeBlendedBackgroundColors"] = true,
    ["includeTextColorOpacities"] = true,
});
 
byte[] json = JsonSerializer.SerializeToUtf8Bytes(capture!.Value);
DomSnapshot snapshot = DomSnapshot.Parse(json, styles);
var viewport = new DomSnapshotMapOptions
{
    ViewportWidth = 1280,
    ViewportHeight = 800,
};
DomSnapshotMapResult mapped = DomSnapshotAdapter.Map(snapshot, doc, viewport);
Console.WriteLine(mapped.Succeeded ? "Snapshot mapped" : $"Refused: {mapped.Refusal}");
 
var options = new CheckOptions
{
    Profile = ValidationProfile.Wcag22AA,
    Facts = mapped.Facts,
};
CheckReport report = new Checker().Check(doc, options);
Console.WriteLine($"Failed: {report.Count(Outcome.Failed)}, " +
    $"needs review: {report.Count(Outcome.NeedsReview)}, " +
    $"needs render: {report.Count(Outcome.NeedsRender)}");

Text

Snapshot mapped
Failed: 4, needs review: 6, needs render: 0

With layout, the target-size results are decided and “needs render” drops to 0.

Limits

One capture, one state. The snapshot describes one viewport at one moment; hover and focus states that script drives aren’t seen. DomSnapshotAdapter.Map refuses a snapshot of a different DOM (Succeeded is false and Refusal says why), so let the page settle before capturing.
The snapshot format isn’t a stable Chrome API. Pin the browser version in CI. Rendered facts documents the accepted format.
Automation finds part of the problems. A person still has to judge the NeedsReview results, such as whether a label describes its field.

Pages that don’t need a browser, such as server-rendered templates, static sites and HTML email, can be checked without Playwright: see Automated accessibility testing in C#.