Measured over 1134 official W3C test cases

# Marlo checks accessibility. Here is our receipt.

yes, the embarrassing lines too

Marlo checks your pages against the official ACT rule corpus. Then it turns the same corpus on itself and commits the score, including the part where it loses.

start here! [Try it on any site](https://trymarlo.pages.dev/try/) [See the numbers](https://trymarlo.pages.dev/accuracy/) [Read the source](https://github.com/KarthikSubramanian07/Marlo)

marlo scan

$ marlo scan apps/demo/checkout.html apps/demo/settings.html

apps/demo/checkout.html
  static renderer  dom, script

  NOT EXAMINED  2 rules need layout and paint, which this renderer does not provide.
                09o5cg afw4f7
                This is not a pass. Use --renderer browser to evaluate them.

  ▲▲▲ critical c487ae  Link has non-empty accessible name
      WCAG 1.1.1, 2.4.4, 2.4.9, 4.1.2  marlo precision 1.00 over 28 official test cases
      disagreement: htmlcs says passed
      img\[src$="product-8842.jpg"\] (not located in source)
        INVARIANT axe-core failed this element and the routed engine did not report it.
        Images must have alternative text
      a (not located in source)
        This link has no accessible name. A screen reader announces "link" with no destination, which is unusable in a links list.

  ▲▲  serious  afw4f7  Text has minimum contrast
      WCAG 1.4.3, 1.4.6  alfa precision 0.83 over 28 official test cases
      INVARIANT alfa reported a failure the routed engine did not, so Marlo may not report clean.
      disagreement: alfa says failed
      disagreement: htmlcs says passed
      xpath /html\[1\]/body\[1\]/main\[1\]/p\[1\]/text()\[1\] (not located in source)
        The highest possible contrast of the text is 3.45:1 which is below the required contrast of 4.5:1

  \------------------------------------------------------------------------
  39 findings   0 fixed   0 flagged   2 not evaluated   0 crashed
  coverage: 41 of 94 published ACT rules
  one-directional invariant enforced on 3 rule(s): a peer reported a failure the routed engine did not.

Our false positive rate 7.7% What `marlo scan` reports, measured as a whole over 41 rules. It reports a failure whenever any of its four engines does, and that costs precision. Marlo's own rule engine alone measures 4.4%. No vendor prints either number, which is exactly why they go near the top. [The whole table, per rule](https://trymarlo.pages.dev/accuracy/)

Alfabest 1.5% false positives

axe-core2nd 1.9% false positives

Marlo3rd 4.4% false positives

HTML CodeSniffer4th 9.4% false positives

Above the findings, not in a footnote

In the output above, two rules need CSS layout. That renderer has none, so it says so first, before anything it did find. "No contrast problems found" and "contrast not examined" are different sentences and Marlo will not blur them.

A disagreement, on the record

The router sends that ARIA rule to axe-core. Marlo's own rule said it could not tell. axe-core said failed, so the failure goes through and the dissent gets printed underneath it.

That is real stdout from `marlo scan` over the deliberately broken pages in `apps/demo`. Trimmed for length, not edited: the test suite checks every line of it against the committed golden file.

The problem

## Detection is not the hard part. Nobody publishing an error rate is.

Lighthouse hands out 100s to pages a screen reader cannot get through. A large AI site builder shipped an unusable product while its own bundled checker reported perfection. The best funded vendor in this category advertises "19x more critical issues" and has never said more than what, or how it counted.

None of that is a detection failure. It is a market where **a claim nobody can check is free to make.**

So Marlo does the tedious version instead. Four engines, every official W3C test case, precision and recall per rule per engine, table committed to the repository, regenerated on every push. When a number gets worse the build goes red and the changelog says which one.

An improvement fails the build too, if nobody committed the new table. A figure that quietly got better is a figure the README is now wrong about.

Official test cases, vendored and hashed 1134 Across 91 rules that publish them, with a SHA-256 per file. `pnpm corpus:verify` fails on a changed byte. If a test case could be edited, an inconvenient result could be fixed by changing the question instead of the answer.

Engine and rule pairs measured 204 Every engine against every rule it maps, through one code path. Regenerated by `pnpm calibrate` in CI. Both a regression and an uncommitted improvement fail the build.

How it works

## Six stages, and stage five can override the other five

Marlo does not pile four engines' output into one list and call the pile thorough. Every failure any engine finds is reported once, by the engine the table routes the rule to, with that engine's evidence and measured accuracy, and one stage forbids the routing from silencing anyone.

1.  1
    
    Render
    
    A Node DOM by default: no browser, no network, milliseconds. Playwright if you want layout, and it is opt-in rather than assumed.
    
    happy-dom, or chromium
    

3.  2
    
    Run four engines
    
    axe-core, Alfa, HTML CodeSniffer, and Marlo's own 41 rules. All in-process. None of them can see the others, and that isolation is a dependency rule rather than a habit.
    
    nobody gets a longer timeout
    

5.  3
    
    Normalise to ACT identifiers
    
    Four vocabularies collapse into one. The axe-core mapping was derived by measurement over the corpus. The other two are documentation matches, and every entry in both says so.
    
    partial mappings are labelled partial
    

7.  4
    
    Route from the table
    
    Whichever engine measured best on a rule reports that rule's findings, with its locator and confidence. Routing chooses provenance, not what is detected: held out, it is no more accurate than the best single engine. 18 go to axe-core, 22 to Marlo, 11 to Alfa, and 0 to nobody at all.
    
    one finding, provenance attached
    

9.  5
    
    Apply the one-directional invariant
    
    Routing picks who speaks. It cannot silence anyone. If _any_ engine reports a failure, Marlo may not report clean. It is allowed to dissent, named, on the record.
    
    tested over all 256 outcome combinations
    

11.  6
     
     Report
     
     Terminal, JSON, SARIF 2.1.0, or a pull request body. Every surface puts what was not examined above what was found.
     
     severity is a text mark first
     

i

Why routing and not a union: engines find largely _disjoint_ issue sets and agree poorly, so unioning them lifts recall and buries the signal. One project integrated ten engines and about a thousand rules, got published at SIGACCESS, got adopted inside a Fortune 50 company, and reached almost nobody. Hand somebody ten engines' output and you have turned their detection problem into a triage problem, which is worse.

Where it will not help

## The list a sales page would leave out

1It is not comprehensive

41 of 94 published ACT rules, and automation reaches a minority of WCAG no matter who writes it. The denominator is on every page here for that reason.

2It will not recolour your design

Contrast gets detected and located. Never changed. Picking a colour is a design decision and it stays yours.

3It will not invent alt text

Decorative images get an empty alt, confidently. A description gets written only where the page already contains the meaning. The rest comes back to you, because a confident wrong description is worse than a missing one. You can spot a gap.

4It cannot certify anything

Neither can anybody else. What you get is verified repair against named success criteria, plus an error rate you can look up. That is the whole offer.

5Repair is not merged yet

Detection, routing, calibration and reporting are. `marlo fix` exits with an error saying that the repair layer is not merged, instead of being a flag that quietly does nothing.

6Two of three mappings are unverified

Only the axe-core mapping was derived by measurement. Alfa's and HTML CodeSniffer's are documentation matches, marked partial, and a test fails if either one ever claims otherwise.

The part that is not negotiable

## Marlo will never sell remediation to anyone it rates.

Every incumbent that both scores you and fixes you has a reason to find work it can bill for. That is not a conspiracy, it is an incentive, and incentives win.

Independence is the only asset a measuring tool owns. Spend it and you do not get it back.

Also never

-   xMerge its own pull requests
-   xPush to your default branch
-   xForce push, or rewrite history
-   xDeploy anything, anywhere

Enforced in the token scopes it asks for and asserted by tests, not promised in prose.

## Running it takes about five minutes.

No API key, no browser download, no account, and no network after install. If any of that turns out to be false, it is a bug and we want the issue.

[Clone the repository](https://github.com/KarthikSubramanian07/Marlo) [Read where it was wrong](https://trymarlo.pages.dev/honesty/)
