Testing cadence

Three tiers, three costs, one rule.

Certificates make the rule mechanical

Each tier carries the content hash of every file it reads, written when it passes. certify run runs only the tiers whose inputs changed; a page-component edit leaves the node tier fresh because the node tier never reads pages. A pre-commit hook runs certify check and refuses a tree that is not certified for every tier. The rule stopped slipping the day it became a check.

node tooling/certify.mjs check     # FRESH / STALE per tier, exit 1 if any stale
node tooling/certify.mjs run       # run the stale tiers, certify on pass

The box-wide lock, and the load rule

Two rules the certifier applies per tier, both measured before they existed:

const TIERS = {
  fast:    { reads, cmd: ['yarn', ['-s', 'test:fast']] },                          // never waits
  node:    { reads, cmd: ['yarn', ['-s', 'test:node']],    lock: 'heavy', maxLoad: 4 },
  browser: { reads, cmd: ['yarn', ['-s', 'test:browser']], lock: 'heavy', maxLoad: 4 },
}

• lock: 'heavy' — the tier runs under a box-wide flock on <CVN_LOCK_DIR or the OS temp dir>/enc-testbox-heavy.lock. Every repo on the box that names the same lock queues its heavy tiers instead of contending: one browser tier at a time, whichever repo. A waiting tier says who holds the lock (host:repo tier pid=… since=…). Two repos' Playwright runs beside each other took the load average to 48 and failed 13–34 specs per run on shell-visibility timeouts — none a defect, all a re-run; that is what this ends, for every repo that takes it.
• maxLoad: 4 — the tier starts only when the 1-minute load average is below the rule, checked after the lock is held so the place in the queue is kept while the box drains. It cannot end contention from a repo that does not take the lock; only the lock can.

CVN_LOCK_WAIT (seconds, default 3600), CVN_MAX_LOAD (overrides every tier), CVN_NO_LOCK=1 (a box you own alone). Adopt with createCertifier:

import { createCertifier } from '@enc-protocol/cvn/tooling/certify'
process.exit(createCertifier({ root, cwd: here, certFile, tiers: TIERS }).main())

impl-cvn/test/certify.test.mjs is the proof: two concurrent holders of one lock never overlap; a certificate goes stale only when an input the tier reads changes; a tier that edits the tree during its run is not certified.

Configure the tiers' input sets and commands for your repository at the top of the script; the MVP's copy is the reference.

Instruments, in the order to reach for them

1. Read the artifact. The trace, the page-level trace, the captured log. A failure re-run before its artifact is read costs the artifact.
2. Profile the page, not the box. A "starved" verdict from load average or frame gaps is not evidence about the machine: the CPU profile of a page at 97 % busy named a read-receipt loop the DOM never showed. The MVP's capture fixture takes CLEAN_PROFILE=1.
3. Count what went on the wire. A regression test for a loop counts frames, not pixels.

What a spec may not do

If a spec has to be edited to pass, stop and report it. A longer timeout is not a fix. A retry on a refusal is not a fix. A run that is green two times in three has a defect, not a flake.