v0.6.2 (2026-08-14)

The v0.6.2 release is a follow-up to v0.6.1, and its substantial changes are in the services rather than the catalog. Publishing a run is now idempotent, the backend serves traffic only once it can resolve a test case, its definition store survives a reschedule, and a run’s token usage is read from a session log rather than the harness stream, so a lost stream tail can no longer record a paid run as a free one.

The console gains version-scoped leaderboards, a way to restore a validator verdict a reviewer overrode, and a Verdict tab that puts the reviewer’s own work first. The catalog gains Carom v2.1.0 and Shatter v2.0.1, a new Arc Foundry audio item, and validation repairs to Arc Foundry and Meltdown. Eleven cases stop their specs contradicting the requirement that a fixed-timestep simulation run decoupled from rendering, and the nine reference implementations that had taken the contradiction at face value now draw between simulation steps. Eleven case versions are frozen.

This remains pre-1.0 software.

Features

Leaderboards rank one cohort at a time

The review leaderboard ranked over every version of a case, so a revised case ranked models against each other that were never set the same task. It now carries the version scope control the Metrics tab already had (current version, current major, a specific version, or all) and aggregates only the runs in scope. Both tabs default to the current version, so they describe the same cohort until one is widened. The scope vocabulary, its membership test, and the control itself moved into a shared VersionScope component.

Fuel totals are only comparable within one scored scenario set, so the performance leaderboard ranks exactly one version at a time. It was pinned to the case’s latest version, which left an older version’s fuel cohort unreachable from the console. It now carries a version picker, backed by useVersionPick, the single-version sibling of useVersionScope. The picker stays mounted alongside the empty state, and that message names the version it found no correct runs for.

A reviewer can restore an overridden validator verdict

A checklist point that the case instruments arrives pre-filled with the verdict automated validation decided, and a reviewer can override it. Nothing recorded that a stored verdict had been auto-set, so re-opening the review seeded every point from the prior review and the machine’s call was unreachable.

The verdicts live in the run record, which never changes, so the editor can offer the way back. An overridden point carries a Restore control beside its note, and the checklist rail offers “Restore validator verdicts” to put every overridden point back at once, confirmed and inert while nothing is overridden. Both restore the pass/fail alone: validation writes no notes, so the reviewer’s prose is theirs to keep or clear. Points that validation leaves to human judgement have no machine value to restore.

Seeding also marks a prior verdict that agrees with validation as auto-set, so re-opening a review shows which points still stand on the machine’s call. See overriding and restoring an automated verdict.

Carom v2.1.0 covers the ends of an obstacle face

The four obstacle bank-shot points each strike a vertical face at its midpoint travelling level, so nothing covered the ends of those faces, which is where deciding what was hit is the hard part. The new Ball point corner-graze grazes three corners at a shallow angle and asserts that the ball banks off one face and keeps its vertical direction, catching a build that reflects on both axes and sends the ball back the way it came.

Common checklist weight goes from 68 to 69, so v2.1.0 scores are not directly comparable with v2.0.1 scores. No seeded spec, prompt, or reference implementation changed.

An Arc Foundry item separates a silent cue from a silent clock

Every audio item measures under the manual clock, so a build that flushes its cue queue only while autoStep is on fails all ten at once, each reporting nothing beyond a cue that did not sound. The new item measures the same build on both clocks, using the three cue mappings the category already relies on. Its other assertions rule out the alternatives, so the one failure is attributable: a build that gates its flush on the clock passes the first three assertions and fails the last.

The case also records why an audio item may drive a given cue only once. Headless Chromium pins AudioContext.currentTime at 0, so a currentTime-keyed debounce never expires and any build drops a second cue of the same key.

Fixes

A run is published exactly once

A publish is not idempotent externally. Every publish job runs wrangler pages deploy, which mints a brand-new Cloudflare Pages deployment, while gh repo create reuses an existing repository. Duplicate publish work is therefore invisible on the GitHub side and accumulates on the Cloudflare side. A prod game jam recorded two publish jobs and two deployments 21 seconds apart, and the orphan had to be found by sweeping all 177 deployments in the project.

POST /runs/{id}/publish ran the publishable gate and then inserted a publish job unconditionally, with no check for an existing one and nothing in the schema to stop a duplicate, so N calls produced N jobs and N deployments. Enqueue now consults the queue for a live job on the run and answers with that job’s id and live URL instead of inserting a second. Repeated calls re-attach to the running publish, which is also the better outcome for the caller, since the console subscribes to the live stream it gets back.

A partial unique index on publish_job (run_id) WHERE state = 'queued' backs the check in the database, because the read and the insert are two statements and two concurrent requests can both observe no active job. The migration collapses any pre-existing duplicates, keeping the oldest, so it applies to a database that already recorded this bug. The index covers queued only: nothing reaps a publish job whose publisher pod died before reporting, so constraining dispatched would wedge that run’s publishing forever. The application check covers that state with a one-hour staleness cutoff instead. Terminal jobs never block, so a failed publish stays immediately retryable.

The publisher is the second layer. It reads GET /runs/{id} before any external work and, when the run is already published, skips the release and reports the links the run already carries. The backend re-asserts those links and preserves the original published_at, so the job is a truthful no-op rather than a failure a reviewer would be tempted to retry. An unreachable backend fails the job rather than falling through to a release. See the backend API.

The backend serves traffic only once it can resolve a case

Both probes pointed at /healthz, whose handler returned a hardcoded ready status without ever consulting the store. On a deployment whose /state volume starts empty, the definition store fills over minutes, so the pod joined its Service seconds after start with nothing to resolve against. Every run launched in that window died on a spurious “is not ingested” 404, and the failures then stopped on their own once the ingest landed. In prod this ran for about three and a half minutes after an AKS node-image upgrade rescheduled the backend.

A separate GET /readyz reports whether the store holds versions, and the readiness probe points at it while liveness stays on /healthz. The split is load-bearing in both directions: readiness on /healthz admits traffic to an empty store, and liveness on /readyz would kill the pod mid-ingest and never converge. The signal latches, seeded at startup from whether the store already holds versions and released when an ingest leaves it populated. It is gated on the store actually being populated rather than on a scan returning success, so a scan against an empty or broken checkout cannot flip a still-empty backend ready.

/healthz now reports a truthful storeReady boolean in place of the hardcoded string. That is the field the console’s Connections page has always read, so it stops reporting a store that is never ready.

The backend’s definition store survives a reschedule

The postgres component gave the backend an emptyDir at /state, on the reasoning that everything there is regenerable: lose it and you pay a re-ingest, not data. That holds, but regenerating means cloning the catalog and re-ingesting every version, and until it finishes the backend cannot resolve a single test case. Reschedules are not rare either, since AKS node-image auto-upgrade drains nodes on its own schedule and moved this pod repeatedly, each move paying the full cold rebuild.

/state is now a 10Gi tcab-backend-state PVC, which turns the cold rebuild into a disk attach. ReadWriteOnce is safe here and depends on the existing Recreate strategy, one pod at a time with the old fully terminated before the new one attaches; a rolling update would deadlock on the disk.

This composes with the readiness probe rather than replacing it. The PVC removes the cold rebuild from the common path, and readiness still covers the cases where the store genuinely is empty: a first deploy, a replaced disk, or a recreated cluster. The ingest sidecar keeps polling /healthz, because /readyz is 503 until the very ingest it is about to trigger completes.

A run’s usage is read from a session log

An exec stream can close as the runner exits and lose its last lines. Most harnesses report a session’s token usage on the very last line they write, so a lost tail took the usage with it, along with the wrapper’s closing sentinel. Session segmentation then discarded the unterminated segment whole, leaving no sessions, no token classes, and a cost of $0.00: a confident claim of a free run, on a run that had cost real money. The run itself exited cleanly and was recorded as a normal completion, so nothing surfaced the gap. Four of twenty-four prod Codex runs on one build hit it.

tcab-session now tees each session’s stdout, sentinels and all, to a session log on the container’s own disk, and the orchestrator reads the log back after the runner exits and extracts usage from it. The stream still drives live events and stands in when the log cannot be read. A pipeline discards the harness’s exit status in POSIX sh, so the status is written to a file inside the pipeline and read back out, and that number decides the run’s terminal state. An unwritable log degrades to the plain invocation, since losing usage is bad and losing the run is worse.

Three things that turned the loss into silent bad data are fixed alongside it:

• segment_sessions keeps an unterminated trailing segment rather than dropping it, so a harness that reports usage incrementally still contributes everything that arrived.
• parse_usage distinguishes output carrying no usage line at all, which is unknown, from a reported zero. Without this, keeping the partial segment would record a confident zero tokens.
• Cost::comparable_from returns None when no token class is known, instead of pricing every unreported class as zero tokens and yielding $0.00.

A session that arrives without its closing sentinel warns on the run’s event stream and in the logs, naming which source was read, so a recurrence is visible instead of silent. See orchestrators and metrics.

Shatter v2.0.1 corrects its color, saucer, and recycle checks

Color sampling composites each canvas sample over the field background before comparing, so a build that leaves its canvas clear is measured on the colors it paints rather than on unpremultiplied glow fringes. The color scene is posed after any opening wave banner, which the star sample sat under.

saucer/avoids-star also drives the saucer into the core at cruise speed, which is the half of “never overlapping it” that a saucer posed at rest cannot test. star-core/rock-recycled measures the edge the replacement enters from and its size rather than its distance from the star. A new saucer/cadence item covers the 25 to 35 second gap between visits.

The only seeded change against v2.0.0 is the render-decoupling rewording every fixed-timestep case received; every other specification, the prompt, and the reference mockups are byte-for-byte those of v2.0.0, and only the changed scripts’ baselines were recaptured.

Arc Foundry reads derived figures against the spec

Scaled HP and aura-buffed damage are exact reals that the specs never ask anyone to round. The helpers rounded because the reference implementation does, so a build reporting the formula’s own value (9.68, 6.6) failed. Both are now compared with one nearest-integer of slack.

Two pathing checks that a mirrored board satisfied are widened. three-maps reads each map’s checkpoints against the coordinates specs/board.md pins, rather than only checking that the three chains differ, and no-build-on-waypoint probes the stem on both sides of the centre row. The stem rule and the map tables moved into _helpers.mjs so the items share one statement of them.

Meltdown measures the pause freeze on the build’s own clock

pause.freezes paused and then advanced, which scores where a build puts its pause gate rather than whether the floor freezes. A build holding the pause in the shell that drives the clock stepped straight through the check while its clip, filmed in real time, showed the unit stopping dead. The specs do not pin the gate’s placement, and the check was blind to the real defect: a pause menu opening over a running floor passes it whenever step happens to be gated.

Both legs now run on the build’s own clock, so the Mote must walk and then hold with simTime stopped. Verified against three run implementations and the reference, and against mutants that never freeze, freeze only under step, or never advance.

The Verdict tab folds its validation table away after review

The Verdict tab dropped the full automated-validation table into the review panel the moment a reviewer submitted their own review, and kept it there once the run was published, putting a screenful of check tables between the top of the panel and the review summary and actions below it. By that point the reviewer has already made every call the list would inform, so it is reference context rather than the task.

DebugScriptList gains a collapsible mode, where the heading becomes a disclosure toggle carrying the count of checks behind it and the list starts closed. The post-submit block on the Verdict tab uses it, matching the live-score bar’s breakdown, which already opened closed. The Metadata tab’s debug scripts section stays expanded, because it ends that tab and its height pushes nothing out of reach.

A reviewer’s note reads as review text

A checklist point’s description, which comes from the case definition, and the reviewer’s note on that point both rendered as plain muted text stacked under the title, so there was no way to tell the rubric’s own prose from what a reviewer wrote. The note is now a quoted annotation: an accent left rule over a tinted ground, full-strength text so it does not recede behind the muted description above it, and a “Reviewer” label so the block reads as review text without relying on color alone.

Test cases

Eleven cases stop contradicting “decoupled from rendering”

Every case that runs a fixed-timestep simulation asked for it “decoupled from rendering” and then undercut that a few lines later with wording like “Rendering reads the state, never the other way around”. Read as a constraint on the renderer, that pins what is drawn to whatever the last completed step left behind, tying the picture to the tick boundary rather than freeing it from one. The requirement now states only the one-way dependency it always meant: the simulation never reads from, waits on, or is driven by the renderer. Nothing was added to what a build must do.

Every reference implementation had taken the sentence at face value, so on a display whose refresh rate does not divide the tick rate the rendered step size varied frame to frame: about half a step of position error, metronomic at the beat frequency between the two rates, and a fully frozen frame when the refresh matches the tick rate. Each step now stamps where the moving bodies stood when it began and the loop hands the renderer the unconsumed fraction of the next step, so it draws between the two. Measured on Carom at 200 px/s, peak positional error against true frame time falls from 0.90 px to 0.004 px on a 120.25 Hz clock.

The simulation itself is untouched: the interpolation state is written by the step and read only by the renderer, so a given seed and sequence of steps reaches exactly the state it did before, and every scripted scenario is drawn exactly as it was stepped. Coil and Cascade are spec-only, having no such defect to fix. Coil’s snake is cell-snapped by design at 8 ticks per second, and Cascade stamps each in-flight card onto a persistent trail layer once per step rather than redrawing it live.

Every non-experimental case has a published reference build

Carom v2.1.0 and Shatter v2.0.1 had no reference implementation deployed at all, and the seven whose reference implementation changed above still served the builds they were deployed from before it: Fathom, Floe, Spectra, Wireworm, Meltdown, Arc Foundry, and Deepcore. A case page’s Reference tab therefore offered a build that no longer matched the source committed beside it. All fourteen variants are rebuilt and deployed to both environments, and their URLs recorded in test-cases/reference-builds.lock.json. See publishing a reference implementation.

The committed baseline validation media is deliberately left as it was. It is the reference half of a reviewer’s side-by-side rather than an input to scoring, the simulation it depicts is unchanged, and recapturing it would move the digest of nine frozen versions, so it is worth refreshing on its own terms rather than as a side effect of a deploy.

Eleven versions are frozen

The newest version of Carom (v2.1.0), Fathom, Shatter (v2.0.1), Deepcore, Cascade, Floe, Spectra, Wireworm, Meltdown, Coil, and Arc Foundry now carries a .frozen marker, so the commit hook and CI reject edits to inputs that runs have been scored against. See frozen versions.

Valence v2.0.0 is marked experimental, which keeps it out of the scored catalog while its specs and checks settle.

Development

The commit hooks run the locked linters

The cspell and markdownlint gates declared their tools with pre-commit’s additional_dependencies, so each was installed into an isolated node env that resolves transitive dependencies at hook-install time and ignores package-lock.json. CI runs npm ci first, so it spell-checked with different dictionary data than the commit gate did. Both gates now run the locked copies.

A test that only passes on retry is a failure

The nextest profile allows two retries rather than one, and sets flaky-result = "fail", so a test that fails and then passes is reported as failing. A flaky test is a defect in the test or the code under it, and absorbing it into a green run hides that.

Deployment

Staging and prod roll to v0.6.1-rc5

Both azure overlays are pinned to the same v0.6.1-rc5 build e376e0fa: staging moves its service images up from e27acecd, and prod up from the v0.6.0-rc2 pin 518e7ae4. Pinning both to one sha is the point of the staging-first flow, since prod runs a build that staging has already exercised. Each bump covers the three places the sha appears: the images: transformer block and the two env values it cannot reach, TCAB_DRIVER_IMAGE and TCAB_PUBLISHER_IMAGE.

Over the 113 commits prod carries: the run-record and contract-codegen contract changes, core validation fixes, the UI package, the game-jam publishing repairs, the dispatcher fix that stops a killed driver orphaning its sandbox pod, and a batch of per-case validation repairs. It also carries one schema migration, m20260731_000023_add_job_test_type, an additive column with a "" default and a working down, already exercised against the staging Postgres.

TCAB_CONTAINER_TAG stays at the v0.6.0-beta4 build e6afbee6, because no build since has run build-containers and nothing under containers/ has changed, so the beta4 run images remain content-current.

The LGTM volume swap takes two more steps

Rolling the added telemetry volumeClaimTemplates onto prod turned up two things the observability note left out. The rejected StatefulSet update aborts kubectl diff -k for the whole overlay at exit 2, so an unrelated image roll gets no preview at all until the offending resource is filtered out. And --cascade=orphan plus a re-apply does not finish the job: the recreated StatefulSet adopts the orphaned pod but leaves it on the old revision, with old volumes and old image, while reporting 1/1 ready. The pod has to be deleted explicitly, and that restart is what provisions the added PVCs, since the default StorageClass is WaitForFirstConsumer.

The note also records checking per-node disk headroom first: one entry is one attached Azure disk, capped at 8 on the D2ps_v6 pool.