Review a Run

Every published run carries one or more hand-written reviews — each a writeup, a rating for each scoring domain, and a verdict on each of the case’s reviewer-checklist items — authored after playing the build. The verdicts and the items’ point weights produce each review’s score; across a run’s reviews the score is averaged and the overall rating is the worst. Every review is attributed to the account that wrote it, a run may carry one review per account (often from people other than whoever produced it), and publishing refuses a run with no review. You review a produced run — one whose build is playable off the artifact service — before it is published. The full workflow is in Reviewing Test Run Results.

Review in a console

The Tauri desktop app and the web console are the primary way to review. Sign in, then open the finished run, play its build, and fill in the review editor:

• Work the reviewer checklist — one item at a time, with a rail of every item alongside (answered items marked done) so you can move freely. Each item shows its point weight and gets a binary verdict (pass / fail), optionally with a note. When the item pairs them, the question shows the case’s expected reference beside the agent’s submitted proof of implementation — image or video — so you compare the target against the evidence before judging. The console will not let you save the review or publish the run until every item has a verdict and every domain is rated (the completeness gate).
• Pick a rating for each scoring domain and write the prose writeup. The run’s overall rating is the worst across the domains; its score is the weight of the passed items over the total.

Every run also has a Proof tab listing all the proof media the build submitted (and any it didn’t), browsable independent of the checklist.

The console writes the review to runs/<id>/writeup.md for you.

Editing a review you already submitted

An account reviews a run once — re-submitting from the same account updates that review in place rather than adding another. Use Edit review (on your own review, from any signed-in console that can submit) to revise a verdict, rating, or the writeup after the fact.

Verdicts that automated validation decided are recoverable in an edit: a point whose answer differs from the machine’s carries a Restore control, and the rail’s Restore validator verdicts puts every overridden point back at once (notes are kept). See overriding and restoring an automated verdict.

An edit that actually changes something requires a short note explaining what changed — the reviewer must say why. Each edit is preserved as a revision in the review’s public edit history, alongside an autogenerated diff (which ratings flipped, which verdicts changed, whether the writeup was revised). The review’s first submission time is kept; an edited marker records the latest revision. A re-submission that changes nothing is a no-op and needs no note.

Review from the CLI

From a signed-in shell, submit a review for a pushed run with its writeup file, attributed to your account:

tcab review runs/<id>/run-record.json --writeup runs/<id>/writeup.md

The review file is also hand-editable. Create runs/<id>/writeup.md beside the run’s run-record.json, with a rating.<domain>: line per scoring domain in YAML frontmatter and any checklist verdicts as review.<id>: <status> [note] lines (a sub-itemed item uses one review.<item>.<sub>: line per sub-item):

---
rating.single-player: flawless
rating.versus: scuffed
review.ball-spin.stationary: pass
review.ball-spin.moving: pass
review.obstacle-bank: fail ball clips the top obstacle corner
---

Single player feels right. Versus has a serve bug that resets the score, so it's
playable but scuffed.

Each domain’s rating must be one of:

• flawless — to spec, no noticeable bugs.
• great — to spec; minor issues that don’t impact playability.
• passable — to spec and playable, but with rough edges beyond a great run’s minor issues; not enough to deviate from spec or impair play.
• scuffed — mostly to spec; playable but noticeably deviates from spec or has bugs that affect play.
• broken — doesn’t follow the spec, or is unplayable.

The body must not be empty, and a run cannot be published while any declared domain is unrated or any declared checklist item — or sub-item — is missing its verdict. The run’s overall rating is the worst across its domains.

Preview the build

To preview an unpublished run the way a visitor will — outside a console — run the gallery dev server; it scans runs/ and plays each run’s local build where one exists:

npm run dev -w @test-cabinet/site

The overall rating badge, score, and writeup preview exactly as they will once live.

Next step

Publish a Run once the review is in place.