How a non-developer reviews AI-written changes: the task-spec → build → review → fix loop, with the real files (2026)

Last verified: 2026-10-25

2026-10-25: Published after source checks, exact replacements, and the second review sheet.

A non-developer can review AI-written changes by turning every request into files that can be checked: a written spec with numeric limits, a separate comparison of the output with the source files, and fixes written as exact current-text → replacement pairs, applied only when each matches once. On the model-benchmark article published October 24, 2026, that loop recorded 57 changes — 54 exact text replacements and 3 CSV-cell updates. The comparison found seven categories of discrepancy in the article text supplied with the spec, and two of them (billing and latency) needed a second round. In a site-maintenance spec the same week, two instructions were checked against evidence and not followed as written — one against the platform’s documentation, one against the cost record’s own definition of a blank cell. The spec, both review sheets and a blank template are at the bottom.

The selected record in numbers

Item	Count	Where it came from
Task specs and review sheets selected for this article	7	Four task specs (site skeleton, benchmark article, site maintenance, email forms) and three review sheets (cost-article edits and two benchmark rounds); not a count of every task that week
Articles published	2	cost record (Oct 22), model benchmark (Oct 24)
Recorded benchmark-article changes	57	Round one: 42 text replacements + 3 CSV-cell updates; round two: 12 text replacements; publication metadata excluded
Text replacements with one match at application	54 / 54	Recomputed sequentially from preserved pre-edit snapshots; this does not measure every attempt before hand-off
Discrepancy categories found against the sources	7, then 2 remaining	The second round revisited billing and latency, two of the seven
Instructions not followed as written after an evidence check	2	Redirect: official platform documentation; cost cells: source captures and the CSV’s definition of a blank
Paid API calls during benchmark article drafting and edits	0 reported	The task reports state that no calls were made for that work; they do not establish an account-wide total for October 20–25

What a spec contains

The example task spec has five parts. Context describes what exists, where it lives and what the last run produced. Hard rules define the action boundaries: no unapproved commits or pushes, no secret output or hard-coding, numeric limits when paid API calls are in scope, stopping on payment or permission errors, and files that must remain untouched. Tasks name exact files, inputs and text, with marked slots where needed. Checks state commands and pass conditions. The report format includes [mismatch] for instructions that disagree with the sources and [unverified] for claims that could not be established.

The September four-model comparison had a task-level cap of 8 calls and an estimated $0.80 ceiling. Other experiments used different limits. That cap bounded the requested comparison; the cost article separately reconciled monthly usage CSVs and payment/balance records. A spending cap does not establish actual consumption or request-level billing.

What a review looks like

In this example, every text change was written as an exact current/replacement pair; a pair that did not match would be reported instead of applied. The safety check is to count each find string in its target file immediately before applying it and stop unless the count is one. In the preserved snapshots, 42 first-round text replacements and 12 second-round replacements each had one match at application. Three CSV cells were updated separately. The two rounds therefore total 54 text replacements and 3 cell updates: 57 recorded changes.

This matters for a non-developer because it turns “fix the article” into something auditable: every change is visible as a before/after line, and the sheet can be read without reading the code that renders the page.

Three cases from the week

Seven discrepancies found before publication. Comparing the benchmark article, built from a spec, with the source reports turned up seven discrepancies in the text the spec supplied: a mean rounded from a rounded number (13.55 s shown as 13.6 s; the raw mean was 13.546 s), a sentence asserting that a timed-out request “was billed” when the only evidence was a monthly total, a causal claim about prompt length, the word “licensed” for stock photos whose licenses the records could not confirm, a claim that the project “paused until September” that the usage data alone could not prove, a changelog entry that said “published” before publication, and a latency sentence that went beyond the measurements. The first review sheet addressed those categories. After it was applied, two categories—billing and latency—still went beyond the evidence, and a second sheet narrowed those claims. The published page uses the narrower wording.

An instruction checked against the platform’s documentation. The spec for site maintenance asked for a _redirects file to send www to the apex domain. The platform’s redirect documentation lists domain-level redirects as unsupported in that file, so the file was not created; the supported method was written up instead (a proxied DNS record plus an account-level bulk redirect). A read-only check on October 25, 2026 confirmed HTTP 301 responses that preserved the path and query string.

Two cells left blank because zero could not be verified. The same spec asked for the hosting row’s paid and consumed cells to be set to zero because the invoice list was empty. Both stayed blank: “no invoices listed” is an observation about a capture, not a verified total, and the cost record defines blank amounts as “unverified or not applicable, not zero”.

What this does not solve

An unsupported claim can pass a review, and source comparison can still miss errors. Official-document checks, exact-match checks, spending limits and keeping a draft until review help organize verification; they do not guarantee correctness. This case also has incomplete measurement: elapsed timestamps are available, but active human time and an account-wide API-call total were not measured. A report’s [unverified] section is itself something to check, not proof that nothing was missed.

This page went through the same loop

The first draft of this page was checked against the same week’s files and had seven discrepancies — among them the 57-change total presented as 57 text pairs, a spending cap presented as the reason the costs reconciled, and three claims not supported by the available measurement records (manual edits, active human time, an account-wide API-call count). Before publication, 46 exact replacements across the English and Korean versions fixed them, and the unsupported claims were removed rather than replaced with estimates. A second sheet then removed statements about who wrote, found or ran each step, and tightened a few details against the same sources. Who did a step does not show whether its result is correct; comparing the result with its source does.

Download the real files

The examples anonymize local paths, product names, private filename patterns and the author identifier; the English spec is a translation. The historical instructions and embedded draft text include wording that was later corrected. Read them with the review sheets as a worked example, not current guidance. Additional review records and a dry-run-by-default review-sheet apply script are included in NonDev Launch Kit.

FAQ

Do you need to read code to use this?

Not for the steps on this page. The specs, reports and review sheets are plain-language Markdown, and every fix is written as a before/after pair that can be checked against its source. The technical checks (type check, build, site checks) still have to run and pass.

What does comparing against the source files catch?

In this record it caught seven categories of discrepancy in the article text supplied with the spec, from a double-rounded mean to billing and latency claims that went beyond the evidence. Two of them, billing and latency, still went beyond the evidence after the first fixes and were narrowed again. The first draft of this page had seven discrepancies of its own.

What stops the agent from doing something expensive or irreversible?

The example specs set boundaries such as no unapproved commits, no secret output, numeric limits when paid API calls are in scope, and stopping on payment or permission errors. Written rules are instructions to check against the results; they are not technical enforcement by themselves.

How long does one article take with this loop?

Available file metadata places creation of the benchmark task spec at 18:35 KST on October 24, 2026; a saved live-site verification is timestamped 22:18 KST. That interval is about 3 hours 43 minutes. It is a file-and-verification timeline, not a timed productivity measurement; human active work time was not recorded.

Where does the method fail?

An unsupported claim can pass a review. Comparing the output with original sources and technical checks offers another chance to catch it, but none of these examples is a guarantee. In this record, two instructions were checked against evidence and not followed as written; that does not establish why every error was or was not caught.