Report what you shipped
The moment a change is live, say so. It is the most consequential thing in the whole surface, and it is the one step nothing else can do for you: the engine watches search data, which tells it a page moved but never why.
A change nobody reported still shows up — content is snapshotted, so a CMS edit, a hand edit and a deploy all become visible as observed changes. What is lost is the measurement. Observed changes have no control group, because by the time they were noticed the moment to freeze one had passed.
What happens next
Reporting a change freezes a control cohort — a set of this site’s own untouched pages, comparable to the one you changed, sampled before the change could affect anything. The verdict is the difference between what the changed page did and what those pages did over the same window.
Comparing against the site’s own pages rather than against last month is the whole method. A seasonal dip, an algorithm update or a competitor’s campaign moves the controls too, and anything that moves both is not your change.
The window is 28 days, because that is roughly how long Google takes to re-crawl, re-rank and settle. Content changes get longer.
The five verdicts
- Positive — the changed page beat its controls by more than the noise in the measurement.
- Negative — it did worse than them. Worth as much as a positive, and the reason a ledger is kept rather than a list of wins.
- No effect — measured, and nothing detectable happened. A real result, not a failure to measure.
- Inconclusive — there was not enough data to look. Distinct from “no effect”, and on a small site it is the common answer for a single change.
- Confounded — something happened during the window that the comparison cannot see past. The whole site moved, your page was edited again, the result pages changed shape, or Google had not re-crawled enough of the window to read it.
What a confounded verdict is still worth
Less than a clean one, and more than nothing. Five of the six ways to be confounded finish the arithmetic before they detect the problem — the effect size and the significance are real numbers, sitting behind a result that cannot be trusted at full strength.
So a confounded verdict is weighted rather than discarded, and the weight is derived from the confound itself: how much of the window was actually read, how much of the control cohort was disturbed, how many comparable queries shifted. A weight below the floor banks nothing, and the two confounds that mark an absence of evidence rather than a weakness in it weigh zero. Everything above the floor teaches the engine a fraction of a trial, and the verdict says how much and why.
Inconclusive is the one that genuinely teaches nothing, and that is not the same failure: it means the cohort could not reach a verdict at any effect size, so there is no measurement to discount. A weight is a discount on a result, never a substitute for one.
Why you cannot ship everything at once
Control pages are finite. Every measured change takes a cohort out of the pool, and those pages are unavailable to every other measurement until its window closes. Ship eleven things on a Monday and you have measured perhaps two of them and guessed at the rest.
Ask the engine to plan a batch and it will say which of a set can ship together and still all be measured, which cannot, and why. Nothing stops you shipping the rest — you just ship them knowing they are unmeasured, which is a choice rather than an accident.
The ledger
Every reported change and what it did, on this site. It is the record that makes the next suggestion better than the last: a site that has shipped four internal-link changes to positive verdicts and two metadata rewrites to nothing is telling the engine something no detector measured.
A verdict marked pending has not finished its window yet. An empty ledger means nothing has been reported — which is worth checking, because it is the most common reason a customer feels the product is not learning.
Links and new pages are measured too
Report a backlink once it is live and the linking page is fetched and judged — a link that is nofollowed, in a footer, or on a page nothing reaches is not the link that was planned for, and finding that out later costs more than finding it out now.
Publish a page the engine proposed and report it, and the proposal half of the engine learns too. Without that report a proposal is a recommendation nobody ever checked.