995 Pages, One Change, No Result Yet: Our Live Test

We run a live test on our own published pages and this report is the notebook: what we changed, on how many pages, against how many we held back untouched, and whether an answer exists yet. It does not. This edition prints the design and the before picture only.

The rule that decides whether this change wins, loses, or continues was written down on August 24, 2026 — before anyone, including us, knew what the pages would do next.

Key Takeaways

• 469 pages got the change; 526 pages did not. Both groups were drawn from the same pool of pages that already had AI-crawler attention.

• The change is a two-sentence factual line inserted directly under the page heading: what we do, our self-serve plans from $32/month billed annually, and a link to /pricing.

• No lift, drop, direction, or "early signal" is stated anywhere in this report, because none exists yet.

• The scheduled readout is September 25, 2026; the decision rule cannot fire before September 7, 2026 at the earliest.

• The before-window ran July 27, 2026 – August 23, 2026, ending the day before the change went live.

The Question, Written Before the Result Was Known

Does adding one factual paragraph about who we are and what we sell, placed directly under the heading of a page, change how often that page earns a click from search — measured against an untouched group of otherwise-similar pages over the same weeks?

That is the whole question. Nothing about tone, placement variants, or button color is being tested here — just the presence of one paragraph, on 469 pages, against 526 pages that did not get it.

How the Pages Were Split

The pool was pages with AI attention: pages that had at least one AI-crawler read on August 23, 2026, or at least one AI-assistant visit in the trailing 28 days. Pages already enrolled in two other running tests — a title-shortening cohort and a zero-click retargeting cohort — were excluded so the three experiments cannot contaminate each other, along with 18 pages retitled the same day the change launched.

From that pool, 995 pages were split 50/50: 469 into the group that received the change and 526 into the group that did not. Two duplicate-slug pages were dropped after the initial split, which is why the treatment count reads 469 rather than the original 471. The pool itself draws on the same AI-crawler attention data behind our AI-crawler census — a page needed at least one read from an AI bot or assistant to qualify for either group.

The two groups were drawn from one pool that already had AI attention, not from every page on the site. The comparison that will be read out on September 25 is between those two groups specifically — it does not generalize to pages outside that pool.

The Design at a Glance

That table is the whole design in one place: what was tested, when it started, and the two dates that bracket when an answer can exist.

The Before Picture

Before either group changed, we measured what each was already earning from search, so the later comparison has something honest to compare against. The baseline was computed twice — once on the original page list, and again after the two duplicate pages were dropped and the measurement window shifted by one day. The second pass is the one the scheduled readout will use.

Both groups earned single-digit-percent click-through rates before the change, on a combined 995 pages. That similarity before the change is what makes any difference after the change worth reading.

The held-back group is not "doing nothing." It is the control, and it is the only reason a later difference between the two groups will mean anything at all — without it, there is no way to tell a real effect from ordinary week-to-week noise.

The baseline above is the restated pass — the one the scheduled readout will actually use. It was not the only pass computed. The table below carries both, so nothing is quietly swapped out from under a reader who bookmarked the first version.

Baseline Detail, Both Passes

The two passes differ because the original count ran on 471 treatment pages before the 2 duplicate-slug exclusions, and the measurement window shifted by one day once those exclusions were applied. Both click-through rates were 0.2% in the original pass, on either side of the split — another sign the two groups were comparable before anything changed.

When the Answer Is Due, and What Counts as a Real Difference

The comparison needs two things before it can be read: at least 14 days since launch, and at least 100 combined clicks across both groups. Both conditions together put the earliest possible read at September 7, 2026. The readout is formally scheduled for September 25, 2026, giving the test a full run before anyone looks.

The rule that decides the outcome, written before launch:

• If the treatment group's click ratio comes in below 0.8x the control group's ratio, the change is reverted.

• If it comes in above 1.2x, the change is called a winner.

• Anything in between, and the test continues.

No outcome exists in the sealed ledger behind this report — the measurement count on file is 0. Whatever number a reader might want to see here is not on the page, because it has not been measured yet.

Method in Plain English

The primary measurement is Search Console clicks to the treatment group compared with the control group, read as a diff-in-diff against the before picture above — not a raw after-number on its own. A secondary read tracks AI-assistant visits per group and /pricing page arrivals per group, both pulled from our own Cloud Run server logs rather than a third-party analytics tool.

Excluded from both groups: pages already enrolled in the title-shortening and zero-click retargeting cohorts, and the 18 pages retitled the same day this test launched. Mixing those changes into this test's pages would make it impossible to tell which change caused which result. Every figure in this report is read directly from a sealed experiment ledger; nothing is estimated, modeled, or extrapolated. Where the test has not been read out, no result is stated at all.

Why We Publish the Losing Arm Too

US Tech Automations treats a failed test the same as a winning one: both get an edition. If the September 25 readout shows the change dragging performance down relative to the control, that edition is the whole point of running this series in public — not a result quietly dropped from the next report. Readers of our AI-crawler census already see us publish first-party numbers that don't always flatter the estate; this test follows the same rule.

Put the Testing Discipline to Work

Most sites that try a page-level change never hold back a control group, never pre-write a decision rule, and never publish the result if it goes badly — they just quietly keep or revert the change and move on. A content lead or an SEO lead running their own page-level experiment usually has the opposite problem: too many one-off tweaks and no repeatable way to tell which one moved the needle.

US Tech Automations builds the automation workflows that make a test like this repeatable at scale — a defined split, a pre-registered rule, and a scheduled readout date fixed before the data comes in, run the same way on the next test and the one after that. Our Search Console operating data uses the same page inventory this test draws its pool from, if you want the fuller picture of what these 995 pages sit inside.

Frequently Asked Questions

Q: Did the change win or lose?
A: Neither — no result exists yet. The sealed ledger behind this report shows 0 measurements recorded. The earliest the decision rule can fire is September 7, 2026, and the scheduled readout is September 25, 2026.

Q: What exactly changed on the treatment pages?
A: One paragraph inserted directly under the page heading: a two-sentence factual description of what US Tech Automations does, that self-serve plans start from $32/month on annual billing, and a link to our pricing page.

Q: Why does the treatment group show 469 pages in some places and 471 in others?
A: The split originally put 471 pages into the treatment group. Two pages turned out to have duplicate slugs and were excluded, and the baseline was recalculated on the corrected 469-page group for the window the readout will actually use.

Q: How will you know if the test worked?
A: A rule fixed before launch: if the treatment group's click ratio comes in below 0.8x the control group's ratio, the change is reverted; above 1.2x, it's called a winner; anywhere between, the test continues running.

Q: Does this test measure leads or sales?
A: No. The primary measurement is clicks from Google search results. A secondary measurement tracks visits from AI assistants and arrivals on the pricing page, but neither of those is counted as a lead, an enquiry, or a sale in this report.

Method and Provenance

Every number in this report is pulled directly from our own sealed blog-experiment ledger and the paired Search Console baseline behind it, snapshot sealed August 25, 2026. This is one controlled test on our own blog pages: a named change applied to one group, an untouched group alongside it, and a decision rule written down before the result was known. No figure here is estimated, modeled, or projected — where the test has not yet produced a reading, this report says so instead of guessing.

Source: US Tech Automations Research — Blog Experiments series, sealed 2026-08 edition, from our own blog-experiment ledger and Search Console baseline.