Cited vs. Never Cited: Our Pre-Registered Test of the Control Group a 4.2x Citation Claim Never Had

Experiment design — published October 13, 2026, in the GEO Lab. This is a pre-registration: the topic set, the scoring checklist, the metrics, and five falsifiable predictions go out now; we collect and score later this week. We’re testing our own content-strategy advice here as much as anyone else’s — if hygiene scoring turns out not to predict citation once a real control group exists, that’s a result we have to sit with, not soften.

What claim are we testing this week?

The hypothesis Monday left open, stated so it can lose: a content-hygiene or quality score meaningfully predicts whether a page gets cited by an AI engine, even when compared against pages that were equally eligible and weren’t cited — not just whether it predicts getting picked up by a second engine once a page is already cited by one, which is the narrower thing the GEO-16 paper actually measured. If that’s true, pages an AI engine cites should score noticeably higher on our hygiene checklist than the other pages sitting in the same results for the same query.

The counter-hypothesis, which our own prior work leans toward: once you control for which pages even show up in the running — i.e., compare cited pages only against other pages that were realistically in contention for the same query — hygiene score adds little, because domain authority already does most of the sorting before content quality gets a vote. On that story, hygiene scoring describes what cited pages tend to look like, but doesn’t move the odds much once authority is accounted for. Both are plausible going in. This experiment is built to make the data pick a side, with the comparison group Monday’s source claim couldn’t run.

What exactly gets collected — the topics and the candidate pool?

Ten topics, deliberately split across two intent types — six product-intent (the only type GEO-16 sampled) and four informational/how-to (a type it never tested), so this week’s sample is broader than, and not directly comparable in composition to, the source claim we’re checking:

For each topic we run the query on ChatGPT (web search) and Google AI Mode — the same two engines the rest of this series uses — logged out, single snapshot. Separately, we pull the top-10 organic Google results for the exact same query string. That top-10 is our candidate pool: the pages that were realistically positioned to be seen and cited for that query, whether or not the AI engine actually picked them. Every URL in the pool gets a label — cited if it appears in that engine’s citation list, not cited if it doesn’t — and every URL in the pool gets scored on the hygiene checklist below, cited and non-cited alike. That last part is the entire point: GEO-16’s 1,100 scored URLs were 100% cited, 0% not. Ours won’t be.

How do we score “hygiene” without the original study’s exact 16 pillars?

We don’t have GEO-16’s internal scoring rules, so we’re not replicating its 16 pillars line for line — we’re approximating the same idea with 8 checks we can verify by reading the live page, each scored 0 or 1, summed and divided by 8 for a 0–1 score we’ll also call G for comparability:

1. Visible published or last-updated date on the page.
2. Named author or byline attributed to a person or identifiable entity.
3. Structured data present — any Schema.org JSON-LD in the page source.
4. FAQ or explicit Q&A block addressing follow-up questions.
5. Direct answer within the first 150 words — the page states its core answer before making the reader scroll for it.
6. At least one original stat, number, or named data point — not just prose claims.
7. A table or list making the content scannable rather than pure paragraphs.
8. Three or more descriptive H2/H3 subheadings that map to real sub-questions, not generic section labels.

Three metrics do the work once every pooled page has a label and a score:

• Mean G, cited vs. not-cited — within the same 10-page pool for the same query, do cited pages score higher on average than their non-cited neighbors?
• Authority-tier split — using the same top-15-domain authority tier from our earlier concentration study, we check whether citation correlates more with sitting in that tier than with hygiene score, and whether G’s apparent effect shrinks once authority tier is held constant.
• Single-engine vs. cross-engine G, inside our own data — among pages cited by at least one engine, do the ones cited by both score higher than the ones cited by only one? This directly replicates GEO-16’s actual (narrower, undisputed) finding on our own sample, as an internal consistency check.

If mean G is meaningfully higher for cited pages even after accounting for authority tier, hygiene is a real, independent lever and GEO-16’s broader marketing claim gets real support it didn’t have before. If the gap mostly disappears once authority tier is controlled, hygiene is describing a correlate of authority rather than an independent cause — the pattern our prior predicts.

What five predictions are we locking before collection?

Pre-registered, numbered, each falsifiable. If the data contradicts one, Thursday’s post says so in plain language — no quiet reframing.

1. P1 — Cited pages score higher, but the gap is modest. Mean G for cited pages exceeds mean G for non-cited pages in the same pool, but the gap is under 0.15 on the 0–1 scale. In plain terms: hygiene matters some, but nowhere near enough to support reading “4.2x” as a real citation-odds multiplier. This is the load-bearing prediction — if the gap is much larger than 0.15, hygiene is a stronger independent lever than our prior work suggests and we’re wrong to be skeptical of the marketed claim.
2. P2 — Authority tier beats hygiene score as a predictor. Pages in the top-15 authority tier get cited at a rate at least 2x higher than pages outside it, regardless of G, and the cited-vs-not G gap from P1 shrinks by at least half once we restrict the comparison to pages within the same authority tier.
3. P3 — GEO-16’s actual finding replicates on our data. Among pages cited by at least one engine, pages cited by both ChatGPT and Google AI Mode show a higher mean G than pages cited by only one — consistent with the one comparison GEO-16’s own design could support. We expect this to hold even where P1’s broader claim doesn’t.
4. P4 — Counter-examples exist in most topics. In at least 6 of the 10 topics, at least one non-cited page in the pool scores a higher G than at least one cited page — direct evidence that hygiene alone doesn’t determine the outcome, not just that it correlates imperfectly.
5. P5 — A G≥0.70 threshold does not approach 4.2x odds in our data. Among pages scoring G≥0.70, the citation rate is less than 2x the citation rate among pages scoring below it — well short of the 4.2 odds ratio the source claim reports, even allowing for our much smaller sample. We flag this prediction as likely underpowered going in: 10 topics give us roughly 20–40 total citations to work with, not GEO-16’s 1,702.

Together these describe one shape: hygiene is a real but minor correlate of citation, riding on top of authority, not a standalone predictor strong enough to justify a multiple-x framing. That lines up with what we’ve already found moves citation — 15 domains controlling a disproportionate share and rank predicting citation at only 34.4% — and it’s a harder test of our own advice than Monday’s post was of someone else’s claim. If P1 fails and the gap is large, our own content checklist just got a real evidence base it didn’t fully have before. If P2 fails and authority doesn’t explain the gap away, that’s also notable. Friday’s verdict says which happened.

What could make this experiment misleading — and how do we stay honest?

Five limits, named in advance. One: top-10 organic is a proxy for “eligible,” not the real universe. The true population of pages that could have been cited is larger than what Google ranks in its top 10 for the same query; we’re approximating “in contention” with “ranked,” which is a narrower and more authority-skewed pool than the true candidate set — if anything, this should work against finding an authority effect, since the pool is already pre-filtered toward authoritative pages. Two: our 8-pillar checklist approximates GEO-16’s 16 pillars, it doesn’t replicate them — some of our checks (direct answer in 150 words, descriptive subheadings) require a judgment call, and we’ll flag borderline scores when we publish the sheet. Three: the sample is small. 10 topics × 2 engines gives us a real but limited number of citations to test an odds-ratio-style prediction against — P5 is explicitly flagged above as likely underpowered, and we’ll report the raw counts so the limitation is visible, not hidden behind a percentage. Four: this is still observational, not a true experiment. Even with a genuine non-cited control group, we’re not randomizing hygiene scores onto otherwise-identical pages — the gold-standard test would publish matched content variants and wait to see what gets cited, which takes weeks of indexing time we don’t have this week. What we’re building is strictly better than GEO-16’s design (which had no control group at all), but it still can’t prove causation, only whether a real correlation survives having a control group to compare against. Five: one logged-out snapshot, same caveat every GEO Lab test carries — citation sets drift and vary by engine, and this is a single point in time.

We’re pre-registering this the same way we did last week’s claim audit and the phrasing experiment before it — predictions locked in public, before the data exists, so we can’t quietly redefine “modest gap” after seeing the number. Wednesday we collect; Thursday we open the full per-topic score sheet; Friday we judge all five predictions against exactly what’s written here.

Frequently asked questions

What is this experiment actually testing?

Whether a content-hygiene or quality score predicts which pages an AI engine cites, when compared against a genuine non-cited control group — pages that were in the running (ranked in the top 10 organic results for the same query) but weren’t cited. It’s the direct follow-up to Monday’s finding that a widely-repeated “4.2x more AI citations” quality-score study never ran this comparison, because every page in its sample had already been cited by something.

Why use the top-10 organic results as the candidate pool?

Because it’s the closest practical approximation of “pages realistically eligible to be cited for this query” that we can pull in a week. It’s an imperfect proxy — the true eligible set is larger and less authority-skewed than a top-10 ranking — but it’s the control group the source claim’s own design never included at all, which is the gap this experiment exists to close.

Why 8 checklist pillars instead of GEO-16’s 16?

GEO-16’s exact scoring rules for all 16 pillars aren’t fully reproducible from the published paper alone. Rather than guess at rules we can’t verify, we built 8 checks we can score consistently by reading the live page — date, author, structured data, FAQ block, early direct answer, original data, scannable formatting, and descriptive subheadings. This is explicitly an approximation of the same underlying idea, not a replication of GEO-16’s exact instrument.

Why might this experiment be underpowered?

10 topics across 2 engines will likely produce somewhere around 20–40 total citations, compared to GEO-16’s 1,702. That’s enough to test whether a mean-score gap exists (P1, P2, P4) with reasonable confidence, but an odds-ratio-style claim like P5 needs more citations than we’ll have to detect anything short of a large effect. We’re flagging that limitation now, before collection, rather than explaining it away after seeing the number.

When are the results published?

We collect Wednesday and publish results Thursday with the full per-page score sheet, then a verdict Friday that judges each of the five predictions and closes the loop this week’s arc opened on Monday. This post is the locked design; nothing here changes once the data is in.

Sources

• GEO Vendors Are Citing a “4.2x More AI Citations” Quality-Score Study — this week’s Monday source check that found the missing control group this experiment builds.
• Do 15 domains control AI citations? — the authority-tier concentration data behind P2’s prediction.
• Rank vs AI citation overlap — 34.4% blended overlap, prior evidence that the obvious ranking signals don’t fully explain citation.
• Last week’s pre-registered citation-chain audit method and its results — the R/M/S scoring precedent this week’s design extends.
• Query phrasing citation experiment — an earlier GEO Lab test built with the same pre-registration discipline this week’s arc reuses.
• Counted vs. inferred GEO metrics — the broader distinction between numbers someone actually counted and numbers inferred from a biased sample, directly relevant to what this experiment is trying to fix.
• GeoParrot GEO Lab — pre-registered design; collection to follow across ChatGPT (web search) and Google AI Mode, logged-out, single snapshot, 10 topics × top-10 organic candidate pool per query.

GeoParrot is a GEO Lab: we test what AI search actually cites and recommends, then publish the method and the misses — including our own. This is a Tuesday pre-registration in a four-part weekly arc. Monday found the missing control group; Wednesday we collect; Thursday the data; Friday the verdict.