The verdict: ❌ Rejected — and it’s worse than “overstated.” Monday’s source check found that a widely-repeated “quality score predicts 4.2x more AI citations” study never built a non-cited control group. Tuesday we pre-registered five falsifiable predictions to build the control group ourselves — and we bet against the marketed claim going in, predicting hygiene would turn out to be a modest, secondary correlate riding on top of domain authority. Even that skepticism undersold it. Across 74 candidate pages on Google AI Mode, cited pages scored lower hygiene (G=0.55) than pages sitting right next to them that weren’t cited (G=0.74). High-hygiene pages (G≥0.70) were cited at an odds ratio of 0.22 — not a weaker version of “4.2x,” but its mirror image. Two of five predictions were confirmed, one failed in the opposite direction from what we forecast, one came back untested, and one landed in a gray zone that opens next week’s question. Full scoring below.
Friday verdict — published October 16, 2026, closing this week’s GEO Lab arc. Monday we traced a “4.2x” quality-score claim to a study with no non-cited control group. Tuesday we pre-registered the control group it was missing, with five locked predictions. Thursday we published the per-topic numbers. Today we score all five against what we actually found — including the parts where our own skepticism wasn’t cynical enough.
What’s the verdict on each of the five locked predictions?
Five numbered, falsifiable predictions went out Tuesday, before a single page was scored. Here’s how each one actually landed:
| Prediction | Verdict | What the data showed |
|---|---|---|
| P1 — Cited pages score higher, gap <0.15 (load-bearing) | ❌ Failed, reversed | Predicted a small positive gap. Actual gap: −0.19 (cited G=0.55 vs. non-cited G=0.74). Not undershoot — wrong sign entirely. |
| P2 — Authority tier beats hygiene; controlling for tier halves the P1 gap | ⚠️ Partial | Tier citation rate was 1.8x non-tier (36% vs. 20%), just under our 2x bar. Controlling for tier cut the gap ~40% (−0.19 → −0.12), close to but under “half.” Unpredicted: within the tier, the gap widened to −0.27. |
| P3 — GEO-16’s narrower finding (cross-engine > single-engine G) replicates | ❓ Untested | ChatGPT’s citation UI broke for 6 of 10 topics (ad links mixed into real ones, zero crawlable links on informational queries). Cross-engine domain overlap: 3. Too small to test — reported as untested, not forced. |
| P4 — Non-cited beats cited on hygiene in ≥6 of 10 topics | ✅ Confirmed, exceeded | 8 of 10 topics showed the paradox, two more than our pre-registered floor. |
| P5 — G≥0.70 odds ratio stays under 2x (nowhere near the claimed 4.2x) | ✅ Confirmed, exceeded | Actual odds ratio: 0.22. Technically satisfies “<2x,” but by inverting rather than merely shrinking — more extreme than what we bet. |
Net ruling: rejected, and more decisively than our own pre-registered skepticism expected. Tuesday’s design was already built to doubt the marketed claim — we predicted hygiene would be a minor, authority-riding correlate, not a strong independent lever. Reality went further than our doubt: on the one engine we could cleanly measure, hygiene didn’t just fail to predict citation, it predicted the opposite. P4 and P5 counted as “confirmed” only because we’d pre-registered conservative thresholds (≥6 topics, <2x odds) that a reversal also satisfies — worth being honest about, since a prediction that’s “confirmed” by something more extreme than you bet isn’t quite a clean win either.
Does this kill GEO-16’s original claim, or just the marketed version of it?
Just the marketed version — and that distinction is the whole point of running the claim through last week’s Reachable/Methodology-disclosed/Stable check first. The GEO-16 paper’s own, narrower finding — that among pages already cited by at least one engine, the ones picked up by a second engine score higher quality than the ones cited by only one — is a different claim than “quality predicts citation,” and our data can neither confirm nor refute it this week (P3, untested). What’s rejected is specifically the version every stats round-up actually repeats: “score 0.70+ and your odds of getting cited go up 4.2x” compared against pages that never got cited. That’s the comparison GEO-16’s own 1,100-URL sample couldn’t run, the comparison we built on purpose, and the comparison that came back negative.
So the honest two-line summary: Monday’s diagnosis was correct — a study with no non-cited control group can’t support a citation-odds claim, and once we built that control group, the claim didn’t survive contact with it. Our own Tuesday hedge was too generous — we assumed hygiene would still carry some positive, if modest, signal once authority was accounted for. It didn’t. Authority (Reddit’s 21% share, authority-tier brands’ 36% share) explains some of the gap, but even inside the authority tier, higher-hygiene pages were cited less. That’s not “hygiene is a weak lever.” That’s “on this data, hygiene and citation move in opposite directions,” which is a stranger and more useful result than either the marketed claim or our own prior would have predicted.
What should you actually do with a GEO quality score, given this?
The result kills a shortcut (score 0.70+, harvest citations), not every use of a quality checklist. Here’s the checklist we’d hand a team Monday morning, split by where the data actually splits:
Google AI Mode
- Stop treating a hygiene/quality score as a citation lever. On our data it predicted the wrong pages. If you’re choosing between shipping a structurally cleaner page and getting coverage on Reddit or an authority-tier outlet, the numbers say the second move wins more often.
- Audit whether your “SEO-clean” page is competing against forum threads and brand homepages your scorer can’t see properly. Reddit’s citations scored G=0.00 under static fetch not because the content is bad, but because the format (forum thread, client-rendered) breaks page-level hygiene scoring entirely. If your competitive set includes Reddit, a hygiene checklist is measuring the wrong unit of content.
- Keep schema, dates, and structure as table stakes, not a scoreboard. The floor argument (clean metadata, legible structure) still has no evidence against it here — it just isn’t what separated the 39 citations we logged.
ChatGPT (web search)
- Don’t use this week’s hygiene verdict for ChatGPT — we don’t have clean data for it. P3 is untested, not negative. Treat any “quality score predicts ChatGPT citation” claim with the same skepticism we applied to GEO-16’s original claim, since nobody ran this comparison cleanly there either.
- If you’re auditing your own ChatGPT citation data, filter ad click-throughs before scoring anything. We found Criteo redirects and utm_source=openai links mixed into the same DOM pattern as genuine “official website” citations on 4 of 6 product-comparison topics — treat every ChatGPT citation export as needing that filter first.
Reading any “quality score → Nx citations” claim from here on
- Ask where the non-cited pages in the sample came from. If every scored URL was already cited by something, the study can only compare degrees of citation, not citation versus none — exactly Monday’s finding about GEO-16, now empirically reinforced rather than just logically argued.
- Treat R/M/S (reachable, methodology-disclosed, stable) as necessary, not sufficient. GEO-16 passed all three checks cleanly and still couldn’t support its marketed headline. A citation-chain audit catches laundered numbers; it doesn’t catch a transparent study answering a narrower question than its marketing implies.
What’s next week’s question?
This week’s biggest confound is also its best lead: Reddit and authority-tier brands together explain well over half of what Google AI Mode cited, and both groups score worse on hygiene than the editorial pages they beat. That raises an obvious follow-up we didn’t control for this round — if you strip Reddit and authority-tier domains out entirely and compare only matched editorial content against other editorial content, does a hygiene score come back to life as a predictor? It’s possible our checklist isn’t wrong, it’s just being swamped by two variables (platform trust, brand trust) it was never built to compete with. Next week we test hygiene inside a cleaner lane instead of the open field.
That closes the loop Week 16 opened and this week ran all the way through: a well-disclosed, R/M/S-clean study’s marketed claim did not survive a real control group, and our own pre-registered hedge against it still wasn’t pessimistic enough.
Frequently asked questions
Does a GEO quality/hygiene score predict AI citation?
Not in our data. Across 74 candidate pages for 10 queries on Google AI Mode, cited pages averaged a hygiene score of 0.55 versus 0.74 for non-cited pages in the same candidate pool — the opposite of what the widely-repeated “4.2x more citations” claim implies. High-hygiene pages (G≥0.70) were cited less often, not more: odds ratio 0.22.
So was the original “4.2x” GEO-16 study wrong?
Not on its own narrower terms. The study’s actual, disclosed comparison — quality score among pages cited by multiple engines versus pages cited by only one — is untested by our data (our ChatGPT arm broke, see below), not refuted. What’s rejected is the marketed extrapolation: that the same 4.2x figure predicts getting cited at all versus never being cited, a comparison the original study’s sample couldn’t run and ours could.
Why were two of five predictions “confirmed” if the results were this negative?
Because P4 and P5 were pre-registered as conservative floors (at least 6 of 10 topics; odds ratio under 2x) that a full reversal also technically clears. The actual numbers — 8 of 10 topics, 0.22 odds ratio — exceeded those floors in a more extreme direction than we forecast, which is why we’re flagging them as “confirmed, exceeded” rather than treating them as a clean prediction win.
What happened to the ChatGPT comparison (P3)?
It came back untested, not failed. ChatGPT’s current web-search UI exposed clean citation links for only 4 of 10 topics, mixed with sponsored ad click-throughs we had to filter out, and informational queries returned zero crawlable citation links at all. The resulting cross-engine domain overlap (3 domains) was too small to test GEO-16’s narrower surviving claim with any confidence.
What’s the next GEO Lab test?
Whether a hygiene score predicts citation once Reddit and authority-tier brands — which explained most of this week’s negative gap — are removed from the comparison, leaving only matched editorial content competing against other editorial content for the same queries.
Sources
- GEO Vendors Are Citing a “4.2x More AI Citations” Quality-Score Study — Monday’s source check that opened this week’s arc.
- Cited vs. Never Cited: Our Pre-Registered Test — Tuesday’s design and the five predictions scored above.
- We Built the Missing Control Group — Thursday’s per-topic data, charts, and the Reddit/ChatGPT confounds referenced in this verdict.
- The Verdict: GEO’s Citation Chain Breaks — Just Not Where We Bet It Would — Week 16’s closing judgment and the R/M/S method applied to this week’s source claim.
- Do 15 domains control AI citations? — the authority-tier data behind this week’s P2.
- Reddit’s role in AI citations — background on the Reddit confound found in Thursday’s results.
- AI Answer Engine Citation Behavior: An Empirical Analysis of the GEO16 Framework — Kumar & Palkhouski, arXiv, the primary source for the 4.2x figure this week tested.
GeoParrot is a GEO Lab: we test what AI search actually cites and recommends, then publish the method and the misses — including our own. This closes Week 17 of a running weekly arc. Next week: does hygiene scoring predict citation once Reddit and authority-tier brands are removed from the comparison?
Leave a Reply