We Asked 4 AI Engines the Same 15 Questions — Only 0.5% of Sources Overlapped

Quick answer: We asked ChatGPT Search, Perplexity, Google AI Mode, and Gemini the same 15 buyer-intent questions and logged every cited domain. Result: of 386 unique domains, only 2 (0.5%) were cited by all four engines, and 86.8% appeared on just one engine. The closest engine pair overlapped only 8.2%. The data is blunt: there is no shared pool of “trusted sources” you can optimize for once and win everywhere. We pre-registered four predictions before collecting — two landed, and one we got plainly wrong.

On Tuesday we published the full method and locked in four predictions before touching the data — the whole point being that you could hold us to them. We’ve now run the collection. Here are the raw numbers, scored against what we said would happen.

How different are the sources, really?

We reduced every cited link to its root domain (e.g. reddit.com) and counted how many of the four engines cited each one. If a universal GEO strategy existed, a big chunk of domains would be cited by most or all engines. Instead:

Bar chart: 86.8% of 386 cited domains appear on only one engine, 11.9% on two, 0.8% on three, 0.5% on all four
  • 86.8% of domains (335 of 386) were cited by only one engine.
  • 11.9% appeared on two engines.
  • 0.8% on three.
  • 0.5% — just two domains — were cited by all four: reddit.com and semrush.com.

Read that last line again. Out of 386 sources the engines reached for, exactly two were common ground. Everything else is engine-specific territory.

Which engines are most alike?

Pairwise Jaccard measures overlap between two engines (shared domains ÷ combined domains). Even the most similar pair barely cleared 8%:

Pairwise Jaccard overlap: Google AI Mode and Gemini 8.2%, Perplexity-Google 5.9%, ChatGPT-Gemini 5.2%, Perplexity-Gemini 4.4%, ChatGPT-Google 3.5%, ChatGPT-Perplexity 1.8%
  • Google AI Mode ∩ Gemini: 8.2% — the closest pair, and unsurprising given both are Google products.
  • ChatGPT ∩ Perplexity: 1.8% — the most divergent. These two share almost nothing.

For context, the industry benchmark we cited on Monday — Averi’s 680-million-citation study — put ChatGPT–Perplexity overlap at ~11%. Our small, hand-run test came in even lower. The direction matches; the divergence is, if anything, starker on buyer-intent queries.

Where our prediction broke: who actually leans on Reddit?

This is the one we got wrong, and it’s the most interesting result of the week. We predicted Reddit would be heavy on Perplexity and near-zero on Google AI Mode and Gemini. The data said the opposite:

Reddit dependence by engine: Google AI Mode 40%, Gemini 38%, ChatGPT 27%, Perplexity only 9%
  • Google AI Mode: 40% of questions cited Reddit — the highest.
  • Gemini: 38%.
  • ChatGPT: 27%.
  • Perplexity: 9% — the lowest, the exact engine we expected to lean hardest on Reddit.

Why the flip? The likeliest read: Google has spent two years surfacing Reddit through its search partnership, and AI Mode inherits that index — so its AI answers pull community threads heavily. Perplexity, on these commercial queries, leaned more on vendor and review sites than forums. We pre-registered the prediction precisely so we couldn’t quietly bury this miss. It’s on the record because the method put it there.

How did all four predictions score?

# Pre-registered prediction Result Verdict
1 Under 15% of domains cited by all four engines 0.5% ✅ Hit
2 Over 70% of domains unique to one engine 86.8% ✅ Hit
3 Reddit heavy on Perplexity, near-zero on Google/Gemini Reddit highest on Google (40%), lowest on Perplexity (9%) ❌ Wrong — reversed
4 ChatGPT cites the fewest distinct domains Perplexity had the smallest pool (56 vs ChatGPT 59) ⚠️ Inconclusive*

*Perplexity and Gemini throttled mid-collection, completing 11 and 13 of 15 questions respectively (ChatGPT and Google got all 15). Fewer questions means a smaller domain pool, so the “fewest domains” comparison isn’t clean. We’re flagging it rather than claiming it — and we’ll re-run those two engines for Friday’s verdict.

What’s the honest takeaway from the data?

The headline numbers — 0.5% all-engine overlap, 86.8% single-engine, sub-9% best pair — point hard in one direction: engines cite mostly different sources, so a single “optimize once” GEO playbook doesn’t transfer. That’s consistent with our first 3-engine test (84% single-engine) and with the 680M-citation industry study. But the Reddit miss is a warning to ourselves: per-engine behavior doesn’t always match the conventional wisdom, and the only way to know your niche is to measure it. We’re not delivering the final verdict here — that’s Friday, where we’ll fold in the re-run of the throttled engines and answer the week’s question: is “no universal GEO strategy” confirmed by our own data, or do we walk it back?

FAQ

Is a 0.5% overlap really meaningful with only 15 questions?
It’s a small sample by design — small enough to run transparently by hand. But the direction matches a 680-million-citation study, so the signal is real even if the exact percentage would move with more questions. Treat it as “the pattern is strong,” not “0.5% to the decimal.”

Why did Perplexity and Gemini only answer 11 and 13 questions?
Both throttled rate-limited our collection after a burst of rapid queries — a limitation we’d already flagged in the method. We report the partial coverage openly rather than padding the gaps, and we’ll re-collect the missing questions before Friday’s verdict.

Does this mean Reddit is the one source worth optimizing for?
Reddit and Semrush were the only two domains cited by all four engines, so genuine community presence helps broadly. But Reddit’s weight swings from 9% (Perplexity) to 40% (Google AI Mode), so it’s no silver bullet — it depends heavily on which engine your buyers use.

Can I replicate this?
Yes. The 15 questions, four engines, and exact metrics are all in the method post. Run it on your own niche — you’ll almost certainly see the same per-engine divergence.

Sources

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *