跳到正文
Hacker News:AI 热帖· mektrik·· 2 天前精选AI 评分76

Trakkr 研究:追踪 1285 条 AI 引用统计的溯源路径

Zombie stats: Where the numbers AI quotes come from

AI 导读

Trakkr Research 向 ChatGPT、Perplexity、Gemini、Claude 和 Google AI Overviews 提出 200 个要求数字的问题,并追踪其中 1285 条统计的引用链路,结果只有 40% 到达说法一致的研究,8% 到达但数字或语境已变,40% 无法核查,11% 无处可溯。

推荐理由

Trakkr 追踪了 1285 条 AI 引用统计的溯源路径,给出各类引用页面的到达率和数字失真比例,可帮助读者判断 AI 数字引用的可靠程度。

正文 · 原文

One number, followed.

We asked ChatGPT, Perplexity, Gemini, Claude and Google’s AI Overviews 200 questions that call for a number, the kind people ask every day. Then we clicked the citations, and kept clicking. Here is one trail, from the answer to the end.

Google AI Overviews

What percentage of salespeople give up after one follow-up?

44% of salespeople give up after just one follow-up attempt.

  • 44%
  • 44%
  • 44%
  • 44%
  • 44%

A sales firm’s stats page2020

44% of salespeople give up after one follow-up attempt (source).

A freelance writer’s sample articleby 2017

80 Percent of Sales Require 5 Follow-up Calls after Meeting; 44 Percent Salespeople Give Up After One Follow-up

No study behind itNo survey, no sample, no date.

  1. In the answer

    Every assistant got the same question: “What percentage of salespeople give up after one follow-up?” All five assistants gave the same figure, and only ChatGPT said the evidence behind it was unclear.

  2. One click in

    Google AI Overviews cited a sales firm’s stats page, from 2020. The number is right there. Its “source” link goes to a writing sample.

  3. Two clicks in

    That leads to a freelance writer’s sample article, online by 2017. The number is in its headline. It lists two slide decks, a blog list and two news stories. None is a study of follow-ups.

  4. The end of the trail

    That’s where it ends. No survey, no sample, no study: every other route to this number we tried stopped at a page that cites no research. It’s a number still walking around with nothing underneath it. That’s a zombie stat.

Most numbers aren’t zombies. But you can’t tell from the answer: a zombie arrives with the same confidence and the same citation, and here, with all five assistants agreeing. So we did the same for 1,285 statistics.

Fewer than half get there.

We took a random sample of the statistics each assistant cited, 1,285 in all, and walked every trail the same way: the cited page, then that page’s source, up to five pages deep. Where a page named a source without a link, we searched for it.

40% reached a study that says the same thing. Another 8% reached the study, but the number or its context had changed on the way. The rest never got there: 40% stopped at a source we couldn’t check, because it was locked, gone or silent on the number, and 11% led nowhere at all.

Four in ten trails reached a study that says the same thing

Each square is one in a hundred of the 1,285 statistics we traced, sorted by where its trail ended.

  • 40%SourcedWe found the study, and it says the same thing.
  • 8%ChangedWe found the study, but the number, who was asked, where or when is different.
  • 40%Can’t checkA page names a real source, but the number isn’t in it, or the source is locked or gone.
  • 11%NowhereNo study anywhere on the trail. Every page repeats the number and credits no one.

The last 1%: trails whose pages wouldn’t load for us.

Trakkr Research, five AI assistants asked 200 questions that call for a number, October 2026. A random sample of each assistant’s cited statistics, weighted by how many each quoted, and corrected with 150 trails checked by hand. Sourced: 40%, 95% range 34% to 46%.

ChatGPT did best, and quoted the fewest numbers: 1.3 per answer, against 6.8 for Claude. The other four landed between 38% and 46%, too close to rank.

ChatGPT quoted the fewest numbers, and sourced the most

The same hundred squares for each assistant, from its own sample of cited statistics.

  1. ChatGPT

    57% sourced

    1.3 statistics per answer

  2. Perplexity

    46% sourced

    2.2 statistics per answer

  3. AI Overviews

    39% sourced

    4.8 statistics per answer

  4. Gemini

    38% sourced

    6.2 statistics per answer

  5. Claude

    38% sourced

    6.8 statistics per answer

Trakkr Research, five AI assistants asked 200 questions that call for a number, October 2026. Traced per assistant: ChatGPT 108, Perplexity 277, Google AI Overviews 300, Gemini 300, Claude 300. The four after ChatGPT sit inside each other’s ranges, so they aren’t ranked.

It isn’t a quirk of our questions. Of 300 statistics in Google AI Overviews answers to real everyday searches, 40% were sourced, the same share.

AI cites the copy, not the study.

Most of the time, the AI wasn’t making numbers up: the page it cited had the figure 82% of the time. The trouble is which page. Only 21% of the pages cited for a number were the research itself.

More than half were company blogs. Add news stories and statistics roundups, and most citations land on pages built to collect figures from elsewhere, often without a link back.

Where the trail starts decides where it ends

Every page an assistant cited first for a number, by kind. Each column’s width is its share of those citations. Green is how often its trails reached a study that says the same thing.

62%

Research firm13% of citations

Research firm or survey: 13% of first citations, 62% of them sourced.

61%

Government7% of citations

Government or official body: 7% of first citations, 61% of them sourced.

37%

Company blog54% of citations

Company blog: 54% of first citations, 37% of them sourced.

37%

News10% of citations

News or magazine: 10% of first citations, 37% of them sourced.

35%

Stats roundup8% of citations

Stats roundup: 8% of first citations, 35% of them sourced.

41%

Other8% of citations

Other: 8% of first citations, 41% of them sourced.

Trakkr Research, five AI assistants asked 200 questions that call for a number, October 2026. 1,268 traced statistics with a readable first page. Page types judged from the page itself. Outcomes corrected with 150 trails checked by hand.

Where the trail starts decides where it ends. From a research firm or a government page, it reached a matching study 61% of the time. From a company blog, 37%; from a statistics roundup, 35%. Even research firms fell short 38% of the time, often because the full report sat behind a form.

A citation tells you where the AI found a number, not where the number came from.

Numbers change on the way.

Reaching the study didn’t always mean the number held. Of the statistics that got there, 17% had changed on the way. The digits moved only about one time in four. Usually it was the context: what was measured, the year, the source, or who was asked.

What changed, across every change we logged

The number changed
26%
What was measured changed
20%
The year changed
17%
The source changed
16%
Who was asked changed
14%
The place changed
4%

Fifteen women judging health websites become every website’s first impression. Three companies become a rule of business. Here are the clearest trails we followed, each one checked by hand.

Watch a number lose its context

Each trail runs from the original study to an answer an assistant gave in October 2026. Step along it: the number stays, and the words around it change.

AskedWhat percentage of first impressions of a website are design-related?

The study · Sillence and colleagues, CHI, 2004

Of the comments 15 women made about rejecting or mistrusting health websites, 94% were about design.

A 2004 study watched 15 British women judge health websites, and 94% of their comments about rejecting a site were about design. Twenty years of retelling made it “94% of first impressions are design-related”, and Google AI Overviews states it as fact.

Step 1 of 4

Every step checked by hand on the open web, with archived copies kept. We name the original studies; the pages that repeated them are described by type, not named.

The zombies.

Ask five assistants the same question and you’ll often get five different numbers. On 28 of the 71 questions where all five gave a percentage, we got four or five different figures.

One question, five answers, from 49% to 96%

Asked “What percentage of marketers use marketing automation tools?”, word for word, on the same day.

49–51%AI Overviews

62%ChatGPT

70–79%Gemini

81%Perplexity

96%Claude

  • ChatGPTThere isn’t one universal figure, it varies by survey and what counts as “marketing automation.” One commonly cited survey found 62% of U.S. digital marketers used automation software, but that data is from 2020, so it may not reflect current adoption.

  • PerplexityApproximately 81% of marketing organizations use marketing automation.

  • GeminiAcross major industry benchmarks, between 70% and 79% of marketers and companies use marketing automation tools in some capacity

  • Claude96% of marketers have used a marketing automation platform or plan to use one

  • AI OverviewsRoughly 49% to 51% of companies and marketing teams currently use marketing automation tools, according to industry benchmarks from Moosend and related research studies.

Trakkr Research, five AI assistants asked 200 questions that call for a number, October 2026. Answers collected on 25 October 2026. A range is drawn from its low to its high.

It wasn’t one odd question. The typical gap between the highest figure and the lowest was 16 points.

On 31 of 71 questions, the five figures were 20 points or more apart

Every question where all five gave a percentage, one line each, from the lowest figure to the highest. Closest at the top.

69 questions. The typical gap between the highest and lowest figure was 16 points.

Closest20 points or more apartFurthest apart

Trakkr Research, five AI assistants asked 200 questions that call for a number, October 2026. A range counts at its middle. 2 questions with figures above 100% are left off the scale.

Agreement feels like proof. It isn’t. All five gave the same figure on 10 questions, and of the 5 we traced, 2 were sourced. The statistics they repeated most, quoted by three or more, did no better: 11 of 29.

And four were quoted by all five assistants, with no study anywhere behind them.

Quoted by all five. Sourced by none.

Four statistics every assistant gave us, each with a trail that ends at a page citing no research.

  • 44%

    Salespeople who give up after one follow-up

    Every trail ends at a page that cites no research, one of them a freelance writer’s sample article. Its full trail opens chapter 1.

    Quoted by all five

  • 90%

    Social media users who follow at least one brand

    Stated with no source. No study with this figure was found.

    Quoted by all five

  • 75%

    Searchers who never scroll past the first page of results

    Repeated with no source, often credited to an old marketing roundup that no longer carries it. No study was found.

    Quoted by all five

  • 5 to 25 times

    How much more it costs to win a customer than to keep one

    A 2014 Harvard Business Review article gives the range “depending on which study you believe”, and names none.

    Quoted by all five

Trakkr Research, five AI assistants asked 200 questions that call for a number, October 2026. Statistics all five assistants quoted whose every trail ended at a page citing no research. Each trail was checked by hand on the open web.

The 29 statistics AI repeats most

Each was quoted by at least three of the five assistants, most often first, and its trail checked by hand.

  1. 95%

    Generative AI pilots that fail to deliver a measurable financial return

    MIT NANDA’s 2025 report says 95% of organizations got no return from generative AI. That isn’t the same as 95% of pilots failing.

    5 of 5ChangedStudy from 2025
  2. 44%

    Salespeople who give up after one follow-up

    Every trail ends at a page that cites no research, one of them a freelance writer’s sample article. Its full trail opens chapter 1.

    5 of 5Nowhere
  3. 4% to 10%

    Unhappy customers who actually complain

    Goes back to TARP’s 1979 research for the White House consumer office. Its author says rates that low apply to small purchases; in 2016, 22% to 41% complained about their most serious problem.

    5 of 5ChangedStudy from 1979
  4. 2 hours 21 minutes

    Time people spend on social media each day

    DataReportal’s Digital 2025 report, from GWI survey data: the typical internet user spends 2 hours 21 minutes a day on social media.

    5 of 5SourcedStudy from 2025
  5. 46%

    Google searches with local intent

    Traces to a 2018 social post about a slide at a search conference. Google never published it, as far as we could find.

    5 of 5Can’t check
  6. 90%

    Social media users who follow at least one brand

    Stated with no source. No study with this figure was found.

    5 of 5Nowhere
  7. 94%

    First impressions of a website that are about its design

    Began as a 2004 study of how 15 women judged health websites. Its full trail is in chapter 4.

    5 of 5ChangedStudy from 2004
  8. 75%

    Searchers who never scroll past the first page of results

    Repeated with no source, often credited to an old marketing roundup that no longer carries it. No study was found.

    5 of 5Nowhere
Trakkr Research, five AI assistants asked 200 questions that call for a number, October 2026. Statistics grouped by figure and meaning. Original year shown where the trail reached the study.

What to do with an AI statistic.

Treat it as a lead, not a source. Six habits, three for each side of the citation.

If you use them

  1. Follow the citation, then follow it again.

    If the cited page is a list of statistics or a company blog, use its own link to go one page further. Keep going until you reach the study itself, or until you can’t.

  2. Check who was asked, how many, and when.

    Most of the changes we found were in the context, not the digits: one country’s poll, a ten-year-old study, fifteen people.

  3. Don’t take agreement as a check.

    Five assistants giving the same number isn’t five sources. The numbers they repeated most were no better sourced than the rest.

If you publish them

  1. Put the method next to the number.

    The sample, the date and a link to the source, on the same page as the figure, so the next page, and the next answer, can trace it back to you.

  2. Keep the findings in the open.

    Reports behind a form were a common dead end: even trails that started at a research firm fell short 38% of the time. Put the headline numbers and the method on an open page.

  3. See which page AI cites for your numbers.

    It is often a roundup or a blog that copied you, sometimes with an older figure. That page decides what people hear.

来源:Hacker News:AI 热帖 · trakkr.ai