How to spot a fake statistic

Most nonsense numbers fail an estimate you can do in ten seconds. Here is the checking routine, and why nobody applies it.

Zoggle · a guide for people who like numbers

A statistic arrives with a kind of authority. It has digits in it; someone presumably counted something. That authority is why bad numbers travel so well: almost nobody checks, because checking feels like it requires access to data you don’t have.

It usually doesn’t. Most impossible claims can be dismantled with the numbers already in your head, in less time than it takes to share the post.

Check 1: divide by the population

The single most productive move. Take any national or global total and reduce it to per person.

“Americans throw away 100 billion plastic bags a year” — divide by 350 million people and you get roughly 285 bags per person per year, or about five a week. Plausible. It survives.

“This industry loses $500 billion a year to theft” — that’s $1,400 per American, which for most industries is implausibly large. Now you have a reason to be sceptical, not just a feeling.

Check 2: convert to a rate

Big totals become tractable when you break them into time.

A claim that a service “handles 5 billion transactions a day” means about 58,000 per second, every second, all day. That is an enormous engineering feat — achievable by a handful of companies on Earth, and a red flag for anyone else. Conversely, a charity claiming to have “helped 10 million people” with a staff of thirty means each employee helped 333,000 people, which invites the question of what “helped” means.

Check 3: does the base rate allow it?

Every claim about a subgroup is capped by the size of that subgroup. If a report says 3 million American teenagers do something, note that there are only about 17 million Americans aged 15–19 — so the claim asserts nearly one in six. Sometimes that’s true. Often it reveals that the study defined its terms very loosely.

The version worth watching for is when several such claims are made together and the fractions sum past 100%.

Check 4: follow the units and the timeframe

An enormous share of misleading statistics are technically true numbers wearing the wrong label.

• Total versus annual. “$2 trillion in damage” means something very different over one year than over thirty.
• Per person versus per household. A factor of about 2.5 in most countries, frequently swapped to make a figure look bigger.
• Power versus energy. Kilowatts and kilowatt-hours are different quantities; conflating them is the most common error in energy reporting.
• Percentage points versus percent. Rising from 2% to 3% is one percentage point, or a 50% increase. Both are honest; only one is usually meant.

Check 5: ask how anyone could know

Some numbers fail not on arithmetic but on epistemology. Before believing a figure, ask what act of measurement produced it.

Claims about undocumented activity — informal economies, undiagnosed conditions, private behaviour — are estimates layered on estimates, and legitimately uncertain. That doesn’t make them wrong. It does mean a figure quoted to three significant figures is projecting a confidence the method cannot support.

The related tell is the number with no origin. When a striking statistic circulates for years with no cited source, it is frequently a mangled version of something narrower — a study of one town, one year, one age group, stripped of every qualifier that made it true.

What to do when a number fails

Failing your check does not prove a statistic false. It means the claim has earned a look. Sometimes the resolution is interesting: the number is real, and your model of the world was missing something. That is the best possible outcome, and it only happens if you do the check.

The goal is not scepticism about everything. It is the ability to tell, in seconds, which numbers deserve a second look — so your doubt lands where it belongs instead of everywhere or nowhere.

This is why estimation is worth practising even if you never need to count piano tuners. Every day, numbers are presented to you by people who would like you to believe something. A ten-second check is the cheapest defence available.