By

Dr. Simon Crawford Welch

Founder, The Critical Thought Lab

www.linkedin.com/in/simoncrawfordwelch

There’s an old saying that goes, “There are three kinds of lies: lies, damned lies, and statistics.” The phrase, often attributed to Mark Twain (who credited British Prime Minister Benjamin Disraeli), perfectly captures the tricky nature of numbers. Statistics, which should be the most reliable form of evidence, are often manipulated, twisted, and used to mislead rather than enlighten.

In an age of big data and analytics, numbers hold immense power. They influence elections, determine policies, and sell products. But here’s the catch: statistics don’t lie – people do. The way numbers are gathered, presented, or interpreted can make them tell almost any story.

The Many Ways Statistics Can Be Misleading

1. Selection Bias: Choosing Data That Fits the Narrative

Let’s say a politician wants to prove that crime has dropped under their leadership. Instead of looking at all crime rates, they select data from one particular city where crime happened to fall. The broader reality may be that crime is actually rising nationwide, but by cherry-picking numbers, they create a misleading impression.

Example: In Freakonomics, authors Steven Levitt and Stephen Dubner discuss how crime rates in the 1990s were attributed to tough-on-crime policies. In reality, the most significant factor was an unexpected one – the legalization of abortion in the 1970s, which led to fewer unwanted children being born into difficult circumstances that might have led them to crime. If you only looked at the surface-level statistics, you’d reach an entirely different (and wrong) conclusion.

2. Misleading Graphs: How Visuals Trick the Mind

Ever seen a bar graph where one bar is huge, and another is tiny – only to realize that the y-axis doesn’t start at zero? This is a classic trick used in advertising and politics. By manipulating the scale, companies and governments can make minor differences look dramatic.

Example: A toothpaste brand claims that “dentists recommend our product twice as much as the competition!” But if you look closely, the actual difference might be 51% vs. 49%. Not exactly groundbreaking.

3. The Average Trap: Mean vs. Median vs. Mode

Statistics often throw around “averages” without specifying what kind of average they mean. The mean (simple average), median (middle number), and mode (most frequently occurring number) can paint very different pictures.

Example: If Jeff Bezos walks into a coffee shop, the average wealth of customers in that shop skyrockets to billions of dollars. But the median wealth – what the middle person actually has – barely changes. Politicians and marketers use this trick to exaggerate economic improvements or business performance.

4. Sample Size Problems: When Small Numbers Cause Big Problems

If you hear that “80% of people love this new product,” that sounds impressive – until you find out that only five people were surveyed. A small sample size can lead to unreliable results, yet businesses and media outlets frequently use them.

Example: Pharmaceutical companies often conduct drug trials with small sample sizes and select the most favorable results to market their product. Only later do larger studies reveal serious side effects.

5. Confusing Correlation with Causation

One of the biggest statistical mistakes is assuming that if two things happen together, one must have caused the other. Just because ice cream sales increase when drowning rates rise does not mean ice cream causes drownings. The real cause? Summer heat leads to more swimming and more ice cream consumption.

Example: Freakonomics explores how a child’s success in school correlates with the number of books in their home. Does owning more books cause kids to be smarter? Not exactly. The real factor is parents who buy books also tend to value education and spend time teaching their children.

How Surveys Can Go Wrong (And How to Do Them Right)

Surveys are a great tool for gathering statistics, but they are ridiculously easy to manipulate. Here are some common pitfalls:

1. Leading Questions

If a survey asks, “How amazing was your experience?” instead of “How would you rate your experience?”, it’s subtly pushing respondents toward a positive answer.

2. Non-Representative Samples

If you conduct a political poll by only surveying people in one wealthy neighborhood, you’re going to get skewed results. A truly representative survey needs diversity – age, gender, location, income, etc.

3. Self-Selection Bias

People who answer surveys voluntarily are often those with extreme opinions. This is why online polls (like “Do you support this policy?”) are unreliable. The people who care the most about an issue – positively or negatively – are the ones who answer, while the majority remains silent.

4. Social Desirability Bias

People often lie on surveys to look good. If you ask, “How often do you exercise?” many people will overreport their gym visits to seem healthier. This is why professional survey designers use indirect questioning techniques to get more honest responses.

The Solution: Hire a Professional

If you’re collecting data for business, research, or decision-making, don’t DIY it. Hire a professional statistician or survey designer who understands how to eliminate bias and structure questions correctly. A well-designed survey can lead to meaningful insights, while a poorly designed one can lead to bad decisions based on false data.

Why This Matters More Than Ever

In today’s digital world, we are bombarded with statistics in the news, social media, and marketing. Understanding how statistics can be manipulated helps us think critically and make better decisions.

Before believing any statistic, ask yourself:

• Who conducted the study, and what was their motive?
• How was the data collected?
• Is there missing context?
• Could there be an alternative explanation?

As Freakonomics brilliantly shows, numbers don’t lie – but people using them often do. So next time you see a jaw-dropping statistic, pause, question, and dig deeper. Because the truth is rarely as simple as the numbers suggest.

Founder, The Critical Thought Lab. Author of “American Chasms: Essays on the Divided States of America” & “The Wisdom of Pooh: Timeless Insights for Success & Happiness” (Available on Amazon) www.linkedin.com/in/simoncrawfordwelch