There are damn lies, there are statistics and there are shitstics (shitty statistics)

Benjamin Disraeli once said - “There are three types of lies – lies, damn lies and statistics”. Today, in the world of marketing and business, we have reached a stage where I have to say “There are three types of lies – damn lies, statistics and shitstics (shitty statistics)” – you read it here for the first time…..

Marketing conferences and the business press are great examples of the adage of “never let facts come in the way of a good story”. The marketing world seems to very susceptible to picking on a few statistics and create a story around it which then leads an entire movement. Most such movements include “death” of something and there seem to be lots of “deaths” in marketing.

I recently read about the Shirky principle which states that “Institutions will try to preserve the problem to which they are the solution”. Seems like it does relate to the world of marketing especially in how statistics are used. You will often hear about the “death of Television” (I told you there’s a lot of death in marketing) and therefore advertising dollars should move to digital. No surprise that these articles, speeches come from those working in the digital media world.

The spend behind digital advertising has indeed been increasing over a period of time as consumers move online. There is no going away from the explosive growth of digital.

However, the fact is that over a long period of time the total amount of time people spend on TV has not changed much. The TV ad industry itself grew a handsome 4.1% in the USA in 2018. What’s most interesting (in a Brit way) is that the spend by the FAANGS on TV is increasing year on year.

There are 5 different ways I see statistics being abused 

1. “Concealing while revealing” - "Statistics are like Bikinis. What they reveal is suggestive, but what they conceal is vital"- Aaron Levenstein. 

If you followed the marketing press around Super bowl, you would have gone away with the impression that TV viewership of the spectacle was tanking (a decline of 5.1%) and that digital was where you needed to advertise (viewership increased by 31%). Let us look at the facts.

TV viewership was indeed the lowest in the last few years. 98.2 Mn Americans viewed the Super bowl on TV in 2019 compared to 103.5 Mn the previous year – indeed a drop of  5.1%. However, the CBS streamed event was also viewed on 7.5 Mn unique devices (increase of 20%). But if you look at people who watched for at least 1 minute (to make it comparable to TV) then only 2.6 Mn Americans watched it online (increase of 31%). So digital viewership was less than 3% of TV viewership – where would you rather advertise?

In my humble opinion, the bigger picture is that the viewership of the spectacle has declined by about 5%. Is that a sign of changing demographics? I suspect this won’t get discussed since it won’t make a good story.

Let me give you another example. Earlier in the year the CEO of YouTube, Susan Wojcicki said that the number of YouTube channels making 5 or 6 sum figures has gone up by 40%. Before you decide to make a “career” as a “YouTuber” think hard. A professor in Germany found that the top 3% of channels on YouTube make an average of $16,800 per year. The odds of you making a career as a “YouTuber” with 5 or 6 sums is about as high as you winning an Olympic gold at your current age (none of my readers will be teenagers !!!).

Percentages are a wonderful tool to create a shitstic!!!

Another way of “concealing while revealing” is using misleading visualisations. We live in a world of infographics. Well done infographics are great at telling a compelling story by simplifying the data. But the devil lies in the detail and the devil does lie or helps the visualizer lie.

During a questioning of the President of the Planned parenthood a Republican senator showed the graph below and said “In pink, that’s the reduction in the breast exams, and the red is the increase in the abortions. That’s what’s going on in your organization.” The graph itself gives the impression that the number of abortions is disproportionate compared to the number of breast examinations. But look closely and carefully and you will notice that the graph has no defined Y axis. While the number of breast examinations has declined it is still around 3 times the number of abortions. While the example does not relate to the world of marketing its one of the best examples of misleading visualizations that I have come across. 

2. “Abuse of surveys” – Market research surveys will give an answer to the question that is asked to people – doesn’t mean the questions are the right questions or that it was asked to people who will know the answer.

In 2017, budget airline Ryanair declared that 92% of their customers were satisfied with their experience. Having lived in the UK I know that the airline has a reputation for being customer unfriendly. What then, explains the high scores? Well the scale they used was – excellent, very good, good, fair, ok”. I guess 8% refused to answer!!! 

There is a postscript to this story. When the Royal Statistical Society criticized their survey, Ryanair bosses said “95% of Ryanair customers have not heard of the Royal Statistical Society and 97% say they don’t care what they say”. It’s the point when you go ROFL.

“Opinions are free and maybe fact free” – If you ask people for an opinion you will get one – especially in surveys. There is no guarantee of the fact that they have a reason to have an opinion. Neither is there any guarantee that their opinion will triangulate with facts in which case it will be rationalized as “perceptions are reality”.

Earlier this year Forbes reported that a survey had showed that only 19% of adults think that “personalized ads are ethical”. OK how many people do you know who don’t work in marketing and know what a “personalized ad” is?

A study by Gallup showed that Americans don’t believe there is a divide between the “haves and have nots” in America and it’s a trend that has strengthened over the last few years. But the fact is that the income distribution has worsened. 

3. Not triangulating data – The Boston Consulting Group put out a report estimating that 80 Mn Americans use a Augmented Reality device. To put this number into context a recent survey by Pew Research found that only 36% of Americans i.e about 120 Mn use a ride hailing app (Uber / Lyft). Interestingly according to the state department only 36% of Americans have a passport. o 120 Mn Americans use Uber, 120 Mn Americans have a password while 80 Mn Americans use an Augmented Reality device. Is it plausible - I’ll let you judge.

Not long ago we had a major scare in the social media world – the Instagram audience had aged. The story was that while the majority of Instagram users were aged 18-24 in 2012, in 2019 the majority of the users are aged 25-34. Oh well the users have aged 7 years between 2012 and 2019. The real story then is that Instagram is not attracting teenagers anymore but again that does not make a newsworthy story.

A study published in “BMJ Sexual & Reproductive Health” found that among straight people, 1 in 5 men had double digit conquests while 1 in 12 women had double digit conquests. With almost a 50-50 split between males and females how does this happen? 

Postscript – the same study found that “athletic women” were 73% more likely to have slept with at least 10 partners, than those who shun exercises. Doesn’t fit into the story but felt like one that I could not leave out.

4. Metrics that don’t matter– The Brits often say - “measure what you treasure and treasure what you measure” but notice that they don’t talk of relevance or meaningfulness.

In the world of start ups there is a massive premium placed on “how much venture capital is raised” with “Unicorn” status being sought after. Look at some of the most successful start ups and how much venture capital they raised – Google – $ 25Mn, Apple - $ 3.6 Mn, Intel - $2.5 Mn, Cisco - $2.5 Mn, Facebook - $2.4 Mn. None of them were Unicorns but you would be hard pressed to argue against the fact that all of them are “successful”.

In the world of marketing, the “death of creativity” (yes there is a lot of death we seem to encounter) is often illustrated using the chart below. Yes, the TV ad enjoyability has fallen relative to TV programs but that does not mean that TV ad enjoyability has fallen. Remember back in the day there was hardly a handful of channels and even fewer TV programs. The quality of most TV programs was poor, and they survived only since there was no alternative. Over the years the quality of TV programs has improved for sure, but I don’t think the quality of TV ads has fallen.  

5. Making bold predictions without any substantiation – Remember how Amazon Dash buttons was going to kill conventional retail (yes more death). Well as it turns out Amazon Dash buttons are the ones that are dead. Remember the bold predictions on ibeacons (see below) – well don’t seem to hear much about them today.

Probably the most “explosive” prediction which impacted marketing was around the spending power of millennials. It was contained in a book titled “How Teens and Twenty somethings are Revolutionising retail”. It said, “Generation Y, those born between 1978 and 2000 has overtaken baby boomers in sheer numbers and is poised to do the same with incomes by 2017….”. The book offered no explanation for their predictions. Alas, no such thing has happened.

The fraternity of market research can help the world of marketing to avoid shitstics by following the following Ten commandments

1. Thou shalt not ask misleading questions in surveys
2. Thou shalt not ask survey questions to people who don’t have the ability to answer them
3. Thou shalt triangulate data within a study and with other sources
4. Thou shalt not commit the school boy error of confusing correlation and causation
5. Thou shalt not use metrics that are inappropriate
6. Thou shalt not report out meaningless statistics
7. Thou shalt tell the truth and only the whole truth
8. Thou shalt not mislead using visuals that conceal more than they reveal
9. Thou shalt not make predictions without a solid basis
10. Thou shalt not take sides but present facts