Benjamin Disraeli once said - “There are three types of lies
– lies, damn lies and statistics”. Today, in the world of marketing and
business, we have reached a stage where I have to say “There are three types of
lies – damn lies, statistics and shitstics (shitty statistics)” – you read it
here for the first time…..
Marketing conferences and the business press are great
examples of the adage of “never let facts come in the way of a good story”. The
marketing world seems to very susceptible to picking on a few statistics and
create a story around it which then leads an entire movement. Most such
movements include “death” of something and there seem to be lots of “deaths” in
marketing.
I recently read about the Shirky principle which states that
“Institutions will try to preserve the problem to which they are the solution”.
Seems like it does relate to the world of marketing especially in how statistics
are used. You will often hear about the “death of Television” (I told you
there’s a lot of death in marketing) and therefore advertising dollars should
move to digital. No surprise that these articles, speeches come from those working
in the digital media world.
The spend behind digital advertising has indeed been
increasing over a period of time as consumers move online. There is no going
away from the explosive growth of digital.
However, the fact is that over a long period of time the total amount of time people spend on TV has not changed much. The TV ad industry itself grew a handsome 4.1% in the USA in 2018. What’s most interesting (in a Brit way) is that the spend by the FAANGS on TV is increasing year on year.
There are 5 different ways I see statistics being abused
1. “Concealing while revealing” - "Statistics are like Bikinis. What they reveal is suggestive, but what they conceal is vital"- Aaron Levenstein.
If you followed
the marketing press around Super bowl, you would have gone away with the
impression that TV viewership of the spectacle was tanking (a decline of 5.1%)
and that digital was where you needed to advertise (viewership increased by
31%). Let us look at the facts.
TV viewership was indeed the lowest in the
last few years. 98.2 Mn Americans viewed the Super bowl on TV in 2019 compared
to 103.5 Mn the previous year – indeed a drop of 5.1%. However, the CBS streamed event was also viewed on
7.5 Mn unique devices (increase of 20%). But if you look at people who watched
for at least 1 minute (to make it comparable to TV) then only 2.6 Mn Americans
watched it online (increase of 31%). So digital viewership was less than 3% of
TV viewership – where would you rather advertise?
In my humble opinion, the bigger picture is
that the viewership of the spectacle has declined by about 5%. Is that a sign
of changing demographics? I suspect this won’t get discussed since it won’t
make a good story.
Let me give you another example. Earlier in
the year the CEO of YouTube, Susan Wojcicki said that the number of YouTube
channels making 5 or 6 sum figures has gone up by 40%. Before you decide to
make a “career” as a “YouTuber” think hard. A professor in Germany found that
the top 3% of channels on YouTube make an average of $16,800 per year. The
odds of you making a career as a “YouTuber” with 5 or 6 sums is about as high
as you winning an Olympic gold at your current age (none of my readers will be
teenagers !!!).
Percentages are a wonderful tool to create
a shitstic!!!
Another way of “concealing while revealing”
is using misleading visualisations. We live in a world of infographics. Well
done infographics are great at telling a compelling story by simplifying the
data. But the devil lies in the detail and the devil does lie or helps the
visualizer lie.
During a questioning of the President of the
Planned parenthood a Republican senator showed the graph below and said “In
pink, that’s the reduction in the breast exams, and the red is the increase in
the abortions. That’s what’s going on in your organization.” The graph itself
gives the impression that the number of abortions is disproportionate compared
to the number of breast examinations. But look closely and carefully and you
will notice that the graph has no defined Y axis. While the number of breast
examinations has declined it is still around 3 times the number of abortions.
While the example does not relate to the world of marketing its one of the best
examples of misleading visualizations that I have come across.
2. “Abuse of surveys” – Market research surveys
will give an answer to the question that is asked to people – doesn’t mean the
questions are the right questions or that it was asked to people who will know
the answer.
In 2017, budget airline Ryanair declared
that 92% of their customers were satisfied with their experience. Having lived
in the UK I know that the airline has a reputation for being customer
unfriendly. What then, explains the high scores? Well the scale they used was –
excellent, very good, good, fair, ok”. I guess 8% refused to answer!!!
There is a postscript to this story.
When the Royal Statistical Society criticized their survey, Ryanair bosses said
“95% of Ryanair customers have not heard of the Royal Statistical Society and
97% say they don’t care what they say”. It’s the point when you go ROFL.
“Opinions are free and maybe fact free” –
If you ask people for an opinion you will get one – especially in surveys.
There is no guarantee of the fact that they have a reason to have an opinion.
Neither is there any guarantee that their opinion will triangulate with facts
in which case it will be rationalized as “perceptions are reality”.
Earlier this year Forbes reported that a
survey had showed that only 19% of adults think that “personalized ads are
ethical”. OK how many people do you know who don’t work in marketing and know
what a “personalized ad” is?
A study by Gallup showed that Americans don’t
believe there is a divide between the “haves and have nots” in America and it’s
a trend that has strengthened over the last few years. But the fact is that the
income distribution has worsened.
3. Not triangulating data – The Boston Consulting
Group put out a report estimating that 80 Mn Americans use a Augmented Reality
device. To put this number into context a recent survey by Pew Research found
that only 36% of Americans i.e about 120 Mn use a ride hailing app (Uber /
Lyft). Interestingly according to the state department only 36% of Americans
have a passport. o 120 Mn Americans use Uber, 120 Mn Americans have a password
while 80 Mn Americans use an Augmented Reality device. Is it plausible - I’ll
let you judge.
Not long ago we had a major scare in the
social media world – the Instagram audience had aged. The story was that while
the majority of Instagram users were aged 18-24 in 2012, in 2019 the majority
of the users are aged 25-34. Oh well the users have aged 7 years between 2012 and
2019. The real story then is that Instagram is not attracting teenagers anymore
but again that does not make a newsworthy story.
A study published in “BMJ Sexual &
Reproductive Health” found that among straight people, 1 in 5 men had double
digit conquests while 1 in 12 women had double digit conquests. With almost a
50-50 split between males and females how does this happen?
Postscript – the same study found that
“athletic women” were 73% more likely to have slept with at least 10 partners, than those who shun exercises. Doesn’t fit into the story but felt like one
that I could not leave out.
4. Metrics that don’t matter– The Brits often
say - “measure what you treasure and treasure what you measure” but notice that
they don’t talk of relevance or meaningfulness.
In the world of start ups there is a
massive premium placed on “how much venture capital is raised” with “Unicorn”
status being sought after. Look at some of the most successful start ups and
how much venture capital they raised – Google – $ 25Mn, Apple - $ 3.6 Mn, Intel
- $2.5 Mn, Cisco - $2.5 Mn, Facebook - $2.4 Mn. None of them were Unicorns but
you would be hard pressed to argue against the fact that all of them are
“successful”.
In the world of marketing, the “death of
creativity” (yes there is a lot of death we seem to encounter) is often
illustrated using the chart below. Yes, the TV ad enjoyability has fallen
relative to TV programs but that does not mean that TV ad enjoyability has
fallen. Remember back in the day there was hardly a handful of channels and
even fewer TV programs. The quality of most TV programs was poor, and they
survived only since there was no alternative. Over the years the quality of TV
programs has improved for sure, but I don’t think the quality of TV ads has
fallen.
5. Making bold predictions without any substantiation –
Remember how Amazon Dash buttons was going to kill conventional retail (yes
more death). Well as it turns out Amazon Dash buttons are the ones that are
dead. Remember the bold predictions on ibeacons (see below) – well don’t seem
to hear much about them today.
Probably the most “explosive” prediction which
impacted marketing was around the spending power of millennials. It was
contained in a book titled “How Teens and Twenty somethings are Revolutionising
retail”. It said, “Generation Y, those born between 1978 and 2000 has overtaken
baby boomers in sheer numbers and is poised to do the same with incomes by
2017….”. The book offered no explanation for their predictions. Alas, no such
thing has happened.
The fraternity of market research can help
the world of marketing to avoid shitstics by following the following Ten
commandments
- Thou shalt not ask misleading questions in surveys
- Thou shalt not ask survey questions to people who don’t have the ability to answer them
- Thou shalt triangulate data within a study and with other sources
- Thou shalt not commit the school boy error of confusing correlation and causation
- Thou shalt not use metrics that are inappropriate
- Thou shalt not report out meaningless statistics
- Thou shalt tell the truth and only the whole truth
- Thou shalt not mislead using visuals that conceal more than they reveal
- Thou shalt not make predictions without a solid basis
- Thou shalt not take sides but present facts
So true Vijay, you have put your finger on the pulse. Spin a story to get brownie points from boss or client has been a norm in this industry or any industry for that matter.
ReplyDeleteI love it! Especially point 1: some people/agencies are artists in using percentages and completely ignoring the absolute numbers which can lead to wrong conclusions. (my personal view)
ReplyDeleteExcellent article Vijay!!! We come across such cases of too much analysis leading to paralysis on a frequent basis... As frequent as yesterday :) :P
ReplyDeleteThis comment has been removed by the author.
ReplyDelete
ReplyDeleteGreat article Vijay, enjoyed reading it. And loved the perspective on concealing more than revealing! So true. I shall remember the commandments :)
Good point! the only way to counter a clever but misleading statistic is with an even more clever but factual statistic! story beats story. Very nice analogies- Bikini statistics etc.. one input-- you could make the 10 commandments more catchy, not as easy to remember as the other great examples
ReplyDeleteGreat article Vijay! very provocative and makes the point loud and clear...
ReplyDeleteVijay thank you for spending the time to collect data and write this. I like it not because I work in our industry but because intellectual rigor is the basis of progress.
ReplyDeleteFirstly it’s seldom that one finds the right articulation backed by sound logic - you’ve hit both. Kudos. The statistics aka shitistics do leave a lot to munch on. There was a time when a BIG IDEA would be synchronised to the media platforms and one would look at a mass movement. Today it’s all about “ eyeball” transactional, lead generation - it’s all transient. Media platforms especially DIGITAL are out to disprove Traditional media platforms rather than find a synergistic space among them. I guess it’s insecurity. A thought provoking article - hope marketeers are able to see beyond what’s thrown at them and have the gumption to resist the urge to “follow” the herd. Cheers
ReplyDeleteRings so true! There is not a single week when we do not have discussions on convenient quoting & interpretations of data! And behaviour-led cases are the most problematic (a la the Ryan Air case).
ReplyDelete