There are damn lies, there are statistics and there are shitstics (shitty statistics)

Benjamin Disraeli once said - “There are three types of lies – lies, damn lies and statistics”. Today, in the world of marketing and business, we have reached a stage where I have to say “There are three types of lies – damn lies, statistics and shitstics (shitty statistics)” – you read it here for the first time…..

Marketing conferences and the business press are great examples of the adage of “never let facts come in the way of a good story”. The marketing world seems to very susceptible to picking on a few statistics and create a story around it which then leads an entire movement. Most such movements include “death” of something and there seem to be lots of “deaths” in marketing.

I recently read about the Shirky principle which states that “Institutions will try to preserve the problem to which they are the solution”. Seems like it does relate to the world of marketing especially in how statistics are used. You will often hear about the “death of Television” (I told you there’s a lot of death in marketing) and therefore advertising dollars should move to digital. No surprise that these articles, speeches come from those working in the digital media world.

The spend behind digital advertising has indeed been increasing over a period of time as consumers move online. There is no going away from the explosive growth of digital.

However, the fact is that over a long period of time the total amount of time people spend on TV has not changed much. The TV ad industry itself grew a handsome 4.1% in the USA in 2018. What’s most interesting (in a Brit way) is that the spend by the FAANGS on TV is increasing year on year.

There are 5 different ways I see statistics being abused 

1. “Concealing while revealing” - "Statistics are like Bikinis. What they reveal is suggestive, but what they conceal is vital"- Aaron Levenstein. 

If you followed the marketing press around Super bowl, you would have gone away with the impression that TV viewership of the spectacle was tanking (a decline of 5.1%) and that digital was where you needed to advertise (viewership increased by 31%). Let us look at the facts.

TV viewership was indeed the lowest in the last few years. 98.2 Mn Americans viewed the Super bowl on TV in 2019 compared to 103.5 Mn the previous year – indeed a drop of  5.1%. However, the CBS streamed event was also viewed on 7.5 Mn unique devices (increase of 20%). But if you look at people who watched for at least 1 minute (to make it comparable to TV) then only 2.6 Mn Americans watched it online (increase of 31%). So digital viewership was less than 3% of TV viewership – where would you rather advertise?

In my humble opinion, the bigger picture is that the viewership of the spectacle has declined by about 5%. Is that a sign of changing demographics? I suspect this won’t get discussed since it won’t make a good story.

Let me give you another example. Earlier in the year the CEO of YouTube, Susan Wojcicki said that the number of YouTube channels making 5 or 6 sum figures has gone up by 40%. Before you decide to make a “career” as a “YouTuber” think hard. A professor in Germany found that the top 3% of channels on YouTube make an average of $16,800 per year. The odds of you making a career as a “YouTuber” with 5 or 6 sums is about as high as you winning an Olympic gold at your current age (none of my readers will be teenagers !!!).

Percentages are a wonderful tool to create a shitstic!!!

Another way of “concealing while revealing” is using misleading visualisations. We live in a world of infographics. Well done infographics are great at telling a compelling story by simplifying the data. But the devil lies in the detail and the devil does lie or helps the visualizer lie.

During a questioning of the President of the Planned parenthood a Republican senator showed the graph below and said “In pink, that’s the reduction in the breast exams, and the red is the increase in the abortions. That’s what’s going on in your organization.” The graph itself gives the impression that the number of abortions is disproportionate compared to the number of breast examinations. But look closely and carefully and you will notice that the graph has no defined Y axis. While the number of breast examinations has declined it is still around 3 times the number of abortions. While the example does not relate to the world of marketing its one of the best examples of misleading visualizations that I have come across. 

2. “Abuse of surveys” – Market research surveys will give an answer to the question that is asked to people – doesn’t mean the questions are the right questions or that it was asked to people who will know the answer.

In 2017, budget airline Ryanair declared that 92% of their customers were satisfied with their experience. Having lived in the UK I know that the airline has a reputation for being customer unfriendly. What then, explains the high scores? Well the scale they used was – excellent, very good, good, fair, ok”. I guess 8% refused to answer!!! 

There is a postscript to this story. When the Royal Statistical Society criticized their survey, Ryanair bosses said “95% of Ryanair customers have not heard of the Royal Statistical Society and 97% say they don’t care what they say”. It’s the point when you go ROFL.

“Opinions are free and maybe fact free” – If you ask people for an opinion you will get one – especially in surveys. There is no guarantee of the fact that they have a reason to have an opinion. Neither is there any guarantee that their opinion will triangulate with facts in which case it will be rationalized as “perceptions are reality”.

Earlier this year Forbes reported that a survey had showed that only 19% of adults think that “personalized ads are ethical”. OK how many people do you know who don’t work in marketing and know what a “personalized ad” is?

A study by Gallup showed that Americans don’t believe there is a divide between the “haves and have nots” in America and it’s a trend that has strengthened over the last few years. But the fact is that the income distribution has worsened. 

3. Not triangulating data – The Boston Consulting Group put out a report estimating that 80 Mn Americans use a Augmented Reality device. To put this number into context a recent survey by Pew Research found that only 36% of Americans i.e about 120 Mn use a ride hailing app (Uber / Lyft). Interestingly according to the state department only 36% of Americans have a passport. o 120 Mn Americans use Uber, 120 Mn Americans have a password while 80 Mn Americans use an Augmented Reality device. Is it plausible - I’ll let you judge.

Not long ago we had a major scare in the social media world – the Instagram audience had aged. The story was that while the majority of Instagram users were aged 18-24 in 2012, in 2019 the majority of the users are aged 25-34. Oh well the users have aged 7 years between 2012 and 2019. The real story then is that Instagram is not attracting teenagers anymore but again that does not make a newsworthy story.

A study published in “BMJ Sexual & Reproductive Health” found that among straight people, 1 in 5 men had double digit conquests while 1 in 12 women had double digit conquests. With almost a 50-50 split between males and females how does this happen? 

Postscript – the same study found that “athletic women” were 73% more likely to have slept with at least 10 partners, than those who shun exercises. Doesn’t fit into the story but felt like one that I could not leave out.

4. Metrics that don’t matter– The Brits often say - “measure what you treasure and treasure what you measure” but notice that they don’t talk of relevance or meaningfulness.

In the world of start ups there is a massive premium placed on “how much venture capital is raised” with “Unicorn” status being sought after. Look at some of the most successful start ups and how much venture capital they raised – Google – $ 25Mn, Apple - $ 3.6 Mn, Intel - $2.5 Mn, Cisco - $2.5 Mn, Facebook - $2.4 Mn. None of them were Unicorns but you would be hard pressed to argue against the fact that all of them are “successful”.

In the world of marketing, the “death of creativity” (yes there is a lot of death we seem to encounter) is often illustrated using the chart below. Yes, the TV ad enjoyability has fallen relative to TV programs but that does not mean that TV ad enjoyability has fallen. Remember back in the day there was hardly a handful of channels and even fewer TV programs. The quality of most TV programs was poor, and they survived only since there was no alternative. Over the years the quality of TV programs has improved for sure, but I don’t think the quality of TV ads has fallen.  

5. Making bold predictions without any substantiation – Remember how Amazon Dash buttons was going to kill conventional retail (yes more death). Well as it turns out Amazon Dash buttons are the ones that are dead. Remember the bold predictions on ibeacons (see below) – well don’t seem to hear much about them today.

Probably the most “explosive” prediction which impacted marketing was around the spending power of millennials. It was contained in a book titled “How Teens and Twenty somethings are Revolutionising retail”. It said, “Generation Y, those born between 1978 and 2000 has overtaken baby boomers in sheer numbers and is poised to do the same with incomes by 2017….”. The book offered no explanation for their predictions. Alas, no such thing has happened.

The fraternity of market research can help the world of marketing to avoid shitstics by following the following Ten commandments

1. Thou shalt not ask misleading questions in surveys
2. Thou shalt not ask survey questions to people who don’t have the ability to answer them
3. Thou shalt triangulate data within a study and with other sources
4. Thou shalt not commit the school boy error of confusing correlation and causation
5. Thou shalt not use metrics that are inappropriate
6. Thou shalt not report out meaningless statistics
7. Thou shalt tell the truth and only the whole truth
8. Thou shalt not mislead using visuals that conceal more than they reveal
9. Thou shalt not make predictions without a solid basis
10. Thou shalt not take sides but present facts

Long Live Insights – Its now time for “Outsights”

It’s the time of the year when its fashionable to ring out the old and bring in the new. So here’s my two bit for the market research industry – its time to ring out Insights and embrace a new idea called “outsights”.

The market research industry, indeed all of marketing, has been obsessed with “consumer insights”. Look at the stupendous increase in the mention of the term consumer insights in books since the beginning of the 1990s.

However, the market research industry needs to pivot away from insights for three reasons

1. With signs of the global economy faltering, growth will becoming increasingly difficult and will be predicated on truly differentiated insights
2. Big data has done wonders for the industry but has “dehumanized” the data. We need to “feel the consumers” rather than read about them.
3. With growth becoming difficult, we need to go beyond insights to ideas that could translate those insights to growth

It is my submission that the research industry needs to pivot away from Insight to Outsight. Humor me and read on as I give you three good reasons (good in my judgement 😊) for my view

1. "Confusion over what is an insight” – Probably the most common question asked in any job interview in the market research industry – client or agency side – is “What is your definition of what is an insight”. Look at the search trend for the term “what is an insight” and it shows a steady increase. 

These two facts put together would suggest that there is a level of confusion as to what exactly is an insight. This is manifested in many a discussion around “what is the real insight” or “is that really an insight”.

As a consequence of this lack of clarity on what is an insight, “Insights” have a tendency to be “obvious” or in other words “in plain sight”. This makes them unactionable. Here is one example that was floating on LinkedIn over the holiday period.

Disclaimer – I have cropped out the name of the agency since this is about quoting an example and not debating the individual case.

Is it an insight that disruptive brands are “different from the others in the category”? If you use the definition wherein an insight involves a causality then yes it is an insight. 

Is it not obvious i.e in plain sight that to grow you need to be distinctive? It’s a good example of why we need insights that are “out of sight”.

The fact that disruptive brands are different is an “insight” and not an “out of sight” but the real question is how to achieve differentiation (more later).

2. Market orientation – The whole aspect of “trusting the gut” is based on the gut being in tune with consumers and the market. However, being in tune with consumers cant be basis purely reading research reports, which are a reflection of the questions that have been asked. See the tweet below as an example of how the questions asked can reveal a lack of “empathy” with consumers.

We very often fall for the trap of projecting our lives onto consumers without realizing that we are not representative of the population.

Lets look at the UK ad industry whose disassociation from the “general UK public” is well documented. See the following statistics on lack of representativeness of the advertising folk

•  80% of people in the advertising industry are based in London. 87% of UK population does not live in London
• Over 50s account for 47.6% of UK household expenditure. 5.6% of the ad industry in UK is aged 50+
• 51% of UK population are women. Only 12% of creative directors are female.

This results in a completely warped perception. See a few examples on media consumption

	Ad people estimate	Actual data
% of people TV viewing is live	49%	87%
Time spent watching TV per day	161 minutes	215 minutes
% TV watched on other devices	37%	2%

As a market researcher I fully understand that it is normal for people to not remember exact amount of time spent on various activities like watching TV or watching VOD. Hence, consumers overestimate the time they spend watching VOD (for instance). What is troubling is to see the extent to which ad people overestimate the time being spent by ordinary consumers on VOD.

If you have any doubt about the fact that this just a case of self projection look at the chart below. Clearly advertisers and agencies spend a lot of time listening to spotify compared to radio and so project that onto the population, incorrectly of course.

It is critical that the market research industry develops a closeness to consumers and the market – for itself and its marketing clients. This is something that should be cultivated. Go out. Experience the products.

We need to experience things out of sight of our office. Hence, “outsight”.

3. Insights to ideas – The research industry has evolved from data to information to insights. The next stage of evolution is moving towards ideas. I referenced earlier how the need for the research industry is not just to provide insights but ideas on how those insights could be acted upon.

As a student of economics, I am firm believer in Karl Marx’s theory of dialectic materialism. Very simplistically put (I am oversimplifying it for the purposes of this blog) “every thesis will be met with an anti thesis and the collision of the two will result in a new synthesis”. 

What it means for market research is that we need a multitude of perspectives to collide to get to ideas to move from insight to action. The different perspectives can only come from different experiences. 

It needs us to go “out of the office”. Even in a country like Singapore I like to go out to the HDB food courts of Toa Payoh and Hougang and smell the coffee, listen to the conversations, look at the menu. Personally, I find it gives me a very different perspective. It’s a perspective that needs you to get outside and see things “out of sight”.

RIP Insights. Its time for Outsights.