Men are more violent than women. Women are more compassionate than men. Black men have the biggest penises. Eastern Europeans are the dumbest people in Europe. All these statements are true as far as a quick Google search can confirm, and I hope that by the end of this text I will correctly explain why they are completely irrelevant in their existing form. Irrelevance doesn’t really imply the need for further inspection, but in some cases the attention and credence awarded to these statements by people in very powerful positions can potentially affect the everyday lives of billions of people, most often in a negative way.
To start a conversation involving the necessity for statistical thinking at any point this day and age, you are opening yourself to a very figurative public execution. So I plan to start a conversation about statistical thinking. Since not many people know me, the potential execution will be a largely private affair, which means unfortunately that I won’t be able to sell any advertising space…
Let’s begin with some credentials. Don’t worry, this will be a short paragraph. While I’m not a statistician, I’ve been working with numbers all my adult life and I’ve incorporated statistics in my working career at an early age. Statistics also have a fundamental role in the science behind the books I’m currently reading, mainly about psychology and behavioural economics. I’m not saying that, because I’m reading these books, I have any sort of authority in statistics, psychology or behavioural economics, but I’ve been mentally pounded with the terms for long enough to think I can translate them into layman’s terms. Let’s be honest, the authors of those books are way too smart to be in touch with the layman at a level above basic everyday interactions. On the other hand, having once listened to a very short exchange between Stephen Dubner (of Freakonomics fame) and Daniel Kahneman (of Nobel prize for basically inventing the field of behavioural economics fame) and not having been able to keep up, even with the introductions, I think I am a true embodiment of the layman.
Humans are not equipped to think statistically, especially when they need to make quick decisions. While helpful in our cave-inhabiting years, when the difference between a quick decision and a considered one meant the difference between life and certain death, the quick thinking mechanism can be a factor in increasing hostility within the densely populated and civilised world we inhabit. I will exemplify two of the ways in which we make statistical thinking mistakes: by ignoring the distribution of values and by ignoring the population size. Both can have serious implications in the way the world is run, so I hope to provide some clarity for the future world leaders reading this text.
IGNORING THE DISTRIBUTION OF VALUES
I’ll start this part with a few words about normal distribution and standard deviation. Not to worry, I won’t go into too many technical terms, mostly because I’m too lazy to do the actual research and I have to rely on what’s in my head. You also don’t need to remember this in order to think statistically, but it’s worth bringing it up as it supports my argument. Normal distribution is a representation of where the values in a studied group sit, in relation to the mean value of the group. The unit of measure for the relationship to the mean is called a standard deviation, and it’s a measure calculated by statisticians which shows how spread out are the values within the group. So let’s say we’re looking at the different heights within a group of 100 men, with the mean height of 178 cm and a standard deviation of 4cm. What the normal distribution will show (by definition, so we’ll just have to trust those who came up with the science) is that just over two thirds (68) of them will sit within 1 standard deviation of the mean (between 174 and 182 cm), about 95 within 2 standard deviations of the mean (between 170 and 186 cm), roughly 99 within 3 standard deviations (between 166 and 190 cm) and so on. The beauty of normal distribution is that it’s symmetrical around the mean, so it looks the same in both directions, above and below the central point. The visual representation of the normal distribution of values is a curve shaped like a bell sat on the wide side. This curve is named… wait for it… a bell curve! Or Gaussian curve if you’re being pedantic, but I’m sure Mr. Gauss did not have a shape nearly as interesting or relevant.
To further conclude, the more exceptional a value in reference to the mean, the fewer the people that would exhibit that value. I hope that makes sense, because that’s all I’ve got as far as explanations go.
So how is this relevant in everyday life? Well, let’s take this statement (which is completely true, thoroughly researched and definitely not made up): Blorks are fatter than Grispins. Want more detail? Blorks have a mean BMI of 29, while Grispins have a mean BMI of 27. Now, let’s assume that the standard deviation for both Blorks is 2 and for Grispins is 3 (this basically says that the Grispin population has slightly more spread-out values). So in this case, 68% of Blorks will have a BMI between 27-31, while 68% of Grispins will have a BMI between 24 and 30. Let’s go one further, and say 95% of Blorks have a BMI between 25 and 33, and 95% of Grispins have a BMI between 21 and 33. Now, imagine you have a scheduled meeting with a Blork and a Grispin who you haven’t met before. The Blork is selling glass fish, obviously, and the Grispin is the Blork’s lawyer. The two slither in, and one is visibly heavier than the other. How do you know to address the Blork first, knowing full well that one should never speak directly to a Grispin or risk being blinded by a spray of acidic saliva? What most people would say here, without the information I presented above, is that they would choose the heavier one as the Blork. And that, my friends, is specifically what a failure in statistical thinking is in real life. As you can see from my flawless numerical example, when looking at the populations of Blorks and Grispins, 95% of them will be within a very similar range of values. Assume now that the population of Blorks is equal to the population of Grispins, and sits around 10 million. There are 19 million Grispins and Blorks whose BMI is roughly within the same parameters (granted, with more skinny Grispins). Knowing this, are you confident in saying that the fatter one is a Blork? Hopefully your answer now is “no, I should not assume that the statistically true and significant group-level statement of Blorks are fatter than Grispins applies to any two random individuals”. The very important distinction is that this is an isolated case, where information about a group was used to stereotype an individual. If you did this enough times, you will end up making the right choice slightly more often, but the result of one particular case cannot be predicted by the group averages. And the answer to my initial trick question of how you know to address the Blork is, as anyone knows, that you should shriek loudly and clap your hands 3 times while doing a twirl, and the Blork will do the same as it’s their official greeting.
Leaving this example aside, now imagine that you are the hiring manager at a top tier company in your country. You are hiring for a position which needs to implement a complete restructure in your company, and this involves firing 50% of the current staff. You are looking at two candidates. Both equally qualified, both equally well recommended, one is a man and one is a woman. Who will you choose? Well, unless you answered that you’ll devise some sort of Solomonesque trial to separate the two, your subconscious biases will indicate that you would lean towards hiring the man: women are more compassionate, men are more violent. A woman might be fazed by having to ruin so many people’s livelihoods, while a man may be better suited to do so. Most times, this sort of choice can be officially motivated; when we talk about hiring people in today’s corporate world, buzzwords like “cultural fit” and “proactive attitude” are get-out-of-jail-free cards for discrimination. What this is, in fact, is a failure of thinking statistically, where potentially the better candidate for the job might miss out on the opportunity for no valid reason.
IGNORING RELATIVE POPULATION SIZE
I’ve read about this example in Kahneman’s masterpiece, Thinking Fast and Slow, so I can’t take credit for it. Also, Mr. Kahneman, when you do stumble upon this text, I must apologise to you for the butchered quoting that’s about to take place, I’m not sure where I placed your book so I’m working based on memory alone…
Think about the following person: a man, in his 30s, clean cut, very tidy. He wears glasses and loves to read. What job do you think is more likely that this man has? Is he more likely a librarian or a professional driver?
If you answered “librarian”, then you haven’t read the title of this section correctly. Let’s assume that all librarian men fit this description, and only 0.1% of drivers do. It is still a lot more likely that this man is a driver, and that’s simply because the population of drivers is so much larger than the population of male librarians that the numbers stack up in the driver’s favour. I’ll use completely made up numbers, unlike the previous examples, but hopefully they can prove my point. Let’s assume that in the Australian state of Victoria, there are 100 librarians, of which 50% are male. This makes the population of male librarians, you guessed it, 50. About 5% of the working population are drivers (this should be at least in the right ballpark). That’s about 150,000 people for Victoria. 0.1% of that number is 150. That’s 3 times more than the librarians! And in this example I assumed all male librarians fit that description, which is definitely not the case.
There are many other ways in which we fail to apply statistical thinking, some with more impact than others, but I’ll stop here hoping that this is enough to exemplify what I meant at the start of the text.
Failure to apply statistical thinking can lead us to the wrong solutions for some of society’s biggest problems. Take the wage gap between men and women, currently sitting at around 75 cents to the dollar in favour of men. It is a true fact that women are making that much less money than men, this is incontestable. The problem comes when people try to solve this without applying statistical thinking.
I know what I’m risking by unpacking the following example, so please bear with me to the end. It’s been calculated, by people smarter than me, that the pay difference between women and men doing the exact same job, with the exact same experience is actually somewhere around 95-97 cents to the dollar. So only a small portion of the earnings gap is due to outright discrimination. Unfortunately, this is the portion favoured by the media, social or otherwise, to be flashed out and talked about. The rest of the difference in earnings comes from a few other factors:
- The fact that women are under-represented in the highest paying jobs, like finance, engineering, IT etc., and over-represented in the lowest paying jobs, like nursing, community services, education etc.
- The fact that women need to take time off for maternity leave, this having the effect of reducing their overall experience as their time in the workforce is reduced by whatever length of time they took off
- The fact that most managerial jobs in the corporate world (which are some of the highest paying) require a level of disconnect from employees. Despite what management books tell you, successful managers at the top are ruthless individuals in their working lives, and they actually need to be in order to best serve the needs of the board. Women are simply not as good at being ruthless as men are, as an evolutionary trait. I’m no evolutionary biologist (the screams of “no shit, Sherlock” are really not necessary), but I think it stands to common sense to assume ruthless women would have had fewer offspring and less of a chance to pass on their genes, while ruthless men would have been favoured to procreate. We are not blank slates at birth, we have a significant genetic baggage we bring along, and traits which command a higher pay in today’s world are, as it happens, male traits.
- Other reasons I haven’t thought about
There are currently, as far as I know, two active solutions for reducing the earnings gap. One is to make it illegal for women to be paid less than men for the same job; I think this is a brilliant solution which was implemented as government policy in Iceland recently, but it only addresses a small portion of the gap. The second solution is to have hiring quotas for the higher paying jobs, and this is the one I believe is misguided. Imposing a quota on, let’s say, the proportion of software programmers which need to be women means that, presumably, half of the available positions are given to a male pool of applicants, and half to a female pool of applicants. IT programming is a field grossly favoured by men, which means that the relative size of the two pools is not equal, with a lot more men than women applying for those jobs. Assuming a similar distribution of applicant quality within the two groups, this means that the quota for female positions will have to be filled by less qualified applicants than the quota for the male positions (we’re working from the top down the bell curve). And this is just not good for business or progress, and it can ultimately be detrimental to the perception of equal abilities between men and women. If you have better male candidates filling the positions, the overall perception would be that men are better than women at software programming, and this is just not true.
This example works for any field, in any discipline, for quotas other than gender based. Fortunately, the solution is obvious, but for some reason not considered simple enough by those in power. Rather than force quotas, how about we level the playing field in a different way? How about we make the wages in jobs naturally favoured by women equal to those in jobs naturally favoured by men? How about we pay a nurse as much as we pay an engineer? How about we pay educators as much as we pay managers or the military? How about we stop rolling our eyes and say this isn’t commercially viable? I’m not an economist or politician, it’s not my job to find the solutions, but we have today more brain power than we ever had in history, how about we use it to come up with practical solutions?
I know I’ve only begun to scratch the surface of the issues with statistical thinking, and there is a lot more that was and will be said on the topic, but I want to at least draw attention to how damaging ignorance in this regard can be to society. Politicians rush to address whatever the social echo chambers are screaming about, and for good reason. The politician who tries to explain a decision through statistical thinking risks a sea of blank stares at best, and a hurricane of invective language at worst. The sad reality is that sensible choices and directions, with long term effects but without short term gratification have about as much chance to get a politician re-elected as releasing a sex tape has in reviving some star’s career. Moves are being made to get this sort of work done outside the public eye by creating so called “nudge departments” within governmental mechanisms, brought into existence through the work of brilliant behavioural economists like Richard Thaler. These nudge departments work by recommending policy which corrects some human biases, without restricting choice in any manner. Think of what difference having an opt-out choice when deciding whether to be an organ donor, versus an opt-in choice, has on overall organ donor numbers (if I remember correctly something like 80%) and you can get a rough idea about the level at which these departments work. I don’t know how a nudge department could work with the specific examples I made, but maybe that’s why I haven’t been offered a job in one already.
Whether is by becoming more aware of our errors, being shoved in the right direction by government regulations or nudged through subtle changes in policies affecting everyday life, there is hope that, in time, cognitive errors of the kind we are making now everyday will become the stuff of history books. Until then, let’s just at least try and challenge our thinking as much as we can.