Governments have “big data strategies” to address ever growing expectations about how data could help answer some of the toughest questions in research and in policy. The public is promised benefits from fairer credit assessments to better predictions of drug side effects. Data science is clearly a powerful tool. If used properly, people will be able to make better decisions far more efficiently.
But what if our journalists, politicians and other decision makers don’t know how to ask the right questions to check the quality of these claims? What if they’re left wondering what terms like “big data” or “AI” even mean? Or they’re passing on flawed information or making bad decisions, because they don’t know how to scrutinize data science analysis? Equally, we can’t always rely on researchers or data scientists to know exactly how their work actually applies to the real world.
The public’s ability to question the quality of evidence can make a huge difference. We know this from debates about patient information, climate science, misleading product claims and military interventions. Society would reject a public health program rolling out on the basis of a study with a 10-person sample size. So we must not let data science become a “black box”, where these questions about evidence quality drop away.
A compelling reason to act now is the increasing weight being placed on the results of data science, such as making custodial decisions about offenders and allocating health and social resources. The quality of this evidence really matters.
This guide isn’t about becoming a data science expert. It explains the language to help talk about it and highlights the key questions to ask those people using data science as evidence in decision making. If we ask more of these questions and turn the discussion to the quality of the data analysis, we can reject hype, recognize what’s useful and ensure data-led decisions are transparent and accountable.