Policymakers seeking empirical evidence on social policy interventions often find themselves confronted with a mountain of academic studies that are potentially relevant to the question. Without some systematic way to sort through the evidence, there is a risk that analysts will become mired in the research, or simply cherry-pick those studies that support their prior beliefs. An alternative approach is to test each study against a hierarchy of research methods.
This article from Economic Roundup Issue 1, 2009 discusses two hierarchies — one used by US medical researchers, and another used by UK social policymakers — and suggests one possible hierarchy for Australia. Naturally, such a hierarchy should not be the only tool used to assess research, and should be used in conjunction with other factors, such as the ranking of the journal in which a study is published. But used carefully, a hierarchy can help policymakers sort through a daunting body of research, and may also inform governments’ decisions on how to evaluate social policy interventions.