Ten ways to optimize evidence-based policy

Working across sectors requires shared language. The need for academics to provide ‘plain language’ research reports has been well described [4]. However, developing a shared language should start well before the final report. For example, the seemingly simple question, ‘What do you define as research?’, can unpack a range of hidden assumptions. Is ‘data insights’ within a government organization ‘research’? What about focus groups with no ethics approval? What is the relative value of ‘gray’ and peer-reviewed literature? Asking and addressing ‘the stupid questions’ – about abbreviations, definitions and terminology for example – optimizes the value of collaborative engagement by creating a shared conceptual starting point.

The line from research to practice is anything but linear, and research evidence is one of the many inputs into policy. Building an understanding of the policy-making process highlights the context in which research-based information is being given [5]. Key political considerations and debates may be well known; however, more subtle nuances and insights that arise through academic–policy collaboration are critical to fully understanding issues such as local implementation challenges. Equally, it is important to manage expectations about what research can and cannot deliver in given timeframes and budgets so that end products of research engagement are also considered within their academic context.

Albert Einstein once famously said: “If I had an hour to solve a problem, I’d spend 55 minutes thinking about the problem and 5 minutes thinking about solutions.” In a world that demands instant solutions, there is often little time or motivation to question the nature of the problems we are asked to solve. However, taking problems ‘as read’ carries risk of wasted investment. A frequent example is investing in an education campaign without establishing if there is a knowledge need, or other behavioral drivers such as attitudes and beliefs. Exploration of the problem and/or the barriers that need to be overcome to effect change is best done through qualitative methods such as in-depth one-on-one or small group interviews [6]. Despite the obvious value of this to everything that follows, this sort of exploration is frequently not undertaken, even when very large investments in change strategies are being made.

Review-level evidence is the preferred unit of knowledge to inform policy and practice as it brings together knowledge from multiple individual studies, reducing bias [6]. Systematic reviews of primary studies are the definitive review type, and are freely available in centralized platforms of systematic reviews such as The Cochrane Library (https://www.cochranelibrary.com/), Health Systems Evidence (https://www.healthsystemsevidence.org/), The Campbell Collaboration (https://campbellcollaboration.org/) and Social Systems Evidence (https://www.socialsystemsevidence.org/). These platforms carry significant advantages over native internet searching by assembling much smaller volumes of more credible and easily navigable evidence. It is recognised that governments are often unable to commission a systematic review due to time constraints. Rapid reviews, which synthesise review-level evidence (i.e., a ‘review of reviews’) enable government and other organisations to gain insights into the state of knowledge in weeks, rather than months or longer [7]. However, their limitations should always be made explicit; rapid reviews cannot cover the same depth of information as a more detailed systematic review of primary studies. Complex issues faced by government (e.g., preventing and managing obesity, optimising transport services or reducing welfare dependency) raise challenges repeatedly over long time periods. In these situations, commissioning a systematic review and keeping this up to date over time may represent a better long-term investment than a one-off rapid review.

Research publications and reports take time to publish and are usually not specific to the end-user context. Both of these disadvantages can be mitigated by reviewing practice in addition to evidence. This enables information from research to be triangulated against on-the-ground realities, constraints and enablers from the perspective of groups vested in the issue. These groups may include (e.g., in health) consumers representing particular healthcare conditions, service populations or geographical regions; healthcare professionals; healthcare delivery organisations and funders. There are a broad range of methods for practice review including one-on-one and focus group interviews. Structured stakeholder dialogues convene representative groups of 18–22 people in a day-long facilitated discussion of possible actions to address a complex challenge. This is informed by a pre-circulated evidence brief synthesising best-available knowledge on the issue. Such dialogues have been shown to have high participant satisfaction and crucially, high intention to act on the basis of relevant evidence [8]. Stand-alone citizen panels and other consumer-specific engagements can empower consumers to contribute perspectives without the potentially inhibiting influence of multi-stakeholder forums. In addition to ensuring that policy initiatives are consistent with underlying values of service users, these have extremely high participant buy-in and satisfaction [9,10].

A common misconception about evidence-based practice is that practitioners take no action unless they can draw upon definitive evidence (for an amusing take on this, refer to Smith and Pell's systematic review of randomized controlled trials for parachute use [11]). However, in most areas of healthcare there is no incontrovertible trial-level evidence about the effectiveness of interventions [12]. Knowledge of the best available evidence, in conjunction with clinical experience and patient values, is therefore used to guide practice and decision-making. Similarly, knowledge of the state of the evidence can usefully inform policy development and delivery. For example, the extent of monitoring and evaluation required is broadly inversely proportional to the strength of the evidence base.

The rise of the ‘nudge’ movement has created widespread interest from governments in simple, relatively cheap behavior change strategies, for example, a minor change in taxation letters to increase tax revenue [13] and use of similar communication techniques to boost vaccination rates [14–16]. However, nudge strategies generally have modest effect sizes at a population level, and not everything can be ‘nudged’. When a behavior or the system within which it occurs is complex, long-term commitment is necessary. One example is a recent project undertaken by BehaviourWorks Australia in partnership with the Victorian Government's Department of Health and Human Services to reduce the volume of non-emergency calls to Victoria's emergency ambulance service. This project required formative review and consultation work to establish appropriate behavioral messaging; upfront investment in shifting attitudes prior to focusing on the desired behavior and extensive monitoring and evaluation [17]. Importantly, the formative review demonstrated that ‘quick fix’ campaigns in this space did not demonstrate effectiveness. The project ran over a number of years, incorporating several public campaign phases. The monitoring and evaluation was critical in demonstrating impacts over this time period [17].

Another misconception regarding the research policy nexus is that it is limited to ‘what works.’ Of course, governments need information about the effectiveness of strategies to address pressing public policy issues. But they also need to know about the nature and impact of the problem, the relative cost–effectiveness of different policy approaches, the acceptability and feasibility of proposed interventions, barriers to implementation and how solutions will be organised, governed and funded [18]. Fortunately, review-level evidence addressing all these types of questions is readily available through centralized databases of synthesized evidence described earlier.

In addition to gathering information on current practice through consultation interviews as described above, it is important to understand and respond to emerging trends. The information age has created a deluge of globally visible conversations and a range of technologies that can track these. In academia, citations – the standard metric of research impact – are beginning to be overtaken by real-time statistics such as article accesses, downloads and other composite metrics. Similarly, governments are able to monitor social media trends, online commentary and directly-sourced online feedback to rapidly evaluate how policy ideas and interventions are being received. The price for such immediacy is decreased confidence in the rigor of information, much of which bypasses traditional peer-review and other checks and balances. In the coming years, new approaches will need to capture and appropriately weight academic, government, traditional, social media and other information streams. The ‘R-Search’ approach of the US-based GovLab, which brings together information from peer-reviewed literature, news, social media and crowdsourcing, is an early example [19].

Close collaboration between the academic and policy worlds underpins most of this ‘top ten’ list. The logical extension to this is co-production of research, rather than packaging and provision of knowledge by academics to policymakers. Co-production of research is a classic case of the whole being greater than the sum of its parts. Governments possess in-depth knowledge of the big systems that are designed to make life easier for citizens, challenges faced in delivering services through these systems and system performance data. Researchers are expert problem solvers who bring research knowledge and skills to systems, challenges and data. Given the substantial challenges faced around the globe today, it is time to move away from the ‘government are from Venus, researchers are from Mars’ narrative and instead focus together on the challenges we are both looking at from different, often complementary, angles.

The author's research institution, Monash University, receives research funding from a range of government entities including the Victorian Department of Health Services and The Victorian Managed Insurance Authority. None of this funding goes directly to P Bragge. In his role at Monash University, P Bragge is involved in the Social Systems Evidence review platform and a number of health-focused behavior change trials that are referenced in this article. The author has no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

No writing assistance was utilised in the production of this manuscript.