blog




  • Essay / Critical assessment of evidence-based policy theory and practice

    It is often noted that there is something inherently attractive about the concept of evidence-based policy (EBP) (Shillabeer et al., 2015). In fact, the antithesis of opinion-based politics is often considered objectionable (Davies, 2004). However, inherent to these sensitivities is a preconceived definition of “evidence.” For example, common conceptions of “evidence” include positivist and empirical modes of input-output observation (Green, 2000). In contrast, normative and theoretical reasoning is often overlooked as an appropriate factual basis (Sarat and Silbey, 1988: 107), although it remains an essential and ubiquitous source of information for policy making (Zane and Welsh , 2017). It seems that the term “evidence” is a very ambiguous and broad term. It will be suggested that these ambiguities are at the heart of the disparity between EBP theory and its implementation and practice. Furthermore, implicit in this disparity is the question of whether EBP is first possible and second desirable. To begin, a review of definitions and understandings of the evidence bases, as well as their conflicts, will be provided. Implications surrounding current conceptions of the stage model of the policy process will follow. Third, the practice of EBP will be discussed within the framework of a sectoral comparison between medical and penal policy. The importance of networks and “guidelines” for appropriate selection of evidence will be emphasized throughout the conference. Say no to plagiarism. Get a tailor-made essay on “Why Violent Video Games Should Not Be Banned”? Get an original essay Conflicts between theory and practice are not unique to political studies. Indeed, such tension is also found in the political (Ozcelik, 2006), mathematical (Malara and Zan, 2003) and aesthetic (Borgdorff, 2012) domains. However, conflict resolution techniques in these areas remain underdeveloped (Ozcelik, 2006; Wells, 2004). For example, in the policy context, Green (2000) and McLeroy et al. (1993) suggest that multiple theories may be needed in tandem to successfully develop programs and policies. However, the absence of 'guidelines' or 'justifications' for selection can significantly hamper the success of a policy (Green, 2000: 126; Kriesberg, 1995: 170; Buchanan, 1994). In this way, the theoretical foundation of EBP can be seen as incomplete: while there is support for integrating evidence into policy, there is little guidance on how evidence the most effective should be selected and assimilated. Furthermore, Davies (2004) notes that, on the one hand, knowledge uncertainty and, on the other hand, the different status of knowledge domains present significant challenges for EBP. In other words, the status conflict between positive and normative knowledge, and the lack of a coherent strategy for resolving conflicts between them, contributes to an unstable theoretical foundation of EBP (Zane and Welsh, 2017 ). In other words, the value placed on empirical research often overlooks the essential function of normative research (Zane and Welsh, 2017). One result is a “knowledge vacuum” in which the surplus and preference for positivist evidence fails to advance policy while alternative forms are neglected (Davies, 2004: 4). Indeed, the report by Bullock et al. (2001) concluded that government departments use a limited range of evidence from a restricted set ofresources. Furthermore, “knowledge uncertainty” may arise from ambiguity surrounding the definition of evidence and, as suggested above, what evidence is. form it should take (Nutley et al., 2002). For example, the Oxford English Dictionary (OED) defines evidence as a “body of facts or information indicating whether a belief or proposition is true or valid” (OED, 2018). However, this definition is limited in two ways. First, there is ambiguity about the existence of an objective “truth” free from social and cultural ideological biases (Nath, 2014). Evidence must therefore be seen as “contingent and contextual rather than universal, determinative and invariable” (Buchanan, 1994: 274). This calls into question the impartiality associated with positivist evidence and the natural sciences (Buchanan, 1994; Green, 2000). Parkhurst (2017: 7), for example, notes that technical biases can result from neglecting rigorous scientific practice in order to provide evidence in support of certain political interests. Normative and positivist evidence can therefore produce equally biased results, calling into question the ability of evidence to provide some "validity." This is closely related to the second limitation: the OED definition is implicit in the separation of “facts” and “information.” . As Zane and Welsh (2017) suggest, positivist understandings (“facts”) remain largely descriptive while normative evidence (“information”) tends toward prescriptive claims – although they differ in intent and methodologies, it It is imperative that the two are recognized as inextricably linked. . Otherwise, the potential of EBP may be limited to “the way the world is” rather than “the way the world should be” (Monahan and Walker, 1988: 467). The use of normative data, for example, incorporates unobservable facts, such as equality and fairness, into policies (Faigman, 1999). Additionally, the theory can be used in policy evaluation to address a variety of inappropriate variables (Green, 2000: 126). However, the exclusion of positivist data from the political process in exchange for normative data can result in the maintenance of power and social structures detrimental to social equity (Sarat and Silbey, 1988: 107). Moreover, as we will see later, it is essential to policy making in areas such as medicine (Chalmers, 2003). It appears that the two are interdependent, although this remains to be established in official policy logic (Nutley et al., 2002: 2). Again, the lack of a “guideline” to inform the appropriate selection of evidence may exacerbate this status conflict. Similarly, there is a conflict between propositional evidence (i.e., formal, academic knowledge) and non-propositional evidence (i.e., informal and obtained through experience) ( Malone et al ., 2004: 83). On the one hand, the latter emphasizes the conditionality of evidence collection: normative evidence based on non-propositional knowledge may be limited by the number of individuals and their combined life experiences (Malone et al., 2004: 83). Therefore, networks combining skills and experience are integral to expanding the evidence base (Nutley et al., 2002; Bowen and Zwi, 2005). This can help bridge the gap between theory and practice, as researchers and stakeholders can use non-propositional knowledge to create alternative pathways (or “guidelines”) for navigating the appropriate use of evidence (Bowen and Zwi, 2005). Furthermore, thepossibility of normative, non-propositional “information” being converted into propositional, positivist “facts” (i.e., theory generation) highlights both the importance of partnerships and the interdependence of concepts (Titchen and Ersser, 2001). However, this reasoning is somewhat idealistic, as evaluation practices differ in terms of funding, capacity, and rigor, thereby limiting the potential for in-depth assimilation of evidence (Head, 2016). A broader interpretation of the term “evidence” should therefore be developed (Green, 2000); an approach that integrates the full range of evidence and recognizes the improbability of finding an objective “truth” (Higgs and Titchen, 1995). If an expanded definition of evidence were to be adopted, it could be argued, the foundations of EBP would stabilize, thereby facilitating its translation into practice. This conditionality is summarized in Nutley et al's (2002:2) four requirements for improving EBP: Agreement on what counts as evidence and in what circumstances A strategic approach to the creation and accumulation of robust evidence Dissemination efficiency of evidence and wider access to knowledge Initiatives to ensure integration of evidence into practice Similarly, Weiss (1998) suggests that for EBP to be implemented successfully, the methodology and results must be unquestionable and support existing political ideologies, that policies have strong “champions” and that consequences must be reversible and robust (Nutley, Davies and Walter, 2002; Weiss , 1998). If these are the optimal conditions for EBP, it follows that policies limited by a narrow understanding of evidence do not provide a sufficiently broad theoretical basis for the practice of EBP (Parkhurst, 2017). Indeed, “Modern Policy Making” (NAO, 2001) concludes that using a variety of knowledge sources and analytical skills is essential to risk management and EBP success. Furthermore, other theoretical inconsistencies may result from inadequate conceptualization of the political process. In other words, a simplistic conception of the policy process obscures when evidence informs policy, thereby complicating the translation of EBP into practice. For example, proponents of the linear model of the policy process suggest that policymaking begins with problem identification (Araral, 2012). Here, it is assumed that evidence would, and for some should (Andrews, 2017), inform policy-making from its infancy. Furthermore, feedback loops and the “policy evaluation” stage imply that the process is continually informed by evidence (SOAS, 2018; Sutton, 1999), implicitly suggesting that evidence provides the impetus for formulation policies (SOAS, 2018). Indeed, the usefulness and ubiquity of the model are often attributed to its heuristic function and simplistic idealism (Roe, 1991), thereby mitigating the need for precise and in-depth description (de Leon, 1999). However, the evidence is not used as comprehensively (Richards, 2017) and the simplicity of the model has caused the development of more complex theories. For example, recognizing the possibility of externalities affecting the implementation or revision of a policy (i.e. a “policy window”), Kingdon (1999) proposes a more complex model. Building on Cohen et al’s (1972) “trash can,” Kingdon recognizes that organizations, problems, solutions, and processes are nonlinear and anarchic (Cairney, 2015). As a result, when problems arise, an organization can rely on a mix of solutions from the "waste",that is, a selection of resources that have not yet been assimilated into a solution that could be either ineffective or preferable (ibid.). The point at which the solutions contained in the “trash can” can be used, however, remains undetermined and therefore reflects the reality of evidence used in the political process: information and facts can be introduced at any time and for an uncertain purpose (ibid.). . ). Furthermore, Kingdon's (2003) description of a “primitive policy soup” recognizes that policies continually evolve due to factors other than evidence, for example the strength of networks (Nutley et al., 2002; Bowen and Zwi, 2005). , although Kingdon offers significant advances in the stages model, conceptions of the political process continue to hinder the theoretical foundations of EBP in two ways. First, simplistic models of policy formulation continue to prevail and thus contribute to an inaccurate representation of the use of evidence, in which it appears to dominate the policy process (SOAS, 2018; Sutton, 1999). Second, the models fail to provide a causal explanation of how evidence enters the policy process (Howlett et al, 2014: 10) and thus hinder the formulation of 'guidelines' for the use of evidence . However, the development of more complex theories undermines the assumption that evidence is an integral part of the policy process. This has implications for the use of the term “evidence-based policy.” Indeed, given the often subordinate use of evidence in policy, there is a preference for the term evidence-based policy – ​​despite the changing agenda (Head, 2016; Nutley, Davies, Walter, 2002). The concept of EBP gained traction in the late 1980s and 1990s with the rise of belief politics (Nutley, Davies, & Walter, 2002). Specifically, Labour's modernization program (1997) centralized EBP with the aim of moving from 'what works' to 'why it works' (Wells, 2004: 6). However, some equate this rise to technocratic policymaking and a managerial agenda (Sanderson, 2002). Furthermore, policies generated by this movement may appear to overlook additional elements that inform policy formulation (Parsons, 2002). For example, the economic climate and political context can determine access to resources affecting policy implementation (Head, 2016). These factors affecting policy are conceptualized by Kingdon (1999) as three streams (policy, problem and policy) which interact and, in the appropriate 'window of opportunity', converge to produce policy. For example, it has been suggested that EBP has developed in tandem with a political climate of accountability in which taxpayers demand efficient use of tax resources (Myers and Spraitz, 2011; Weiss et al, 2005: 28). To set an example of maintaining order; between 2010 and 2014, the “problematic stream” of taxpayer demand and limited budgetary resources converged with the “political stream” of austerity, resulting in a 25% reduction in funding (NAO, 2015: 4) . However, evidence suggests that these cuts could undermine the statutory basis of the networks and increase demands on police forces, thereby having negative effects on the fight against crime (Karn, 2013: 21-31). The political and socio-economic context therefore receives equal, if not more, attention than evidence in policy formulation.As Davies points out in the Labor Party programme: policy is not just a question of 'what works', but also 'at what cost' (2004: 5). However, like the normative versus positive debate above, this example should not be indicative of the polarization between ideology and evidence. For example, Chang and Wang's (2016) study of the relationship between policy discourse and empirical evidence suggests that they are inextricably linked: the use of empirical research in policy is designed to make causal inferences and is therefore essential to the normative reasoning of political rhetoric. Furthermore, it appears that the interaction between the two is an integral part of the functioning of open democratic societies (Davies, 2004: 5; Dillow, 2014). This begs the question; To what extent should evidence inform policy if other factors are equally important? There is certainly no correct 'ratio' for policy formulation and equal use of resources and types of evidence is not necessarily preferable across all public sectors (Nutley, Davies and Walter). , 2002; etc.**). Medicine, for example, has established a “hierarchy of evidence” that prioritizes systematic reviews and randomized experiments on the one hand, and devalues ​​political ideology, observational studies, and professional consensus on the other (Hadorn et al, 1996; Nutley, Davies and Walter, 2002: 3). Two reasons can be given for this. First, there is consensus about the desired outcome of health policy (reduced mortality and better health), thus facilitating methodological selection (Nutley et al, 2002: 3). Secondly, the consequences of a less rigorous methodology are significant: reduced effectiveness of the treatment and therefore harmful effects on health (Moher et al, 1998; Nutley et al, 2002). Indeed, a greater status accorded to positivist evidence in medical policy has led to theoretical coherence. across health care and the establishment of beneficial practices (such as integrating patient values ​​into physician behavior) and institutions (such as the Cochrane Collaboration) (Sackett, 1996). Therefore, EBP minimized the effects of bias and chance (Chalmers, 2003). Additionally, while this does not necessarily prevent the dissemination of methodologically weak evidence, the systematic approach to evidence selection and standardized practices for critical appraisal (e.g., by the Cochrane and Campbell Collaboration) offer a prominent example of optimal EBP practice (Davies, 2004). ): 'what matters' as the evidence is easily understood and guidelines are offered to ensure theoretical consistency enabling successful translation into practice. However, its practice is not easily transferable to other sectors. For example, the introduction of the What Works Center for Crime Reduction in 2013 aimed to “improve the way government and other organizations create, share and use high-quality evidence for decision-making” (Hunter et al., 2016: iv). ). However, seven boundaries have been identified, three of which will be expanded. First, it has been noted that there is “no well-developed roadmap” offering guidelines on the evidence used in practice (Hunter et al., 2016: xi). Significantly, this has been attributed to the limited development of a theoretical foundation (Hunter et al., 2016: xi). Second, the report identified that limited resources prevent the optimal use of “evidence champions” and the creation.