Advances In Evidence-Based Policing (Crime Science Series)
Editors: Johannes Knutsson and Lisa Tompson
Publisher: London: Routledge, 2018. 248p.
Reviewer: Jack R. Greene | May 2018
It is perhaps odd to start a book review by beginning at its conclusion. Johannes Knutsson and Lisa Tompson, editors of Advances in Evidence-Based Policing, conclude that “a central message of this book is that at certain times and places RCTs [randomized control trials] can be very useful, but those times and places are rather limited”. It’s a refreshing conclusion that perhaps might have been the opening of the volume.
This is the case as the EBP [evidenced-based policing] community, broadly speaking, has largely shunned the idea that anything short of RCT or quasi-experimental study is irrelevant to understanding the police and their impacts. As this line of inquiry has begun to mature, a moment of thoughtful reflection, represented in part by this volume, suggests that EBP will be enhanced to the extent that its frame of reference is expanded and its integration with police practice to shape research design and interpretation is accommodated. In short, policing is too important to be left to academics alone.
The evolution of innovation and change and its adoption is ongoing, whether that be in policing or something like medical practice, something EBP says it emulates. Yet in medicine, for example, evidence has evolved from bloodletting and the use of leaches, to surgeries where doctors rooted around in their medical bags, while patients died on the operating table, to the introduction of antibiotics to reduce very high post-surgery infections, and presently to robotic surgeries offering minimal invasion and quicker recovery times. What is useful in observing this trajectory in medical science is that in each era the medical profession was said to be using the best evidence to guide practice. Yet if they remained fixated on one of the earlier eras over others, medical surgery itself would have hardly advanced. One reason for this is that medical evidence advances on several levels, most of which are eschewed by EBP. It is important to acknowledge that the practice of evidence-based medicine rests on the interaction of the best external medical evidence (mostly scientifically derived), the clinical experience of medical practitioners, and the values and expectations of patients.
So it is with the evidence-based policing and crime prevention (EBP) movement; a time for introspection and critique of what EBP actually means and accomplishes. As has been noted by many, the EBP movement has caught the imagination of academics and policy makers, often for differing, yet overlapping reasons; the former being concerned with the validity and reliability of their findings, the latter with concerns with “value for money”. So it might be argued that scientific credulity as well as bureaucratic efficiency are deeply imbedded in the EBP movement at present. Reflecting on these interactions is at the core of this volume.
What is interesting is that the EBP movement is now entering its 20th year since Sherman put evidence in play. While there has been considerable critique of these issues over this period, Johannes Knutsson and Lisa Tompson, editors of Advances in Evidence-Based Policing (Routledge, 2018), together with a notable array of social and crime scientists, parse the achievements and blind-spots in the EBP movement. The editors suggest that the book is organized around four themes, 1) the underlying logic and philosophy of EBP, 2) generating and disseminating EBP, 3) the problematics of police using EBP, and 4) building stronger partnerships and linkages between the police and external others, especially academia. Taken together the book is a solid, and appropriately critical assessment of the state-of-the art of EBP and pathways for its improvement. It is also a critique of some of the shortcomings of EBP conceptually, methodologically and pragmatically.
Chapters 2-4 spotlight some important concerns with how evidence in policing is characterized. In each chapter there is the suggestion that the EBP movement is anchored quite narrowly from the perspective of what research undergirds this approach. Tilley and Laycock (Chapter 2) raise important questions about the what, when, why and how of EBP. In many ways they demonstrate the thin veneer of science that currently characterizes EBP. Collectively, they suggest that the EBP movement is supported by “relatively rare and sporadic” scientific inquiry; that good intentions guide much of EBP, rather than clear theory, leading on occasion to unintended consequences; that ideas associated with evidence are too restricted to random-controlled trials, which narrow what we know about evidence in its many manifestations; and that it is not demonstrated that evidence guides police resource allocation. They offer several ways to mediate these issues and to move EBP forward. In Chapter 3 Michael Scott offers a thoughtful assessment for the reconciliation of problem-oriented policing (POP) with EBP, noting the degree to which these approaches overlap, yet are independent. Much of this concerns differing sources of evidence or knowledge (narrowly defined versus professional) and the need to preserve each. Chapter 4 opens like a meeting of those formerly addicted to EBP. John Eck acknowledges that he has “marched in the evidence-based movement” much like a person entering AA or ANON starts by acknowledging their addiction. “Hi I’m John a formerly addicted EBP person” – “Hi John” the group replies. Eck’s trek through the logic (often logical fallacies) models that underlay EBP, questions of internal and external validity of these approaches, the need for theory not method driven approaches, and the need for better translation from research for academic audiences to research to inform practitioner decision making. These three chapters carve out an important assessment of how EBP has come to the forefront, and especially how its limitations will likely restrict its continued growth.
Chapters 5-8 shift the discussion about EBP from concepts to actual research practice and how information is shaped in the name of evidence. Mike Maxfield and colleagues (Chapter 5) take us on an important tour of practice-based evidence and the methodological tools for conducting research on these issues. To the credit of these authors, and rooted in the Evidence Generated Initiative at John Jay College, the authors take readers through a painfully clear discussion of problem definition, causal processes, measurement and generalization, concluding that “practice-based evidence and more flexible approaches to evidence generation are potential ways to expand the field of vision…”. Ken Pease and Jason Roach (Chapter 6) return us to a theme that is at times latent and at time manifest in this book, experience and expectations count, and are important lenses to assess what constitute evidence, especially usable evidence. Using Bayesian thinking –what stakes (judgements and beliefs) are involved, with what information and feedback, and with whose investment, Pease and Roach call for broadening the EBP movement into one that helps “morph experience into evidence”. In a chapter entitled “Reviewing evidence for evidence-based policing”, Bowers and her colleagues (Chapter 8) provide an insightful and thorough review of the ways in which evidence has been defined and operationalized in policing, assessing their strengths and weaknesses. Using the EMMIE framework for assessing policing research (effects, mechanisms, moderators, implementation and economics) they focused on linking outcomes to the processes presumed to produce such findings. Their assessment reveals that current EBP research efforts, which are focused on outcome, are often more concerned with measuring effect sizes than understanding how those effects are produced. Taken together these three chapters converge on the notion that EBP is too narrowly focused, often leaving practitioners out of the information translation loop.
Chapters 8 through 10 provide insights from several perspectives around the question of what to do with evidence once obtained. Mazerolle and her colleagues (Chapter 8) see evidence as a disruptive medium for improving policing, and have created the Global Policing Database (GPD), a joint venture between Australia and the UK, an in-process tool to assist such disruption. The database takes a broad view of policing, collecting experimental and quasi-experimental studies of police inputs, processes, outputs and outcomes. The selection for study inclusion is broader than RCT, yet it stays close to such methods, thereby contradicting in part earlier chapters in this volume. Illustrative of this bandwidth of studies, the GPD initially identified 103,048 unique records. From this pool the GPD winnows the eligible to 80 that are used in the beta testing of the database. Cases were excluded by many criteria, nonetheless like many of the RCT advocates, the resulting pool of studies is minuscule compared to the original. Chapter 9 focuses on the efforts of the What Works Centre for Crime Reduction in the UK — a collaborative involving seven What Works Research Centre’s, each seeking to better interject evidence into policy making. Much of this effort is translational, digesting ‘evidence’ from several sources, thereby making it accessible to the police. An interesting idea employed in this effort is the use of “evidence champions”, those within police agencies that help broker information, making it more useful to their agencies. This is an interesting account of capacity building for evidence; that is, strengthening agency competence and confidence to use scientific evidence.
Knutsson in Chapter 10 offers a cautionary tale of the non-linear way evidence seeps into police thinking, if it seeps in at all. In some way the chapter can be seen as a lament of an academic about how findings are used or cast off in the “real world.” Alternatively, the chapter can be seen as an account of struggles to translate academic research for police practitioners, and the often structural and cultural inhibitors of such translation, within the academy and the police. The chapter over-relies, however, on academic evidence as leading police decision making.
Chapters 11 and 12 focus our attention on the necessary and underlying partnerships that evidence-making requires, most especially the nature of police-research partnerships. Both chapters underscore the importance of trust and communications for improved application and ownership of evidence in policing, themes that have been around for some considerable period of time, perhaps ever since the police and the academy collided. What is clear from both chapters is that acceptance and adoption of evidence-based practice remains largely an individual rather than an institutional practice. Both outline ways underway to better integrate evidence into policing, and the structural and cultural barriers in both the academy and in policing that hamper such adoption. Adam Crawford (Chapter 12) suggests that “science is not enough,” referring to the underlying social values and preferences that shape definitions of social problems and acceptable ways of addressing them. For Crawford, like many others, “RCT and EBP have blind spots when it comes to politics and values.” Knowledge co-production is urged in both chapters, wherein police and researcher turn away from the “dialogue of the deaf” to a more interactive, communicative and engaged model of police research. Tompson and her colleagues (Chapter 11) echo these concerns suggesting that absent such respectful interaction, police decision-making and police research will remain distant.
Advances in Evidence-Based Policing reflects some of the schizophrenia of EBP efforts. On one hand the field and indeed policing should be informed by the best evidence available. On the other hand, by maintaining a narrow definition of what constitutes evidence, the complexities of social policy making, including crime control and prevention policy and practice are often minimized in the discussion. It is always quite interesting to see systematic reviews of police practice or crime prevention begin with statements like we identified 5,000 or so possible studies for inclusion, but only 12 met our criteria. While the 4,988 discarded studies may be less optimal, it certainly cannot be the case that they offer nothing in understanding police practices, interventions, and the like. Advances in Evidence-Based Policing reveals some of this schizophrenia, while at the same time calling attention to ways EBP can be expanded and made more impactful. It is perhaps a start in reconciling EBP with police and crime prevention scholarship writ large.
Jack R. Greene, Professor Emeritus, School of Criminal Justice and Criminology, Northeastern University
