EVALUATION CONCEPTS AND PRACTICE

• Evaluation of attempts to control crime is generally forgotten in the rush to address the next big issue, but is essential if practitioners are to learn what works, and what doesn’t to reduce crime.

a. What are we evaluating?

• This is often the most difficult point for practitioners to grasp. Intelligence‐led policing is a business model and as such can be largely successful when analysts interpret the criminal environment effectively, and use that intelligence to influence decision‐makers. If decision‐makers then choose an appropriate crime reduction strategy, but officers in the field fail to implement it correctly, does this mean that the business model of intelligence‐led policing fails?

b. Types of evaluations

• Outcome evaluations tell you whether a crime reduction initiative worked or not. If not, a process evaluation can often tell you why.

c. Operation Vendas and Operation Safe Streets

• The process evaluation of Operation Vendas is immensely useful because it helps to show that the strategy may still be viable, but the implementation was flawed.

• The Safe Streets evaluation by Lawton et al is quantitatively strong, but the more‐process evaluation type writing of Giannetti helps to understand the actual street implementation of the policy across the whole city. Both evaluations work in tandem to provide a fuller picture of the operation.

d. Evaluation skills

• Of all the analytical skills required for analysis, spatial statistics and crime mapping may be the most important for police analysts.

• Spatial skills will have address perennial questions that come up about displacement of crime, and the impact of crime around criminogenic facilities such as bars and nightclubs.

• The weighted displacement quotient helps address many of these questions.

e. Pure evaluations and realistic evaluations

• Both scientific and scientific realist approached have merit in understanding the outcome of attempts to reduce crime and to better organize the flow of information and criminal intelligence around a police department.

• These approaches to evaluation are not mutually exclusive.

f. Case study: Operation Anchorage

• A degree of latitude is required to accept a fixed value for the societal impact of a residential or non‐residential burglary. Obviously each burglary is different in terms of the impact and cost of the crime, but the attempt by staff at the Australian Institute of Criminology to place a value based on aggregate impacts is still valuable.

• per burglary = US $2,240

• per residential burglary = US $1,800

• per non‐residential burglary = US $4,200

• during Anchorage = US $1.17m

• benefit after Anchorage = US $5.5m

MEASURING SUCCESS IN DIFFERENT WAYS

a. The cost‐benefit of surveillance and confidential informants

• The cost of informant handling is disputed, and centers along whether more abstract societal costs (impact on police legitimacy, for example) can be incorporated into a cost evaluation. Furthermore, the true costs of training officers, meeting time and so on are rarely factored into a cost estimate.

b. Measuring disruption

• Disruption is poorly defined and loosely applied.

• Some agencies take considerable liberties in how they apply disruption as a measure of their success. Others under value the importance of their activities.

• The Disruption Attributes Tool is relatively new, and further details will becoming in the second edition of the book Strategic Thinking in Criminal Intelligence.

c. Measuring success in changing business practice

• Hybrid governance is a relatively new term, but is a good description of where security governance is going.

• In New Zealand, I found that assessing each component of the three‐i model was a good way to examine the health of intelligence processes in police departments.

d. Measuring success in performance indicators

• Generally, the approach is often be careful what you wish for. By setting performance indicators, police executives can often drive commanders in ways that they never intended. This is especially the case with Compstat driving short‐term thinking about long‐term crime problems.