The Evidence Generation initiative views research and evaluation as key components of a developmental journey that organizations follow. The generation of evidence is an orientation to practice, part of the daily routine. It is not simply a professional service that organizations purchase from outside contractors and consultants. It is not something that can be put off until an organization needs evidence for new funding applications. Evaluation evidence should be part of an organization’s operational routine from the very beginning.
“Knowledge Changes”
The approach adopted by Evidence Generation was inspired in part by a 2009 essay in The Criminologist, a newsletter of the American Society of Criminology. The essay takes a critical view of conventional evidence-based thinking. It reminds readers to heed the advice of Donald Campbell, who viewed all social policies as experimental and all expected outcomes as hypotheses. Community agencies should pursue the continued generation of evidence and not simply follow the existing knowledge base as if it were fixed and finite.
Public safety practitioners and community leaders must embrace the quest for better evidence while they appreciate the limitations of existing evaluation research. There will never be enough funding or enough competent researchers to test every single aspect of public safety systems or to assess the value of every single practice, program, or policy.
Research funding is scarce and competitive. Only a few models have been fortunate enough to receive the sustained investments required to generate high-quality evaluation evidence. More importantly, research investments are not impartial and unbiased. Investments in research are shaped by budgetary limitations, administrative obstacles, evaluation complexities, political values, organizational self-interest, and simple marketing and promotion. In any field, the practices established by rigorous research may be only a small portion of the total operational environment.
The published findings of past evaluations are not a sufficient basis for making all of the choices involved in building and operating community safety systems. Researchers and practitioners must work together and collaborate in the search for better evidence, and the most urgent need is for evidence about innovative programs and practices that may be overlooked in the competitive struggle for grants and contracts.
If an agency is devoted to cognitive-based therapy or youth mentoring, for example, there are plenty of evaluations to consult in designing a program. If, on the other hand, a program provides educational enhancements, employment readiness, or creative activities that support positive behavior, there is much less evidence — not because these are inherently bad ideas, but because they may not have received adequate funding to participate in competent evaluation efforts.
Moreover, many critical elements of community safety systems cannot be judged by recidivism outcomes. A policy reform effort might be designed simply to lower costs. A restorative justice program might be asked to increase the speed of victim compensation. A legal aid organization might want to measure its effectiveness in court proceedings. A community support effort might focus on building stronger networks of neighborhood residents. There are no compendia of evidence-based practices for all possible strategies related to community safety. Researchers and community organizations must collaborate in developing a more diverse evidence base.
This is the mission of the Evidence Generation initiative — to help innovative community organizations develop stronger capacities for data collection and measurement, and to prepare for future evaluations of whatever strategies they are pursuing to build safer, healthier, and more equitable communities.
Learn More
