- 1 slider, Cure Violence, Jeffrey Butts, Jeremy Porter, JohnJayREC Products, Monographs

Denormalizing Violence

Evaluation Framework for a Public Health Model of Violence Prevention

button_downloadpfby Jeffrey A. Butts with Lindsay Bostwick and Jeremy Porter
Research & Evaluation Center
March 2014

Acknowledgements

Preparation of this report was supported by grants from the New York City Council and the Robert Wood Johnson Foundation. The author wishes to thank the members of the New York City Council and especially Jumaane Williams and Fernando Cabrera for this support, as well as the officers and program staff of the Robert Wood Johnson Foundation, especially Dr. Laura Leviton. In addition, the authors are grateful for the support offered by the City Council staff. Finally, this report would not have been possible without the guidance and contributions of our colleague, Dr. Caterina Roman of Temple University, as well as several staff members in the Research & Evaluation Center, including Sheyla Delgado, Evan Misshula, Laura Negredo, and Kathy Tomberg.

Executive Summary

Despite having one of the lowest murder rates among major U.S. cities, gun violence continues to be a serious problem in New York City. In 2011, the New York City Council created the Task Force to Combat Gun Violence. In a December 2012 report, the Task Force recommended the initiation of a multi-agency and multi-disciplinary “crisis management system” to reduce the incidence and severity of gun violence. The system was based on the Cure Violence model of violence reduction.

Cure Violence utilizes a public health approach. It considers gun violence to be analogous to a communicable disease that passes from person to person when left untreated. According to the logic of Cure Violence, gun violence is most effectively reduced by changing the behavior of individuals at risk to participate in gun violence and “denormalizing” violence by working to change the community norms that support and perpetuate gun violence.

The Research & Evaluation Center at John Jay College of Criminal Justice is currently evaluating the effectiveness of the Cure Violence approach to violence reduction. Between April 2013 and February 2014, staff from the Center began the project by visiting Cure Violence sites in New York City and Chicago, the home base for Cure Violence. Researchers observed the operation of the program and assessed the suitability of the model for detailed evaluation. The team reviewed documents and websites about the project, interviewed program staff, and spoke with local officials involved in the design and launch of the initiative in both cities. Researchers also met with staff of the Robert Wood Johnson Foundation, which supports the Cure Violence model and contributed partial funding for this research.

The following report addresses the operations of the Cure Violence model and how it differs from other approaches for reducing gun violence. It reviews the evidence underlying these models and proposes an agenda for future evaluation research.

INTRODUCTION

web_fig1Communities throughout the United States and around the world are experimenting with varying methods to prevent and reduce violence — especially gun violence. A number of strategies appear to be promising. Most existing models focus on suppression, deterrence or both, and they involve varying theoretical assumptions and implementation challenges.

Suppression models attempt to extinguish violent behavior with aggressive law enforcement alone. These approaches can generate immediate results, but they require the coordination of complex bureaucracies that must be supported and sustained to control the diffusion of violence, which is difficult and costly. The difficulties inherent in suppression are illustrated by how often police officials use the cliché, “we can’t arrest our way out of this problem.”

Deterrence models change the incentives that attract individuals to violence, mostly by increasing the severity or certainty of punishment but also by creating alternatives. Deterrence requires more time to generate results, but it has greater long-term potential and it can be less costly. Deterrence can be combined with supportive and rehabilitative efforts that entice people to change their behavior without relying on force, but some models depend heavily on the threat of punishment and the power to make good on such threats.

The particular strategy involved in violence prevention can have profound cost implications. Some deterrence models require large investments in infrastructure. If the goal of a deterrence-based model is to impose swift and certain punishment on some offenders in order to deter offending by others, the program must be able deliver on the threat of punishment. To create a credible threat, the entire system of police, prosecutors, and prisons must be available and deployable, and this entails significant costs. The key question is whether violence can be reduced without relying on justice infrastructure to deliver swift and certain punishment. Is there another choice? Does public safety always require the formal bureaucracies of justice?

This report describes a different model, one based on the concepts of public health (Leviton, Rhodes and Chang 2011). A public health approach to reducing gun violence does not involve the use of force or even the threat of force. It presumes that violent behavior — like all behavior — responds to structures, incentives, and norms. In other words, people behave violently when violence is normal, accepted, or even expected by their friends, family and neighbors. A public health approach creates incentives to encourage desired behavior and to discourage unwanted behavior, but it does this without relying on punishment. It tries to “denormalize” harmful behavior (Bell et al. 2010).

web_fig2Among existing strategies for reducing violence that embody more of a public health approach, one of the most established programs is Cure Violence (formerly known as the Chicago-CeaseFire project). Cure Violence works to harness the forces that shape behavior and to do so in a way that denormalizes violence to stop its transmission. It focuses on social relationships and norms. Cure Violence is designed to operate independently of law enforcement, while hopefully not undermining law enforcement.

Cure Violence is not inherently incompatible with models that depend heavily on law enforcement. A community trying to reduce gun violence might reasonably choose to implement more than one approach in a comprehensive strategy. Currently, however, public officials are not likely to see all models as equals. Policymakers are more likely to invest in law enforcement — not necessarily because law enforcement is always the best strategy for any problem, but because enforcement is familiar. Violence reduction models that rely on public health concepts, social services, and nonprofessional staff do not fit as easily in traditional policy frameworks. When a program staff involves former gang members and previously incarcerated offenders, it will be much harder for public officials to embrace it.

One way to increase the utility of alternative approaches to violence reduction would be to expand the array of research literature and other materials that policymakers may draw upon to support such approaches. The remainder of this report is designed to contribute to such an effort. It describes the thinking behind the Cure Violence model and reviews the available evidence in support of the model. The report concludes by recommending a strategy for pursuing high-quality evaluations of Cure Violence. First, however, the report addresses the extent of research evidence for deterrence-based models.

DETERRENCE MODELS

web_fig3Deterrence-based crime reduction strategies have proliferated in recent decades. The growth of deterrence as an explicit program goal is consistent with a general shift in American crime policy, from a focus on suppression and control to approaches inspired by social and behavioral science (Weisburd and Eck 2004). Research suggests that deterrence holds real promise for the reduction of crime and violence. In a recent review, McGarrell and his colleagues (2013) described four approaches that appear to be effective in reducing gun violence. One of the strategies they identified was Cure Violence, but the other three models depended on deterrence and targeted law enforcement:

  • Project Safe Neighborhoods / Project Exile. Through collaborations of federal and local prosecutors, gun cases are shifted to federal courts, with no-bail provisions, long sentences, and incarceration in federal rather than state prison. Evaluation results suggest that such efforts may have a modest but independent effect on gun crimes, even when controlling for other factors usually thought to influence rates of gun violence.
  • Boston Ceasefire. Once known as “pulling levers,” this approach traces its heritage to the Boston Gun Project. Specific people, groups, and places are pinpointed for enhanced enforcement due to their association with gun violence. Enhanced enforcement is achieved with a collaboration of law enforcement and community partners. Evaluations suggest the model can be effective in reducing incidents of violence.
  • Drug Market Intervention. In a variant of the Boston Ceasefire approach, law enforcement officials shut down illegal, open-air drug markets. Police concentrate their efforts to incapacitate chronic offenders involved in drug selling and to divert lower-level drug offenders to community-based alternatives.

These strategies have been replicated across the country, often with federal funding. Some of the more well-known initiatives include the Strategic Approaches to Community Safety Initiative (SACSI), Project Safe Neighborhoods (PSN), and the Group Violence Reduction Strategy (GVRS).

Strategic Approaches to Community Safety Initiative (SACSI)

Inspired by the apparent success of focused deterrence efforts in Boston during the mid-1990s, the U.S. Department of Justice (DOJ) replicated the approach in other cities by funding the Strategic Approaches to Community Safety Initiative (SACSI). The program facilitated the implementation of deterrence strategies in ten additional cities across the United States. Phase 1 of the program began in 1998 in Indianapolis, Memphis, New Haven, Portland (Oregon), and Winston-Salem. Phase 2 launched in 2000 with efforts in Albuquerque, Atlanta, Detroit, Rochester, and St. Louis.

Each of the SACSI sites formed a multi-agency collaborative, including federal partners in the U.S. Attorney’s Office. These collaboratives shared responsibility for coordinating the intervention effort. Each city’s strategy was allowed to evolve as needed. For example, projects in Albuquerque, Atlanta, Detroit, Indianapolis, and New Haven had core collaborative partners belonging to the law enforcement or criminal justice sectors. In other cities, core collaborative groups were more varied, often including social service agencies and community groups.

In addition to establishing these core collaborative groups, each SACSI site was directed to work with local researchers to identify target areas, to implement the intervention strategy in those areas, and to evaluate the initiative’s effectiveness. Most SACSI sites used strategies associated with the focused deterrence model, including offender notification or “call-in” meetings. In some sites, the SACSI strategy resembled a suppression model, including increased monitoring of probationers and parolees, law enforcement “sweeps” of high-crime areas, and the collaboration of state and federal prosecutors to enhance the enforcement of all gun crimes.

An evaluation of SACSI funded by the U.S. Department of Justice compared the implementation of the program across the sites using local crime data as well as data from the FBI’s Uniform Crime Reporting (UCR) program (Roehl et al. 2008). Data availability varied by city. Some sites relied exclusively on data from the particular areas where SACSI was implemented. Others looked at city-wide changes. Violent crimes declined in targeted cities and in some sites the decreases were greater in targeted neighborhoods than in the city as a whole.

web_fig4The SACSI demonstration and evaluation suffered from the same problem facing nearly all evaluations from this period. Without a research design to control for factors not related to the initiative — such as a randomized experiment — it was impossible to draw firm conclusions about impact. The intervention was tracked during a time of generally declining crime rates in the U.S., so the fact that violent crime fell in the SACSI sites was not proof of effectiveness.

Evaluators in the SACSI study used UCR data to examine crime trends in each city before and after the intervention. Most of the sites had been seeing generally declining violent crime rates during the pre-intervention period. Researchers did see steeper decreases after intervention (2000 to 2003) and the declines were greater in SACSI sites than in all cities with populations of 100,000 or more. Cities in Phase 2 of SACSI, however, had smaller declines than cities in Phase 1, and when crime trends in Phase 2 sites were analyzed along with matched comparison cities they were not significantly different.

The results of the SACSI evaluation provided some evidence that the initiative contributed to falling crime rates, but the conclusions were less than solid and it was difficult to draw definitive conclusions. Even in cities where violence in SACSI neighborhoods fell more than in non-SACSI neighborhoods, such a difference would be consistent with the phenomenon known as “regression to the mean.” Each city placed its intervention in an area with a high rate of violent crime. When crime rates began to fall everywhere, the rates in high-crime areas were likely to fall more dramatically — simply as a matter of probability. Most crime prevention studies during the past two decades have faced this same problem.

web_fig4bProject Safe Neighborhoods

web_fig5Following the SACSI effort, the DOJ folded the program into another initiative known as PSN, or Project Safe Neighborhoods. The PSN program included many elements of the focused deterrence approach along with some elements of the suppression model. In the Offices of the United States Attorneys, working groups of federal, state, and local criminal justice agencies collaborated to implement intervention strategies aimed at reducing gun violence. Working groups included researchers to assist in identifying and understanding the drivers of gun violence.
PSN sites were required to include five core components in their strategies:

  1. Multi-agency collaborations of local, state, and federal law enforcement and criminal justice agencies as well as researchers. Sites were encouraged to include social service providers, community organizations, and local businesses in the working groups.
  2. Strategic planning that drew upon information and research. The plans had to include methods for monitoring and evaluating the effort.
  3. Training and technical assistance on elements of each site’s intervention strategy.
  4. Public education and outreach to educate the public and potential offenders about the sanctions and costs of gun crime.
  5. Progress reports on a bi-annual basis about project activities and routine measures of gun crime and violence.

McGarrell and his colleagues (2009; 2010; 2013) confirmed that PSN was consistent with focused deterrence. It targeted specific individuals or places in need of increased attention and resources. The results of the effort, however, were mixed. Sites varied in their adherence to key components of the model. Some failed to integrate research into the effort. This variability prohibited definitive conclusions. The research team asserted that the PSN model’s multi-agency and problem-solving orientation “holds promise,” but the study mainly pointed out the need for ongoing experimentation (McGarrell et al. 2010: 165).

Group Violence Reduction Strategy

The most well-known model within the focused deterrence framework evolved directly from the Boston Gun Project, or Operation Ceasefire in the 1990s. Once referred to as “pulling levers” (Kennedy 1997; 1998; 2006), and more recently the Group Violence Reduction Strategy (GVRS), the approach is being employed in numerous gun violence intervention programs throughout the U.S. and internationally (Kennedy 2011).

web_fig6Responding to a growing number of homicides in the early 1990s, the City of Boston developed its program for addressing youth homicides by working with researchers from Harvard University’s Kennedy School of Government. Researchers identified the most influential drivers of youth violence and by working with a multi-agency collaborative from criminal justice and social services, they fashioned the Boston Gun Project.

By 1996, the project had created a program to focus enforcement resources on gangs and traffickers of guns. Individuals involved in violence, particularly gun violence, were invited to “call-in” meetings where they were informed directly that further violence would not be tolerated. Until the violence stopped, they were told by police, prosecutors and other project partners that heavy enforcement efforts in their communities would continue. Researchers quoted a Boston law enforcement official to characterize the message delivered to active gang members (Kennedy, Braga and Piehl 2001: 27-28.):

“We’re here because of the shooting… We’re not going to leave until it stops. And until it does, nobody is going to so much as jaywalk, nor make any money, nor have any fun.”

The Boston Gun Project is known as a targeted enforcement strategy, but it was also a comprehensive approach that incorporated street outreach. The street workers in Boston were essentially social service case managers and advocates who worked closely with those youth thought to be at high risk for future violence. Street workers linked youth with services and mediated conflicts whenever possible. Community groups were also engaged to spread the message to gangs that violence would be tolerated no longer.

These efforts became known as the Cease Fire model, and later the Group Violence Reduction Strategy. Like all focused deterrence approaches, the GVRS relies on close partnerships with law enforcement agencies and prosecutors to deliver swifter and more certain punishment for individuals and groups who persist in using violence. In most cities, GVRS involves the participation of families, neighbors, community agencies, faith organizations, and social service agencies as well, but the model depends on the availability and demonstrated power of enforcement. The most active and violent offenders must know that the justice system will follow through on its promise to focus all available resources on anyone continuing to participate in violence.

Early studies of the Boston program found a 63 percent reduction in average monthly youth homicide victims, a 32 percent reduction in average monthly calls to police for “shots-fired,” and a 44 percent reduction in youth gun assault incidents (Braga et al. 2001). Researchers compared the monthly number of youth homicide victims (ages 24 and younger), reports of shots fired, and gun assaults before and after the intervention (1991-1996 versus 1996-1998). Significant improvements were detected in all of the measures. Again, however, violence was falling nationwide at that time. Without the use of either comparison jurisdictions or any form of randomized controlled trial, it was not possible to attribute these reductions to the intervention alone. Other researchers pointed out that the time period used as the pre-intervention baseline was also an unusually active period for youth homicides in Boston. When this was taken into account, the effectiveness of the program was less clear (Levitt 2004; Cook and Ludwig 2006).

Still, the basic strategy first developed in Boston remains one of the most successful deterrence-based approaches to community-level violence reduction. It is being tested in cities through the country and internationally (Kennedy 2011). [For further information about the effects of the Boston strategy, see Corsaro and McGarrell 2009 and 2010, Kennedy 2009, and Rhoel et al. 2008.]

The weight of available evidence continues to support the focused deterrence model (Braga and Weisburd 2012; Braga and Winship 2006; Kennedy 2006 and 2011; McGarrell et al. 2006). None of the research supporting the approach is perfect. Existing studies underscore the difficulties inherent in implementing and measuring the effects of gun violence programs. Few research designs are able to control for the many types of confounding factors that influence violence apart from whatever intervention is being studied. A small number of evaluations in the field of gun violence have used experimental designs, but most have relied on weaker techniques. Moreover, data quality and data availability issues always complicate the estimation of effect size.

Braga and Weisburd (2012) conducted a meta-analysis that assessed the overall effect of the approach based on 11 studies that utilized either randomized controlled trials or quasi-experimental designs with comparison groups. The strategies included two Drug Market Intervention (DMI) programs arising from the Boston work, three CeaseFire programs, two Project Safe Neighborhood programs, and the Hawaii Opportunity with Probation Enforcement (HOPE) program, which maximizes the certainty rather than the severity of sanctions.

Weighing the results of all 11 studies, the analysis showed that focused deterrence has a moderate and desirable effect on crime with no studies reporting significant displacement of crime (e.g., criminal activity did not simply move from one location to another after implementation). Deterrence programs that concentrate enforcement resources on gangs or groups seem to have stronger effects than deterrence focused on individuals. Few randomized controlled trials are available to support the approach, however, and studies with weaker designs may be inflating the apparent effects of the model. Researchers must continue to evaluate these programs.

The focused deterrence approach is also potentially expensive. The model depends on the demonstrated capacity (and occasional use) of what must appear to be the overwhelming power of the state. For focused deterrence to have the intended effect, targeted individuals and groups must believe that the state has the ability to follow through on its threats, including aggressive prosecution and extended confinement (often in federal rather than state prison). Maintaining these resources is costly, and while the justice system obviously exists independently of any particular program, some of these costs would have to be included in a comprehensive evaluation of focused deterrence.

THE CURE VIOLENCE MODEL

The focused deterrence approach is likely to be a key component in any effort to reduce gun violence, but is it the only strategy that should be employed by communities? Will enforcement-backed approaches suffice if they are not followed by deeper, social change? Real and lasting progress in the fight against gun violence may require changing the social norms and attitudes that perpetuate violence. This brings the conversation back to public health approaches.

The original developers of the focused deterrence model recognized the importance of the public health approach. As the Boston gun strategy evolved in the 1990s, the project developers supplemented the efforts of law enforcement with street workers and community partners who worked to change attitudes among high-risk youth and to provide them with supportive services and opportunities (Kennedy 2011).

An effective public health approach to denormalize gun violence may be a valuable complement to the law enforcement strategies of focused deterrence. All such efforts, however, must be designed intentionally and monitored carefully. Policymakers need clear guidelines about what a public health model for reducing gun violence looks like. It should be more than just the provision of social services. A public health model should be constructed with its own theoretical framework. Fortunately, a violence reduction model compatible with the public health perspective already exists.

Key Components

Cure Violence (CV) is a promising effort to bring the insights and methods of public health into an organized program for preventing gun violence (McGarrell et al. 2013). The CV model works with the forces that shape behavior in a way that denormalizes violence to stop its transmission. The model relies on three key elements to stop the transmission of violent behavior:

  • interrupting transmission directly;
  • identifying and changing the thinking of potential transmitters; and
  • changing group norms regarding violence.

The mix of staff members in Cure Violence programs reflects these components. Some workers in Cure Violence programs are hired to stop violent incidents through direct intervention. These individuals, known as “violence interrupters” (VIs), are selected for their experience and backgrounds and their ability to establish relationships with the most high-risk young people in the community, usually young men between the ages of 15 and 30. The VIs form relationships with high-risk youth and monitor ongoing disputes to learn about potential acts of retaliation before they happen. When someone is injured or shot, the victim’s friends and associates are likely to seek revenge. The VIs from Cure Violence seek out those associates and try to “talk them down” or persuade them that there are other ways to negotiate the conflict without engaging in more violence that could risk their liberty and their own lives.

web_fig8Violence interrupters must be carefully recruited. They need to be seen as “credible messengers” by the most high-risk young people in the community. They need to know about the daily routines of people who are involved in criminal lifestyles. They cannot be judgmental or be perceived as outsiders. Ideally, they should come from the same neighborhoods in which they are working, and they should have had some personal experience with the justice system themselves, perhaps as a gang member or a formerly incarcerated person.

Another key position in Cure Violence is the “outreach worker” (OW). Outreach workers are similar to case managers. Like the VIs, the OWs need to have trusting relationships with the most high-risk individuals in the community, and it helps if the OWs have also had prior justice system involvement. Both the VIs and the OWs need to be seen as credible by young people living high-risk lives.

web_fig9The daily tasks of OWs, however, are not as focused as those of the VIs on monitoring threats of violence and intervening directly. Instead, OWs use their relationships with program participants to help connect those individuals with opportunities and resources in the community, including employment, housing, recreational activities, and educational opportunities. Outreach workers provide high-risk young people with positive alternatives to violent lifestyles. The central goal of an OW is to facilitate the process by which potentially violent individuals learn to think about violence differently and to change their behavior accordingly.

Outreach workers and violence interrupters work in teams along with their supervisors and program directors. They meet on a regular basis — often daily — to review their interactions around the neighborhood and to discuss those individuals who are thought to present the greatest current risk of violence. They compare notes on potential incidents of violence and assess the needs and interests of program participants in order to match participants with resources and opportunities that may draw them out of a violent lifestyle. Regular observations from all the workers in a CV site are organized in case planning sessions and much of the information is recorded in a continually updated database — with minimal identifiable information. Individual participants in CV programs are described in meetings using pseudonyms (e.g., Individual A, Individual B) in order to preserve their anonymity and their cooperation.

While the VIs and OWs focus their efforts on young people who are most at risk of transmitting violence, the Cure Violence program pursues another key element of the model. Staff members work to change social norms about violence in the broader community using a variety of activities, including media campaigns, signs and billboards, and public events such as anti-violence marches and post-shooting vigils. The CV program pursues any activity that exposes the larger community to an effective anti-violence message and that builds a general social consensus against acts of violence. In this way, the Cure Violence model works at both ends of the spectrum of behavioral transmission—both the senders and receivers of social messages related to violence and the acceptance of violence.

Research Evidence

Several studies have documented the efficacy of the Cure Violence model by monitoring implementation and outcomes associated with the program. Researchers have studied Cure Violence in Chicago (a program known as “Chicago-CeaseFire”), Baltimore (the “Safe Streets” Program), and New York City (“Save Our Streets” in Brooklyn).

Chicago

web_fig11In the first rigorous study of the Cure Violence approach, researchers worked with a number of sites in Chicago to track both process and outcomes of the program (Skogan et al. 2009). Process measures focused on issues such as site selection, staff training, and the capabilities and quality of the host organization. Outcomes included shooting incident patterns in intervention neighborhoods compared with trends in nearby areas that matched the characteristics of the selected sites. In addition, researchers used social network analysis to document and assess changes in the density of gang violence and the reciprocity of homicide across gangs. Geographic analyses assessed changes in the size and intensity of hot spot areas before and after intervention. Twenty-one neighborhoods were involved in the study.

The program sites faced implementation obstacles almost immediately. Neighborhoods were selected based on their high rates of violence, but the communities tended to be high-poverty areas as well. Skogan and his colleagues noted that selected neighborhoods were inevitably harder to serve due to high levels of disorganization and a dearth of community leaders willing to serve as hosts for violence-reduction work. For many neighborhoods, the initiative was a new program, taking a new approach to the reduction of violence. Some community leaders were cynical about the program from the very beginning of the initiative.

Program participants (i.e., clients) were recruited through outreach workers in each neighborhood and the programs focused on the most high-risk individuals involved in street violence. A survey of program participants by the research team indicated that the outreach effort was successful in reaching the intended clientele. More than four of every five (82%) clients had been arrested previously. Nearly half (45%) had five or more prior arrests and more than half (56%) reported at least some previous incarceration.

Participants believed the program was “very important” despite the large number of obstacles and pressures they faced in their day-to-day lives (Skogan et al. 2009). They viewed the violence interrupter staff, in particular, as essential to the program. The VIs were successful in conveying the program’s anti-violence message to participants. Study results suggested that VIs were critical in diffusing the very type of violent confrontations that often lead to retaliation. The personal networks of violence interrupters often crossed geographic boundaries and the evaluation suggested that VI collaborations across neighborhoods helped to uncover brewing conflicts that would have been missed by an exclusive focus on one neighborhood. Skogan and his colleagues estimated that as much as 70 percent of the individuals and 60 percent of the conflicts mediated by program workers originated outside of the target areas designated for particular interrupters.

web_fig10Like all studies of gun violence, of course, the results of the Chicago study were mixed. Shootings and homicides declined in most of the intervention sites, but crime was decreasing generally in Chicago and continued to do so during the study period. The study included comparison sites, but their selection was not controlled. Only seven of the 21 program sites were able to produce enough data to examine crime trends and possible covariates in any detail during the study time period. Researchers were unable to obtain enough data to address why and how crime fell in each neighborhood. Inconsistencies in the actual timing of program implementation made trend analysis problematic and not reliable.

In addition, variations in the strength of programs across different sites could not be measured accurately given the level of resources devoted to the study. Finally, the violence interrupter component of the program could not be tracked to a specific target location because — as noted above — the work of the interrupters sometimes spilled over geographic borders.

Other important limitations of the evaluation include the following:

  • The study did not evaluate changes in community norms regarding gun violence.
  • The study did not assess the impact of program involvement on behavior change among participants.
  • The study did not study the behavior change process and norm change process through ethnography or some type of in-depth qualitative study.
  • The study did not link implementation data to changes in shootings.
  • The study did not systematically compare intermediate outcomes or performance measures with outcomes.

Baltimore

web_fig12Researchers from the public health school at Johns Hopkins University recently measured the effects of Baltimore’s “Safe Streets” program, a replication of the Cure Violence strategy. The key elements of the study were similar to the work of Skogan and his colleagues in Chicago. The study included: 1) a review of implementation data; 2) an analysis of differences in homicides and non-fatal shootings between implementation areas and nearby comparison areas; 3) surveys of community attitudes toward gun violence; and 4) surveys of program participants that measured their opinions and perceptions about the program and its impact on their lives (Webster et al. 2012a; 2012b).

Implementation began with a solicitation for program proposals from community organizations in Baltimore’s most violent neighborhoods. Safe Streets attempted to gain traction in five neighborhoods, but in one (Union Square) the effort was unsuccessful as it proved too difficult to establish a reliable group of outreach workers (Webster et al. 2012a). Ultimately, the intervention program was implemented in four neighborhoods, all located in Southwest and East Baltimore. As part of the evaluation, border neighborhoods were examined for trends associated with crime mobility and spatial lag effects, but only informally. Formal comparison neighborhoods were dispersed throughout the larger Baltimore area.

Learning from some of the issues that were problematic in the Chicago study, the Baltimore evaluation team required program staff to keep detailed records of each conflict they mediated. Over a 3.5 year period, 276 incidents were successfully mediated; 88 percent of the conflicts involved individuals with violent histories and 75 percent involved known gang members. Webster and his colleagues (2012a) reported that outreach workers believed that nearly all (85%) of the mediations involved incidents that were “likely” or “very likely” to have resulted in shootings had they not been interrupted.

web_fig12bTo evaluate overall program effects, shooting and homicide data from the intervention sites were compared with data from other precincts that ranked in the top 25 percent of city precincts for number of shootings during the 2003-2006 period. Similar to the Chicago results, the findings of the Safe Streets evaluation were promising, but mixed. Overall, there was very little deviation from general trends during the implementation of the project. In some cases, homicides spiked at the same time that non-fatal shootings decreased significantly.

In all four neighborhoods, one of the two gun violence indicators improved, but not always the same indicator. The researchers believed that the inconsistent pattern was likely due to bias from unmeasured confounders and they offered a series of anecdotal explanations for the pattern. In the end, the final conclusion from the Baltimore project was similar to that of the Chicago study—there were some promising trends, but they could not be attributed reliably to the intervention and they could not be disaggregated from the ongoing national decline in violence.

Brooklyn

web_fig13Cure Violence was implemented in Brooklyn, NY, by a program known as Save Our Streets (or, S.O.S.). Operated by the Crown Heights Community Mediation Center and the Center for Court Innovation (CCI), the program was evaluated by a team of CCI researchers (Picard-Fritsche and Cerniglia 2013). Researchers attempted the same quasi-experimental design used in the Chicago study in order to replicate the findings, even if the intervention model itself was not exactly the same. In the Brooklyn project, outreach workers performed the work of violence interrupters. The research team believed that this alteration did not harm the quality of conflict mediations.

According to the CCI evaluation, gun violence decreased in targeted areas during the intervention period while it increased in comparative neighborhoods, and the difference in the relative trends was statistically significant. The fact that gun crime appeared to decline in the intervention area while not declining elsewhere was seen as a promising result, especially when coupled with the fact that Brooklyn as a whole saw increases in shootings during that time.

Community measures from the study were also encouraging, although they were not obtained with highly rigorous methods. Surveys of community residents showed an increase in awareness of the project’s mobilization against gun violence (from 27% to 73%) and increased confidence in the SOS program itself (from 29% to 55%). However, survey data also indicated that community residents did not feel any safer and they were actually more likely to support the right to carry a gun if they had witnessed a gun-related crime in the past.

What’s Next for Cure Violence

Previous studies of Cure Violence suggest that it is a promising public health approach to violence prevention. In Chicago, gun violence dropped in all sites included in the study’s longitudinal analysis. Geographic concentrations of violence lessened, perhaps due to fewer pronounced hotspots and decreasing gang activity. In Baltimore, program implementation was inconsistent, and even in the strongest neighborhoods the modal outcome was an increase in homicides and a decrease in non-fatal shootings. Brooklyn produced the most encouraging evidence. Gun violence fell while neighboring communities and Brooklyn as a whole experienced increasing violence. Even the Brooklyn study, however, was unable to include a rigorous test for the variety of confounding factors that could affect the results.

All studies of Cure Violence faced common challenges—the principal outcomes of interest were changes in aggregate rates of crime and violence measured at the neighborhood level, and they took place during a time when gun violence (most serious crimes, in fact) were declining nationwide. Downward trajectories in gun violence in one city or in one neighborhood could not be disaggregated from larger national trends. Despite these difficulties, the evidence suggests that Cure Violence may be an effective component in a comprehensive effort to control gun violence.

The public health approach pursued by Cure Violence is different than most models of violence reduction. Cure Violence attempts to shape behavior by relying on the normative power of the social environment rather than on the coercive power of law enforcement and prosecution. It also has the potential to be more cost-effective. While focused deterrence and other law enforcement oriented models require at least the availability — if not always the action — of the bureaucracies of justice (police, prosecutors, prisons), the CV model requires a small group of semi-professional staff working in their own neighborhood.

The key question is whether the Cure Violence model’s simpler and potentially cheaper approach to violence reduction can operate alongside the more established focused deterrence model. The Cure Violence model is appealing in theory, but does it work in practice in a policy environment that is likely to be less welcoming and less supportive of any program whose employees are former offenders and gang members? Can the Cure Violence model co-exist with programs that are closely tied to law enforcement?

Figure 1_commonelements
click to enlarge

For the CV approach to be sustainable, it must be supported by sound evaluation findings. This is true for all social programs and policies, but it is especially true for Cure Violence. A public health approach to violence reduction is not an instinctive choice in American political culture. When elected officials hear the word “violence,” their very next thought is to maximize safety, and public safety is traditionally purchased with bigger, better law enforcement, prosecution, and confinement. In the market for public safety, law enforcement is an appealing and proven product. Asking policymakers to invest in Cure Violence is asking them to take a chance on an entirely different approach.

web_fig14bNo police officer or prosecutor would claim that they alone can stop all gun crime. Everyone recognizes that there are broad cultural, economic, and political forces at work in America’s epidemic of gun violence. Prevention is also universally recognized as an important policy goal. Nobody believes that punishment after the fact is a completely adequate response to crime and violence. Thus, the public health model is in nearly every conversation about the nation’s problem with firearms, and Cure Violence is one of the most well-known programs fitting within a public health approach. The research literature on the CV approach, however, remains inadequate. Building a stronger base of evaluation information is a critical task for the future of Cure Violence.

Evaluation Challenges

A number of programmatic features of the Cure Violence model create significant challenges for evaluation researchers. Front line staff in CV programs — the “violence interrupters” and “outreach workers” — reduce the social harm of violence by focusing their efforts in one neighborhood. They form relationships with anyone already engaged in violence or likely to become involved in violence. Staff members rely on these relationships to learn about and then stop retaliatory acts of violence, and they encourage participants to choose alternative methods of solving disputes and conflicts.

Cure Violence staff do not try to persuade their clients to stop all types of illegal behavior, or to “straighten up.” They focus on violence, especially gun violence. This makes the CV approach different from law enforcement approaches. Active participants in Cure Violence programs know that the staff do not represent any threat to their freedom and liberty, so they are free to interact with staff members and to maintain confidential relationships with them. The participants know that the CV staff is not out to “bust” anyone. Gun violence is the sole focus.

In this way, Cure Violence is a little like Alcoholics Anonymous. The entire focus of the program is on a single problem behavior, although once that behavior is under control staff members are available to help out with jobs, housing, family relationships, etc. Also like AA, participants in a Cure Violence program can be confident that they will not be rejected for relapsing. Staff members may know that a participant is carrying a weapon or even that he or she may be about to use a weapon, but they maintain their singular focus on that individual’s behavior and they work to encourage the person to make a different choice. They do not need to threaten or control program participants. They do not need to collect evidence about them in case a crime is committed. In fact, they don’t even need to know the participant’s real name. Measuring the quality and intensity of these relationships presents a significant challenge for evaluators.

The fact that the CV approach requires a staff of “credible messengers” also introduces complications for evaluators as well as the programs themselves. Key staff members in CV programs need to have some past criminal experience in order to be accepted and fully trusted by participants. This means, of course, that there is always some chance that a staff member may still be involved in criminal behavior. Whether still involved or not, the fact of the staff’s prior criminality increases scrutiny of the program. Public officials and government agencies are likely to react very strongly to even the slightest impropriety involving a Cure Violence staff member. In addition, and perhaps because of this added scrutiny, the staff members of CV are likely to be more skeptical of outsiders, including evaluation researchers.

Without the inclusion of former offenders and formerly incarcerated persons as staff, the Cure Violence strategy would likely not be as effective. Violence interrupters and outreach workers interact with social networks of high-risk offenders and they are able to obtain information that would be unreachable by conventional entities, such as law enforcement authorities and social service agencies. This feature, however, is a serious obstacle for evaluation.

web_fig16If effectiveness of the model depends on the informality and confidential relationships of CV staff and program participants, how can researchers measure program activities and outcomes at the individual level without imposing uncomfortable administrative burdens on staff and without generating negative reactions and suspicions from both staff and clients? The natural instinct of evaluators in this circumstance is to avoid the potential for conflict and to measure outcomes only at the level of neighborhoods and communities. In fact, this is what most previous studies have done. (Of course, this is true in the focused deterrence literature as well.)

Yet, measuring the impact of Cure Violence at the neighborhood level inevitably means that program effects cannot be measured with enough statistical power to generate strong conclusions. It would be far too expensive for most funding agencies to consider an evaluation design that involved rigorous program implementation in 30 intervention areas and 30 control areas, and researchers would still have to measure fidelity to the model in all 30 program areas. Evaluations that measure program effects at the level of individuals would be more affordable and easier to manage and to sustain.

EVALUATION AGENDA FOR CURE VIOLENCE

The main reasons to evaluate any social program or intervention model are: 1) to estimate program effects; and 2) to develop methods for replicating effective programs in other locations. Evaluation studies need to document programs in enough detail to transfer knowledge to new communities and new jurisdictions. It is not sufficient to demonstrate that a strategy worked in one or two sites. Evaluations should show exactly how programs reduce violence and they should point out which practices are related to effectiveness and which are not.

Future evaluations of Cure Violence need to involve measure key components of the program and their effects on intended outcomes, and ideally this would be done at the level of individuals rather than communities. The theoretical framework displayed here was developed by the Research & Evaluation Center at John Jay College to facilitate evaluations of the Cure Violence model and to suggest effective analytic designs.

Figure1_cvlogicmodel
click to enlarge

 

According to the CV program, violence is reduced through two causal pathways: 1) by changing the violent behavior of individuals (e.g., active criminal participants, gang members, and high-risk youth); and 2) by changing the broader social norms that perpetuate violence. Activities pursued by Cure Violence programs are designed to focus on both of these causal pathways. Staff work actively with violent individuals and gang members to prevent ongoing violence, and at the same time they participate in public education campaigns, post-shooting vigils, and other public demonstrations to denormalize violence.

At its heart, Cure Violence is a community-based violence intervention—but part of its theory of change is focused on changing the behavior of high-risk individuals. Past evaluations of Cure Violence have focused largely on changes in aggregate neighborhood units and have not examined individual behavior in detail. Such studies leave many questions unanswered about the workings of the model. Is violence affected at the community level because a large number of individuals were directly influenced by the program to stop shooting, or were community residents in general affected by hearing or seeing the program’s message? Or perhaps the primary causal pathway to the larger community is from individual participants who then influence their immediate social networks. Evaluation research needs to identify and test these hypotheses.

The theoretical framework portrays the two principal pathways of the CV program’s hypothesized effects. The path in shades of blue depicts how program activities lead to changes in individual behavior, both among CV program participants as well as other high-risk youth from the same neighborhoods. The path in shades of green describes the way that the CV model is hypothesized to denormalize violence.

The challenge facing evaluators is how to measure both of these causal pathways using methods of data collection that are sufficiently targeted on key variables, that are feasible in terms of cost and human subjects protections, and that respect the integrity of the program model itself. Measuring individual-level behavior change can be very expensive if a study is designed to collect primary data from individuals themselves rather than relying on imperfect proxy measures from bureaucracies (e.g., police and courts). To collect detailed information about personal behavior and to detect changes in behavior, studies typically have to conduct interviews (or surveys) with active program participants.

This is an expensive and labor-intensive process even with very cooperative and accessible research subjects. The participants in CV programs, however, are not likely to be motivated to collaborate with researchers. An evaluation team cannot simply call up and get a list of local gang members and send them e-mail invitations to participate in a survey. Primary data collection from individuals is essential in an evaluation of Cure Violence because the program is designed to intervene in behaviors, attitudes and norms that are often unknown and may never be known to the formal bureaucracies of social services and law enforcement. In addition to the literacy problems that often prevent the use of written, self-administered surveys with high-risk populations, the participants in Cure Violence programs are unlikely to volunteer their real identities and contact information with program staff. Evaluation teams have to be very careful to establish their independence from the program, but they need to devise some way to make contact with interview subjects. If an evaluation team asked program staff to coerce participants into appointments, they would risk undermining the very nature of the program strategy they are trying to evaluate.

Interviews may need to take place during more than one interaction. Participants in a Cure Violence program are unlikely to be willing to sit through a long protocol of questions about their personal opinions and behavior. Researchers may have to collect interview data in short, intermittent encounters. This approach involves considerable labor costs, as interviewers have a lot of “down time” when they would be on the job but without continual contact with research subjects.

An attractive alternative to face-to-face interviews would be to use Internet-based surveys with touch screens and sound. This could help to overcome literacy problems and might be more engaging for the respondents. This strategy, of course, would add to the project costs and it also presents challenges. Where would the subjects complete the survey? Could they use portable devices such as smart phones or tablet computers? Who would maintain and control access to the equipment? How much theft would occur? Even if successful, how would the study ensure the identity of each subject as the person enters data into the survey site?

A number of sampling challenges also confront CV evaluations. How does one identify a program participant? Would any young person who has had any contact at all with a violence interrupter qualify, for example, or would they have to be more deeply involved and have a longer relationship with an outreach worker? If a known gang member is well acquainted with other program participants, but has never interacted with any staff from the Cure Violence program, should that person qualify as a research subject and be eligible for the interview portion of the study?

The CV model is designed to affect not only the behavior and attitudes of program participants but their social networks as well. A sampling design for using interviews and surveys in an evaluation of Cure Violence may need to distinguish at least three types of research subjects: 1) program participants; 2) other high-risk individuals who are known to and socially networked with program participants; and 3) the broader resident populations of high-risk communities. Surveying or interviewing each of these three groups would present issues related to complexity and cost and could require different strategies.

web_fig21

Measuring Key Elements in the Framework

An evaluation of Cure Violence would likely need to use different data collection strategies for the various areas within the theoretical framework. In the norm change (green) pathway, many of the intermediate outcomes and final outcomes are subjective perceptions and beliefs of community residents. One intermediate outcome refers to the residents of a CV community developing the “self-efficacy” (or confidence in one’s own ability) to bring about change in public safety and violence. To measure this construct with enough precision, an evaluation would have to conduct a probability sample of neighborhood residents, asking questions about perceived community safety and the respondent’s belief that conditions could ever improve.

Many of the activities leading up to these outcomes, however, could be measured using objective data. The role of “community outreach and focus groups,” for example, could be estimated with simple counts of these events, provided that CV program sites were diligent about recording and reporting such events. Similarly, evaluators could track the frequency of public education campaigns by recording how often and where they occur. The bottom two activities in the denormalization pathway (coordination with other community groups and relationships with law enforcement and political leaders) could be measured with a combination of objective data (counts of communications and meetings) and subjective data (surveys of agency staff and local officials).

In the behavior change (blue) pathway, some measures would be relatively simple. Several are already tracked routinely by the databases of the CV national program office. An evaluation, for example, could easily count the frequency of contacts between outreach workers and CV participants with the database. The same approach could be used to monitor street mediations, hospital interventions and other groups and meetings with CV participants and high-risk youth. Each CV program site is already recording these events. An evaluation would just have to make sure that everything was recorded using consistent methods and that the data were captured on regular basis.

Measuring outcomes in the behavior change pathway is more complicated and more expensive. It could be the most expensive portion of any study of Cure Violence. The first consideration would be to decide whether the contribution of program activities should be measured at the individual level in order to establish their empirical relationship with individual behavior. In other words, would it be sufficient to know that the behavior of individuals changed more in jurisdictions with stronger CV implementation, or should the evaluation track each individual’s engagement with the various activities in order to establish their association at the level of individuals? Pursuing the latter would add significant complexity to data collection contacts with CV participants and other high-risk youth.

Even if an evaluation was unable to create detailed measures of program activities at the level of individual participants, more conventional measures could be included. For example, the two outcomes referring to CV participants’ improved access to educational and employment opportunities would be fairly simple. During interviews with youth, researchers could ask a series of questions about these topics. In addition, an evaluation study could ask CV staff to include regular reports about whatever knowledge they had of participant involvement in educational and employment activities.

The third behavior change outcome from the top, “CV participants form pro-social bonds and relationships,” could be measured with fairly simple methods (by asking CV staff to report on the quality and frequency of their contacts with youth) or more complex methods (including questions about relationships and social bonds in youth interviews). Complex methods would be required if the logic of the CV theoretical framework were to be followed closely. This particular casual pathway could also have a secondary influence on other high-risk youth in the community (i.e., non-CV participants), and the transmission of anti-violence attitudes from individuals to groups is critical to the public health approach of violence prevention. As such, researchers should measure this construct at the highest possible level of precision.

The Research & Evaluation Center’s theoretical framework suggests a parallel data collection effort for two additional outcomes among both CV participants and other high-risk youth who are not CV participants. An evaluation design would have to measure these outcomes (learning “nonviolent conflict skills,” and adoption of “nonviolent goals and values”) using face-to-face interviews, as there would be no other source of data about such outcomes for non-participants.

Finally, the last three outcomes in the behavior change pathway present different levels of difficulty. One of the outcomes (“CV participants avoid situations involving the risk of violence”) could be measured during youth interviews, but it could also be estimated using law enforcement or court referral data, as one could assume that avoiding the risk of violence would result in fewer contacts with the justice system. Of course, measuring this outcome would require identifiable data, which could introduce complications. The other outcome (“High-risk youth apply nonviolent approaches to conflict resolution”) would almost have to be measured through youth interviews.

The final outcome in the behavior change pathway is the most traditional outcome measured for violence reduction evaluations. Hypothesized reductions in violence could be measured using official data about gun crimes, shootings, and homicides. While conceptually easy to imagine, measuring this outcome would still require considerable effort. Not all cities are both able and/or willing to share crime data with researchers at the level of addresses or even neighborhoods. In some locations, evaluators would have to negotiate with law enforcement officials to gain access to such data.

CONCLUSION

As cities continue to implement the Cure Violence approach to preventing and reducing gun violence, it is essential that researchers work in collaboration with public officials and community leaders to document and evaluate the effectiveness of the model. Any sound evaluation requires a thorough understanding of the program’s key components and an effective strategy for data collection. The goal of this report is to contribute to these efforts.

The findings of the three most prominent studies of Cure Violence to date are mixed. All three of the evaluations revealed at least some evidence that supports the approach at the level of jurisdictions or communities, but none of the studies could clearly disentangle the results from national and regional trends in violent crime, and there were always “confounding” effects from factors related to sample design, participant selection, and variations in implementation. Of course, this same criticism could be (and is) leveled at the evaluations of other comprehensive violence reduction programs, even those widely perceived as successful.

As described in the Department of Justice’s crimesolutions.gov database — the website that chronicles and synthesizes evidence on criminal justice prevention and intervention programs — the Cure Violence approach currently merits the label “promising” rather than “effective.” Cure Violence, however, offers something to communities that other well-known violence reduction models cannot — it is potentially cost-efficient and it places less demand on the political and administrative resources of law enforcement and the larger criminal justice system. For this reason alone, the model deserves additional investment and investigation.

__________________________
Copyright by the Research & Evaluation Center, John Jay College of Criminal Justice, City University of New York (CUNY).

524 59th Street
New York, NY 10019
www.johnjayrec.nyc
Published online March 2014

Any views expressed here are those of the author and do not necessarily reflect the policies of John Jay College, the City University of New York, or its funders.

Recommended Citation
Butts, Jeffrey with Lindsay Bostwick and Jeremy Porter (2014). Denormalizing Violence: Evaluation Framework for a Public Health Model of Violence Prevention. New York, NY: Research & Evaluation Center, John Jay College of Criminal Justice, City University of New York.

Author
Jeffrey A. Butts is director of the Research & Evaluation Center at John Jay College of Criminal Justice.

Contributors
Lindsay Bostwick is a Ph.D. student at Carnegie Mellon University.
Jeremy Porter is a member of the faculty of Brooklyn College.

 

REFERENCES

Bell, Kirsten, Amy Salmon, Michele Bowers, Jennifer Bell, and Lucy McCullough (2010). Smoking, Stigma and Tobacco ‘Denormalization’: Further Reflections on the use of Stigma as a Public Health Tool. A Commentary on Social Science & Medicine’s Stigma, Prejudice, Discrimination and Health Special Issue (67: 3). Social Science & Medicine, 70(6): 795-799.

Boyle, Douglas J., Jennifer L. Lanterman, Joseph E. Pascarella and Chia-Cherng Cheng (2010). The Impact of Newark’s Operation CeaseFire on Trauma Center Gunshot Wound Admissions. Justice Research and Policy, 12(2): 105-123.

Braga, Anthony A. and David L. Weisburd (2012). The Effects of Focused Deterrence Strategies on Crime — A Systematic Review and Meta-Analysis of the Empirical Evidence. Journal of Research in Crime and Delinquency, 49(3): 323-358.

Braga, Anthony A., David M. Kennedy, Ellen J. Waring, and Anne Morrison Piehl (2001). Problem-Oriented Policing, Deterrence, and Youth Violence: An Evaluation of Boston’s Operation Ceasefire. Journal of Research in Crime and Delinquency, 38(1): 195-225.

Braga, Anthony A. and Christopher Winship (2006). Partnership, Accountability, and Innovation: Clarifying Boston’s Experience with Pulling Levers. In Police Innovation: Contrasting Perspectives, edited by David L. Weisburd and Anthony A. Braga. New York: Cambridge University Press.

Christensen, Clayton M., Heiner Baumann, Rudy Ruggles, and Thomas M. Sadtler (2006). Disruptive Innovation for Social Change. Harvard Business Review, December 84(12): 94-101, 163.

Cook, Philip J. and Jens Ludwig (2006). Aiming for Evidence-based Gun Policy. Journal of Policy Analysis and Management, 25(3): 691-735.

Corsaro, Nicholas and Edmund F. McGarrell (2010). Reducing Homicide Risk in Indianapolis Between 1997 and 2000. Journal of Urban Health 87(5): 851-864.

Corsaro, Nicholas and Edmund F. McGarrell (2009). Testing a Promising Homicide Reduction Strategy: Re-assessing the Impact of the Indianapolis ‘Pulling Levers’ Intervention. Journal of Experimental Criminology, 5(1): 63-82.

Engel, Robin S., Marie Skubak Tillyer, and Nicholas Corsaro (2011). Reducing Gang Violence Using Focused Deterrence: Evaluating the Cincinnati Initiative to Reduce Violence (CIRV). Justice Quarterly, 30(3): 403-439.

Florence, Curtis, Jonathan Shepherd, Iain Brennan, and Thomas Simon (2011). Effectiveness of Anonymised Information Sharing and use in Health Service, Police, and Local Government Partnership for Preventing Violence Related Injury: Experimental Study and Time Series Analysis. British Medical Journal, 342(7812): 1415.

Kennedy, David M. (1997). Pulling Levers: Chronic Offenders, High-Crime Settings, and a Theory of Prevention. Valparaiso University Law Review 31(2): 449-484.

Kennedy, David M. (1998). Pulling Levers: Getting Deterrence Right. National Institute of Justice Journal (July) 236: 2-8.

Kennedy, David M. (2006). Old Wine in New Bottles: Policing and the Lessons of Pulling Levers. In Police Innovation: Contrasting Perspectives, edited by David L. Weisburd and Anthony A. Braga. New York: Cambridge University Press.

Kennedy, David (2011). Don’t Shoot: One man, a Street Fellowship, and the End of Violence in Inner-city America. New York: Bloomsbury.

Kennedy, David M., Anthony A. Braga, and Anne M. Piehl (2001). Reducing Gun Violence — The Boston Gun Project’s Operation Ceasefire. [Research Report]. Washington, DC: U.S. Department of Justice, National Institute of Justice.

Leviton, Laura C., Scott D. Rhodes, and Carol S. Chang (2011). Public Health: Policy, Practice, and Perceptions. Health Care Delivery in the United States (10th Edition). Anthony R. Kovner and James R. Knickman (Editors). New York, NY: Springer.

Levitt, Steven D. (2004). Understanding Why Crime Fell in the 1990s: Four Factors That Explain the Decline and Six That Do Not. Journal of Economic Perspectives, 18(1): 163-190.

Lipsey, Mark W. and David B. Wilson (2000). Practical Meta-analysis. Applied Social Research Methods Series, Volume 49. Thousand Oaks, CA: Sage Publications.

McGarrell, Edmund F., Nicholas Corsaro, Natalie Kroovand Hipple, and Timothy S. Bynum (2010). Project Safe Neighborhoods and Violent Crime Trends in U.S. Cities: Assessing Violent Crime Impact. Journal of Quantitative Criminology, 26(2): 165-190.

McGarrell, Edmund F., Natalie Kroovand Hipple, Timothy S. Bynum, Heather Perez, Karie Gregory, Candice Kane, and Charles Ransford (2013). Promising Strategies for Violence Reduction: Lessons from Two Decades of Innovation. Project Safe Neighborhoods Case Study Report #13. East Lansing, MI: Michigan State University, School of Criminal Justice.

McGarrell, Edmund F., Natalie Kroovand Hipple, Nicholas Corsaro, Timothy S. Bynum, Heather Perez, Carol A. Zimmermann, and Melissa Garmo (2009). Project Safe Neighborhoods – A National Program to Reduce Gun Crime: Final Project Report. Submitted to the National Institute of Justice. East Lansing, MI: Michigan State University, School of Criminal Justice.

McGarrell, Edmund F., Steven Chermak, Jeremy M. Wilson, and Nicholas Corsaro (2006). Reducing Homicide through a ‘Lever-pulling’ Strategy. Justice Quarterly, 23(2): 214-231.

Papachristos, Andrew V. and Christopher Wildeman (2014). Network Exposure and Homicide Victimization in an African American Community. American Journal of Public Health, 104(1): 143-150.

Picard-Fritsche, Sarah, and Lenore Cerniglia (2013). Testing a Public Health Approach to Gun Violence. New York: Center for Court Innovation.

Ritter, Nancy (2009). Ceasefire: A Public Health Approach to Reduce Shootings and Killings. National Institute of Justice Journal, 264: 20-25.

Roehl, Jan, Dennis Rosenbaum, Sandra K. Costello, James R. Coldren, Jr., Amie M. Schuck, Laura Kunard, and David R. Forde (2008). Paving the Way for Project Safe Neighborhoods: SACSI in 10 U.S. Cities. National Institute of Justice Research in Brief. Washington, DC: U.S. Department of Justice, National Institute of Justice. [NCJ216298]

Skogan, Wesley G. (2008). Brief Summary: An Evaluation of CeaseFire-Chicago. Evanston, IL: Institute for Policy Research, Northwestern University.

Skogan, Wesley G., Susan M. Hartnett, Natalie Bump, and Jill Dubois (2009). Evaluation of CeaseFire-Chicago. Evanston, IL: Institute for Policy Research, Northwestern University.

Spergel, Irving A. (2007). Reducing Youth Gang Violence: The Little Village Gang Project in Chicago. Lanham, MD. Rowman AltaMira Press.

Webster, Daniel W., Jennifer Mendel Whitehill, Jon S. Vernick, Elizabeth M. Parker (2012a). Evaluation of Baltimore’s Safe Streets Program: Effects on Attitudes, Participants’ Experiences, and Gun Violence. Baltimore, MD: Johns Hopkins Center for the Prevention of Youth Violence.

Webster, Daniel W., Jennifer Mendel Whitehill, Jon S. Vernick, and Frank C. Curriero (2012b). Effects of Balitmore’s Safe Streets Program on Gun Violence: A Replication of Chicago’s CeaseFire Program. Journal of Urban Health, 90(1): 27-40.

Weisburd, David and John E. Eck (2004). What Can Police Do to Reduce Crime, Disorder, and Fear? Annals of the American Academy of Political and Social Science, 593(1): 42-65.

Wilson, Jeremy M., Steven Chermak, and Edmund F. McGarrell (2011). Community-based Violence Prevention: An Assessment of Pittsburgh’s One Vision One Life Program. Santa Monica, CA: RAND.