Angela Silletti, Anthony Vega, Community Interventions, Evaluation, Jeffrey Butts, John Roman, JohnJayREC Products, Monographs, NYC Mayor's Action Plan, Violence, Wogod Alawlaqi

Measurement Plan and Analytic Strategies for Evaluating the Mayor’s Action Plan for Neighborhood Safety


Update 2

button_opendoc_hiresby Jeffrey A. Butts, John K. Roman, Angela Silletti, Anthony Vega, and Wogod Alawlaqi

INTRODUCTION

This is the second in a series of reports on the evaluation of the New York City Mayor’s Action Plan for Neighborhood Safety (MAP). This Evaluation Update:

  • summarizes the goals and methods used to evaluate the Mayor’s Action Plan;
  • describes the quasi-experimental design used to test the outcomes and impacts of MAP as well as the data sources assembled by the research team and how they are used; and
  • portrays a logical framework the research team used initially to identify causal pathways through which various elements of MAP were intended to achieve their desired effect.

The MAP initiative relies on social supports and public safety improvements to enhance the vibrancy of public spaces, build trust between government and residents, and develop local networks in 17 public housing developments operated by the New York City Housing Authority (NYCHA).* When the evaluation is completed, researchers will gauge the effectiveness of MAP by comparing key outcomes in MAP communities with a matched group of NYCHA communities not participating in MAP (Delgado et al. 2018). This Update introduces the measurement framework and analytic strategies used to design the study.

*  The MAP initiative is often described as an intervention focused on 15 housing developments, but NYCHA considers three of those developments (Red Hook, Queensbridge, and Van Dyke) as comprising two distinct communities each. Thus, this study defines MAP as involving 18 sites. One of those sites, however, is exclusively for older residents (Van Dyke II) and it was excluded from the study. Thus, the evaluation conceptualizes MAP as an initiative affecting 17 communities.

NYC MAYOR’S ACTION PLAN

New York City launched the Mayor’s Action Plan for Neighborhood Safety in 2014, describing it as a “targeted and comprehensive approach” to public safety in housing developments operated by the New York City Housing Authority. The City’s goal is to improve public safety in MAP developments through community empowerment, community connections, and community space. According to the NYC Mayor’s Office of Criminal Justice (MOCJ), the NYCHA developments involved in MAP accounted for one-fifth of all violent crimes reported in the City’s public housing developments in the years preceding MAP.

When MAP began, MOCJ described its core strategies:

  • strengthening police/resident joint problem-solving capabilities in high-crime areas;
  • expanding access to youth employment and community centers;
  • improving security infrastructure in NYCHA developments;
  • creating permanent and routine ways for both residents and City officials to monitor and improve public safety; and,
  • improving shared, outdoor space in NYCHA developments in ways that increase neighborhood activation and reduce opportunities for crime.

John Jay College’s Research and Evaluation Center (JohnJayREC) began the MAP evaluation in 2017 with funding from MOCJ and the City of New York. Researchers designed the study in partnership with NORC at the University of Chicago, a nationally respected public opinion and polling firm. The quasi-experimental (matched comparison group) evaluation combines a wide array of administrative indicators and survey data to estimate differences between MAP developments and NYCHA developments not involved in MAP. Researchers expect to complete the study in 2020.

EVALUATION STRATEGY

The research team is assembling administrative and programmatic data to monitor MAP activities and outcomes in NYCHA developments. Incident reports from law enforcement and patient information from the health care system provide key public safety metrics (crime and injuries due to crime). Researchers are also conducting interviews with staff from MAP partner agencies. Those interviews will allow the study team to create descriptions of MAP activities and identify any obstacles and weaknesses in the MAP strategy. In partnership with NORC, the evaluation team is also surveying residents in each NYCHA development to measure their perceptions of community safety, the availability of services and social supports for local residents, and other indicators of community well-being. All data collected by the research team will be used to identify differences in outcomes between MAP and non-MAP communities and to estimate causal relationships between the efforts of MAP and the outcomes expected to result from those efforts.

As a comprehensive, inter-organizational partnership designed to change basic social conditions in distressed neighborhoods, MAP involves many components. The most effective measurement strategy for such a complicated initiative is to:

  1. assemble the broadest set of salient, pre-existing administrative data sets that can be used to measure inputs, outputs, and outcomes related to MAP; and
  2. target new data collection efforts on the most important and efficient set of other factors that will allow researchers to test the core hypotheses suggested by MAP’s goals and purposes.

Geography of MAP

The MAP initiative is a “place-based” intervention. Its mission is to change places in ways that improve the well-being and safety of the people living in those places. This means the evaluation’s “unit of analysis” is the places involved in MAP and not the people living in those places, which results in a small sample for statistical procedures. Moreover, places do not exist in a vacuum. Each development is made up of smaller places (streets, buildings, etc.) and events that happen in the surrounding neighborhood may affect the lives of NYCHA residents. The study must account for many of these neighborhood characteristics and how the introduction of MAP may have interacted with those characteristics.


The study focuses on the experiences of each NYCHA development as MAP was implemented, and on the degree to which the presence of MAP may have been affected by pre-existing features of the development. The trajectory of public safety in any MAP development is shaped by the history of that development, a history that includes the interaction of the development with the surrounding area as well as the strengths, assets, and risk factors present in that community. The outcomes of MAP are the result of how it affects each place, and the broader crime trends and social indicators present in each place provide important context and added causal factors.

Predictors

Community attributes affect crime in specific places and those attributes can be divided into two types:

  1. the aggregated socioeconomic experiences of the people in a place; and
  2. the attributes of the place itself that either increase or decrease the likelihood of crime.

The prevalence of low-income youth (especially males) who are unconnected to employment, education, or training is often a strong predictor of crime and disorder (Wilson 2012). In practice, such a predictor may be measured using data about the age of residents, unemployment, average income, educational attainment, and the number of residents affected by criminal justice contact (often measured as the average number of residents on formal community supervision) (Brantingham and Brantingham 1993; 1999). In addition, the percentage of recently married residents, those who recently relocated to the neighborhood, the percentage of single-headed households, and the percentage of foreign-born residents may be correlated with levels of crime (Gruenewald et al. 2005).

A neighborhood’s physical characteristics may also be correlated with public safety concerns (Covington and Taylor 1991). Researchers find the number of vacant housing units, neglected yards and lots, the presence of graffiti, and the need for structural repairs, for example, tend to be associated with higher levels of crime and less public safety (Branas et al. 2011; Sampson, Raudenbush, and Earls 1997).

The amount of crime in a community is often considered an indicator of “social disorder,” along with various other deficits related to poor quality housing, low household incomes, and family disruption. A community’s level of social disorder is the result of many factors, including patterns of historical discrimination, economic isolation, and a sustained absence of social investment.

Research suggests the history of disorder in a place is the best predictor of future disorder. Moreover, disorder tends to be concentrated. One study reported that just four percent of street addresses in Minneapolis produced half the calls for police service in that city (Sherman et al. 1989). Half the criminal incidents reported to police in a Seattle study occurred in just five percent of the city’s street segments (Weisburd et al. 2004). Without consistent effort to strengthen communities, disorder may be persistent. The number of street segments accounting for half the crime in Seattle did not vary much over the 14 years covered by the study.

Challenges of Sample Size

The goal of any program evaluation is to detect meaningful, statistically significant relationships between an intervention and its intended outcomes by estimating the difference between two conditions: either the intervention is present (treatment) or it is absent (control). The ability of a study to detect a difference depends largely on two factors:

  1. the actual, true size of the difference; and
  2. the sample size available to estimate the difference.

A study that analyzes 10 people or 10 places would only find evidence of a difference if the true difference were very large, while a sample of 10,000 people or places would likely enable a study to detect even small differences.

 

Researchers must weigh their desires for detailed, specific, and wide-ranging data against the need for sufficient sample sizes. The easiest way to maximize sample size in a place-based study would be to define the sample units (treatment versus control areas) as very small geographic spaces, but this would restrict the availability of data. Information sources relevant to evaluation research tend to be collected at relatively large geographic units, including census tracts, census block groups, and various service districts.

Researchers measuring crime in specific places typically use government census data to measure community attributes while law enforcement data is used to track the occurrence of crime. Law enforcement data, however, may be available only at the level of police districts and census data rarely mirror those districts. Even if the geographic boundaries in datasets match or if some data sources are available in geocoded formats (e.g., specific latitude and longitude), an evaluation is usually disappointed by the variations in key data sets and by how politically-derived boundaries capture community characteristics. Political boundaries rarely reflect how residents think of their own neighborhoods (Groff, Weisburd, and Morris 2009).

Fortunately, the most important boundaries in the MAP evaluation (NYCHA developments) are well-defined, and crime information is available from the New York Police Department with very specific geographic coordinates. Thus, much of the relevant data about MAP can be placed in precise geographic boundaries.

The sample size for the MAP evaluation, however, remains as a challenge. As a place-based intervention, the evaluation needs to detect meaningful differences with a sample of just 34 observations: 17 MAP developments and 17 non-MAP developments. This may be an adequate sample from which to draw many statistical inferences, but small differences may escape detection and the study may find some inputs and outputs do not vary enough across geographic areas to contribute sufficient differences in expected outcomes.

One response to these problems would be to increase the sample size. There are two standard approaches to increasing sample size in placed-based evaluations:

  1. examine effects over time; and
  2. analyze sub-units, such as individuals within places.

Examining effects over time. The evaluation will measure the effects of MAP by analyzing changes in 34 communities during the MAP time period (2014-2019) and a pre-MAP time period (2010-2014). Adding a pre-MAP time period effectively doubles the sample size. Whenever possible, the evaluation will also test the effects of MAP on outcomes measured in time units shorter than years, such as months or weeks. Unfortunately, relatively few key outcomes are measured so frequently.

Adding sub-units. The evaluation will also explore what researchers call “subgroup units,” such as individual respondents in the study’s survey of NYCHA residents. Analyzing two response levels—NYCHA developments and individual responses across developments—will allow the study to answer different questions. If the average effects of MAP are very different within developments, the within-development variation may suggest that MAP was difficult to implement consistently. If the effects of MAP vary across developments, on the other hand, it could suggest that characteristics of the developments themselves shaped the effects of MAP.

MAP as Mediator

A mediating variable lies between the cause of something and its effects. Mediators are often partly responsible for the apparent cause-and-effect relationship between two variables. When a company experiences improvements in staff productivity after introducing casual Fridays, for example, the effect of the relaxed dress code was probably mediated by improvements in staff morale.

In this study, the presence of MAP in a NYCHA development could be considered a mediating variable. Without MAP, whatever factors were producing existing crime and disorder trajectories in a development would be expected to continue as before. The introduction of MAP is hypothesized to produce a change in those trajectories by acting as a mediating variable.

Measuring the influence of MAP presents a number of other challenges. MAP is an ecological framework rather than a discrete intervention. Thus, no single measure can capture the degree to which MAP was implemented in a place. It is what researchers might call a “latent” construct, or an idea that cannot be directly observed and that can only be described empirically by combining other, more readily observable data. Ideally, the data represent valid indicators of the presence and strength of the latent construct.

The MAP evaluation team developed a wide array of such indicators by collecting data about MAP-related activities in each NYCHA development participating in MAP. In the evaluation framework, the resources and activities associated with MAP are considered “inputs” while their immediate results are considered “outputs” and their long-term effects are “outcomes.”

MEASUREMENT FRAMEWORK

Inputs, Outputs, and Intermediate Outcomes

Many MAP activities can be thought of as both outputs (something the MAP program does) and in some cases intermediate outcomes (something the MAP program seeks to improve).

Attendance at a community meeting, for example, serves both objectives. The number of individuals who attend MAP events is a useful indicator of the extent to which MAP was implemented in a given place at a given time—i.e., attendance as an output. But, higher attendance is also an intermediate outcome of MAP—a community is expected to be more engaged when residents attend meetings in large numbers.

Distinguishing outputs from intermediate outcomes is important for understanding the evolution of MAP and provides a method of testing whether differences in implementation lead to expected differences in end outcomes (i.e., crime reduction and public safety).

The MAP initiative involves a wide range of services and supports provided by public and private agencies. To include an estimate of MAP implementation in the evaluation, researchers need to compile an assortment of indicators to represent the activities undertaken in NYCHA developments as part of the initiative.

The activities (inputs) of MAP fall into three broad areas:

  1. Community Empowerment: Increasing a community’s interest in working with government to improve public safety and enhancing the capacity of its residents to do so.
  2. Community Connections: Increasing resident access and involvement with social services and other positive community resources that improve public safety and build a sense of belonging among residents.
  3. Community Space: Increasing the security and quality of shared community space by enhancing a positive sense of ownership among residents, improving the maintenance of public spaces, and expanding the perceptions of activated space in ways that improve public safety.

Some indicators in these three areas are informed by the study team’s direct observations and participant interviews, but most are constructed from the administrative records of service providers and MAP partner agencies. Researchers use administrative data to create indicators of MAP implementation in each area and to test their relationship to key outcomes (Figure 1).

In the area of community empowerment, for example, the MAP initiative builds partnerships and programs to address public safety issues, perceptions of public safety, and trust in government. To support community connections, MAP expanded resident access to a range of social services and supports, while community space improvements included efforts known as Crime Prevention through Environmental Design (or, CPTED). MAP Engagement Coordinators (MECs) and stakeholder groups received training in CPTED strategies and techniques and the MAP sites had access to funding for CPTED projects. Each development proposed specific CPTED projects that were reviewed by a committee and funding was awarded to the approved projects. Some of the CPTED projects addressed crime and perceptions of public safety through the installation of new lighting, new locks, etc. while other projects focused on green space, other improvements to outdoor space, educational and recreational programming, or support for community centers.

Each NYCHA development involved in MAP also participated in Neighborhood Stat (NStat) meetings, both citywide and locally-based meetings. Residents, stakeholders, and representatives of city agencies gathered quarterly to discuss ongoing problems and identify effective solutions to those issues. In between NStat meetings, stakeholder groups in each development met separately to plan and organize other actions to advance the goals of MAP for their community. Stakeholder team members were recruited for their knowledge of community issues and their commitment to the MAP process. They served as a bridge between the general community and the various partner organizations affiliated with MAP. The MECs and stakeholder teams met several times a month to share data, identify public safety concerns, create action plans and implement collaborative solutions.

Stakeholder teams were recruited by the MECs or by other residents. Team members met to discuss community-based issues ranging from garbage collection to youth delinquency and then worked collaboratively to formulate solutions to address the issues. For example, residents of the Saint Nicholas development were concerned about youth congregating in areas between buildings and engaging in unproductive behavior. Services and activities were available in the neighborhood, but the residents believed youth were not interested in becoming involved with existing programs. In collaboration with the youth, the residents proposed on-site social programming to locate activities in the very spaces youth were already occupying. The aim was to involve youth in prosocial activities that addressed high unemployment and low high school graduation rates.

Activities implemented by the stakeholder teams and the actions planned during NStat meetings are available to be used by the evaluation team to measure the presence and intensity of MAP efforts. In addition, the evaluation monitors the efforts of various partner agencies, including Groundswell, the Police Athletic League (PAL), the NYC Department of Parks and Recreation, Cornerstone, and many more. Each agency works with NYCHA developments in ways that expand resident supports and lead to improvements in public space. The evaluation is compiling administrative records of their activities to create indicators of such efforts and to include them in the analysis of MAP’s potential effects.

Similarly, the evaluation is tracking MAP efforts focused on forging stronger connections among NYCHA residents and between residents and the providers of services and supports. The New York City Department of Probation, for example, offers its “Next Steps” program to residents of NYCHA. The Department of Parks and Recreation provides physical activities for youth through its “Kids in Motion” drop-in program, and the Police Athletic League operates several programs that are designed to engage young residents of NYCHA developments.

The central goal of MAP is to improve the social and physical environments of public housing developments in ways that support public safety and enhance the well-being of residents. City agencies and nonprofit service providers offer an expanded range of social services and supportive resources in the NYCHA developments involved in MAP. The evaluation team maintains records of each program as a means of estimating the strength and consistency of MAP implementation and estimating the initiative’s effectiveness in achieving its three core goals: community empowerment, community connections, and community space.

Outcomes and Impacts: MAP Effects

Evaluations must begin with a clear strategy for data collection and analysis—a plan similar to a logic model. Government organizations typically monitor community improvement efforts with logic models that portray the activities and expected outcomes of each activity. These are programmatic logic models, however, and they are largely aspirational. They articulate the best hopes of managers who use the model to hold participants and partners accountable for playing their parts. Evaluation logic models cannot be aspirational. A research logic model (or, measurement framework) must reflect the causal hypotheses suggested by intervention as well as a realistic assessment of data availability.

A useful measurement framework:

  1. identifies key components;
  2. proposes the expected chain of cause-and-effect relationships between those components and their intended outcomes;
  3. portrays how each activity involved in the effort should combine to result in outputs and outcomes; and,
  4. suggests how each component may be measured.

Understanding what outcomes are expected and why they are expected is the basis for establishing an evaluation’s goals and methods. It is a first step in data collection and data analysis. A properly developed, theoretically informed measurement framework increases the ability of an evaluation to demonstrate that an intentional intervention effort led to its desired outcomes rather than an unplanned or fortuitous set of circumstances (Coryn et al. 2010).

Evaluation designs should be developed by systematically organizing what is known about a particular problem and the intended goals of an intervention. The end goal, or impact of the MAP initiative is to improve the quality of life and well-being of NYCHA residents by improving public safety and reducing crime and the fear of crime in selected NYCHA developments. The evaluation hypothesizes that MAP results in these effects by achieving a series of inter-related outcomes (Figure 2).

MAP is expected to result in increased community well-being and public safety by leveraging the influence of improved government competence and effectiveness, broader collaborations between government and communities, expanded access to social resources and resident supports, as well as improvements to the shared spaces of NYCHA developments in ways that create more defensible space and activated space. In turn, these efforts are expected to create stronger government legitimacy (residents have faith in the government’s ability to protect public safety) and enhanced collective efficacy (residents have faith in their own ability to enhance community wellbeing AND are willing and capable of doing so).

The MAP evaluation’s plan for analyzing outcomes and impacts takes its direction from a lengthy heritage of research findings. Previous studies have tested the origins and impacts of these factors in research in a wide range of substantive fields, including economics, social science, public policy, social welfare, healthcare, and criminal justice.

Crime and the Fear of Crime

Crime affects people directly when they become crime victims, but it also affects anyone who fears the possibility of victimization (Box, Hale, and Andrews 1988; Farrall, Jackson, and Gray 2009; Lorenc et al. 2012). Fear of crime can lead to psychological, emotional, and social deficits. When people believe their community is unsafe, they may avoid social ties and be less likely to participate in recreational activities (Stafford, Chandola, and Marmot 2007). Pervasive fear of crime can harm residents’ sense of community and weaken informal social controls. With increased isolation and reduced activity, fear of crime in a community may lead to secondary issues, including mental health problems, depression, and cardiovascular disease (Pain 2000). Researchers also find links between fear of crime and rates of handgun possession (Stroebe, Leander, and Kruglanski 2017).

Fear of crime is not a straightforward reflection of the actual incidence of crime (Skogan 1986). Even people who believe they live in a neighborhood characterized by low levels of violent crime may still experience considerable fear of crime (Foster et al. 2013). When crime begins to fall in a neighborhood formerly affected by high crime rates, it may take years for residents to feel less fear of crime. Social context, however, has a mediating effect. When people feel a strong connection with their neighbors and the larger community, they experience greater resilience and are often able to overcome the fear of crime (Gibson et al. 2002).

Collective Efficacy

Collective efficacy is the willingness of residents to help each other and to intervene in problems affecting the entire community (Browning, Feinberg, and Dietz 2004; Sampson, Raudenbush, and Earls 1997). Higher levels of collective efficacy have been associated with lower rates of violent crime and criminal victimization, even after controlling for neighborhood characteristics including economic and educational disadvantages, residential instability, and resident demographics. Researchers find that higher levels of trust between neighbors lead to more effective uses of informal social control (resident willingness to scrutinize unwanted, public behavior).

Surveys are often used to measure collective efficacy (Collins, Neal, and Neal 2016; Hipp 2016; Sampson, Raudenbush, and Earls 1997). Studies of collective efficacy may include questions such as, “how likely are you to intervene if you see someone in trouble?” “How comfortable would you feel asking a neighbor for help,” and, “would someone call the police for help in your neighborhood?” Researchers find that levels of collective efficacy are often associated with crime rates. Neighborhoods with high levels of economic disadvantage, racial inequality, and low levels of collective efficacy may have higher homicide rates (Morenoff, Sampson, and Raudenbush 2001). Studies also suggest that fear of crime can be a mechanism through which high levels of crime suppress collective efficacy (Markowitz et al. 2001).

Government Legitimacy

Legitimacy is a critical part of the public’s acceptance of governmental authority. According to Suchman (1995: 574), legitimacy is the “generalized perception or assumption that the actions of an entity are desirable, proper, or appropriate within some socially constructed system of norms, values, beliefs and definitions.” Legitimacy derives from the reciprocal relationships of government authority, citizens, and the agencies and organizations comprising government. Citizens and governmental organizations must hold each other accountable to establish mutual trust. When government leaders work for the benefit of the entire population and do so with demonstrated fairness, and when citizens develop confidence and trust in government authority, the government gains legitimacy. Two conditions are required for citizens to perceive a government as legitimate: the decisions and actions of government are objectively fair; and, the government and its agencies demonstrate trustworthiness (Levi, Sacks, and Taylor 2009).

People are more likely to follow the law and cooperate with legal authorities when they view the representatives of law (e.g., police officers and judges) as both legitimate and fair. The combination of legitimacy and fairness in the legal context is often referred to as “procedural justice” (Tyler 2006). Procedural justice contains four key components: civilian involvement and voice, impartiality, respect and dignity, and trustworthy intentions (Goodman-Delahunty 2010; Tyler and Huo 2002). Police, for example, develop legitimacy with community residents by making social connections with them—sometimes even a simple gesture such as remembering a person’s name. When police officers build relationships with citizens, their interactions become less combative. When individual police officers are perceived as fair, neutral in their actions, and respectful to the community, the legitimacy of law enforcement increases (e.g., Mastrofski, Snipes, and Supina 1996; Sunshine and Tyler 2003).

As legal authorities demonstrate their legitimacy and gain the trust of the community, residents of the community begin to experience “legal socialization,” or the process by which persons develop an appreciation for societal rules, the institutions that create those rules, and the individuals responsible for enforcing them (Trinkner and Cohn 2014; Hogan and Mills 1976; Tapp and Levine 1974). Two key processes occur during legal socialization: people internalize social norms, including those that control behavior and constitute the legal system; and, people develop positive feelings and attitudes toward legal authority (Trinkner and Cohn 2014; Tapp 1976, 1991; Tapp and Levine 1974).

When citizens have positive experiences with legal authorities, they are more likely to see police officers and law enforcement itself as legitimate. Perceived legitimacy is important as it shapes people’s willingness to obey police and comply with legal authority (Tyler and Fagan 2008), aid in crime detection, and support public safety (Schulhofer, Tyler, and Hug 2011). A community’s trust in the government responds to social, political and sociodemographic forces. Trust in government tends to fall when people experience unemployment and financial stress, perhaps signaling government’s failure to provide sufficient opportunity (Weinschenk and Helpap 2015; Wilkes 2015). Business cycles including prosperity, depression and recession also appear to affect citizen trust in government (Stevenson and Wolfers, 2011).

Trust in government and the legitimacy of government are the moral bases of political power and the reasons citizens are ever inclined to abide by the decisions of others (Birch 1993). Citizens create political legitimacy when they consent to comply with the officials, structures, and processes of government. When governmental officials extend their authority through procedures and practices perceived as fair by citizens, they are more likely to be considered as legitimate and deserving of respect (Levi, Sacks, and Tyler 2009; Tyler 2001). When community members have input in governance, it promotes the responsiveness of government and legitimizes local representative democracy (Sonpar, Pazzaglia and Kornijenko 2010). Legitimacy is also dynamic, requiring consistent engagement and renegotiation based on the changing needs, demands or conditions of the governed (Suchman 1995). Legitimacy is inherently unstable, especially when institutional structures and social norms are volatile (Beetham 1991).

Government Competence and Effectiveness

An organization’s legitimacy depends on the perceptions of those it serves. Institutions such as schools, churches and other community organizations help to sustain community norms and to provide social settings where residents interact with one another (McQuarrie and Marwell 2009). The level of support and citizen interaction enjoyed by an organization varies based on its access to organizational and community resources, the manner in which it uses and targets organizational power, and the larger social structure of the community (Vermeulen, Laméris, and Minkoff 2016).

Government organizations earn the trust of the community not only when they support the community, but also when they engage citizens effectively and align their goals with those of the community (Parsons 1960; Ruef and Scott 1998). Effective government organizations integrate the beliefs and knowledge of those they serve in their decision-making (Meyer and Rowan 1977; DiMaggio and Powell 1983; Eesley and Lenox 2006). When an organization or agency conducts itself in a socially appropriate manner and aligns itself with a community’s values, beliefs, norms, and rules, the community is more likely to believe it is acting legitimately (Sonpar, Pazzaglia, and Kornijenko 2010; Dowling and Pfeffer 1975; Suchman 1995). Transparency is always critical. Organizations must openly communicate with the public and demonstrate sound decision-making to be perceived as legitimate and trustworthy (Raphael and Karpowitz 2013; Fung 2003). In such conditions, the public is more likely to engage with and collaborate with the efforts of government.

Defensible Space

Comprehensive crime policy includes interventions to address the built environment and its effect on behavior, known as Crime Prevention through Environmental Design, or CPTED, a crime deterrence strategy that promotes defensible space, public safety, and quality of life by improving the physical features and social consequences of shared space (Jeffery 1971). Communities create defensible space when the physical characteristics of an environment help residents to feel a sense of ownership and to participate in collective guardianship.

Projects and research in the CPTED field fall into two categories, often called first generation and second generation CPTED. First generation CPTED focuses almost exclusively on the physical attributes of communities and consists of the following principles: territoriality (encouraging ownership of space), natural surveillance (increased visibility), activity support (promotion of public activities), and access control (limiting means of entry for a space) (Cozens and Love 2015). Second generation CPTED focuses on the social aspects of a community, such as cohesion (the solidification of relationships between stakeholders in the community), connectivity (mechanisms that connect and encourage communication), community culture (events that encourage a sense of community, and neighborhood capacity (presence of social stabilizers and balanced land use) (Saville and Cleveland 1998; Cozens and Love 2015). In a CPTED effort, residents should be included in the identification of neighborhood problems and in the selection of strategies to solve those problems.

Researchers report that CPTED interventions focused on the physical characteristics of buildings can help prevent property crime, including efforts to improve lighting, locks, surveillance, garbage removal, and street maintenance (Ekblom 2011; Gill and Turbin 1999; Poyner 1993; Ramsay 1991). Some studies, however, find less support for the key hypotheses of CPTED. Crime reduction in one neighborhood may be offset by displacement of crime to other neighborhoods (Cozens and Love 2015), and crime could be lessened at one time of day while increasing at a different time (Johnson, Guerette, and Bowers 2014). Researchers also acknowledge that CPTED projects are inherently difficult to evaluate because of the many methodological issues associated with place-based crime interventions.

CONCLUSION

The evaluation of MAP began with an effort to assess its implementation in NYCHA developments. Researchers at JohnJayREC are documenting various meetings and trainings with agency partners, stakeholder groups, and resident leaders of NYCHA communities, as well as conducting face-to-face interviews with dozens of key actors from the public and private agencies involved in MAP. The implementation assessment will help the study to create a set of variables representing the extent to which the launch and management of MAP was consistent with the City’s intentions and whether that consistency varied among MAP sites. Researchers will use the information to separate MAP communities into varying levels of fidelity and then examine the relationship between adherence to MAP and various outcomes among the NYCHA developments involved in MAP.

In addition to assembling data about the activities, services, and supports offered to NYHCA residents as part of MAP, the evaluation team is relying on law enforcement data, victimization statistics, and other indicators of social and economic well-being to measure social conditions in New York City neighborhoods. Whenever possible, this information is geocoded to locate events in very small areas (actual X/Y coordinates if available). The information is used to estimate social conditions in all MAP areas before and after MAP and to compare them with statistically-matched, non-MAP areas.

Finally, researchers are working with NORC to design and implement surveys of representative samples of NYCHA residents in MAP developments and non-MAP comparison sites. Two waves of the survey will be administered in 2019 and 2020 to measure the experiences, perceptions, and attitudes of residents and to model other critical outcomes that would be unknown if the evaluation focused only on available administrative data about agency efforts and crime incidents. All outcomes from administrative sources and resident surveys will be tested for their relationship to MAP components and any significant differences between MAP sites and non-MAP sites will serve as one basis for estimating the effects of MAP.

Evaluating MAP is a complex endeavor, but the study deploys varying approaches to increase statistical power and examine effects for a wide array of outcomes, including analyzing individual survey responses and multiple observations across time for some measures. The evaluation’s sample size is a key constraint, but may be less concerning when the evaluation investigates whether particular MAP activities lead to particular outcomes, when it tests fidelity to the MAP mission, or in analyses of MAP’s effects on resident attitudes and beliefs. The primary question explored by the evaluation is MAP’s effect on public safety. Acknowledging these methodological challenges, the evaluation employs a multi-modal approach that should yield a rich portrait of the MAP process and its effects in ways that offset the statistical constraints on the study’s test of its central hypothesis—i.e. the presence of MAP in NYCHA developments will be associated with greater reductions in crime and fear of crime than those observed in other NYCHA developments not involved in the MAP initiative.

REFERENCES

Beetham, David (1991). The Legitimacy of Power. London, England: Macmillan Publishers Limited.

Birch, Anthony H. (1993). The Concepts and Theories of Modern Democracy. London, England: Routledge.

Box, Steven, Chris Hale, and Glen Andrews (1988). Explaining Fear of Crime. The British Journal of Criminology, 28(3): 340–356.

Branas, Charles C., Rose A. Cheney, John M. MacDonald, Vicky W. Tam, Tara D. Jackson, and Thomas R. Ten Have (2011). A Difference-in-Differences Analysis of Health, Safety, and Greening Vacant Urban Space. American Journal of Epidemiology, 174(11): 1296-1306.

Brantingham, Patricia L., and Paul J. Brantingham. (1993). Environment, Routine, and Situation: Toward a Pattern Theory of Crime. In Clarke, Ronald V. and Marcus Felson (Editors), Routine Activity and Rational Choice. Advances in Criminological Theory, 5: 259-294. New Brunswick, NJ: Transaction Books.

Brantingham, Patricia L., and Paul J. Brantingham. (1999). Theoretical Model of Crime Hot Spot Generation. Studies on Crime and Crime Prevention, 8: 7–26.

Browning, Christopher R., Seth L. Feinberg, and Robert D. Dietz (2004). The Paradox of Social Organization: Networks, Collective Efficacy, and Violent Crime in Urban Neighborhoods. Social Forces, 83(2): 503–534.

Collins, Charles R., Zachary P. Neal, and Jennifer Watling Neal (2016). Transforming Social Cohesion into Informal Social Control: Deconstructing Collective Efficacy and the Moderating Role of Neighborhood Racial Homogeneity. Journal of Urban Affairs, 39(3): 307–322.

Coryn, Chris L. S., Lindsay A. Noakes, Carl D. Westine and Daniela C. Schröter (2010). A Systematic Review of Theory-Driven Evaluation Practice From 1990 to 2009. American Journal of Evaluation, 32(2): 199-206.

Covington, Jeanette and Ralph B. Taylor (1991). Fear of Crime in Urban Residential Neighborhoods: Implications of Between -and within- Neighborhood Sources for Current Models. The Sociological Quarterly, 32(2): 231-249.

Cozens, Paul and Terence Love (2015). A Review and Current Status of Crime Prevention through Environmental Design (CPTED). Journal of Planning Literature, 30(4): 393-412.

Delgado, Sheyla, Wogod Alawlaqi, Richard Espinobarros, Laila Alsabahi, Anjelica Camacho, and Jeffrey A. Butts (2018). Quasi-Experimental Comparison Design for Evaluating the Mayor’s Action Plan for Neighborhood Safety. Evaluation Update No. 1. New York, NY: Research and Evaluation Center, John Jay College of Criminal Justice.

DiMaggio, Paul J., and Walter W. Powell (1983). The Iron Cage Revisited: Institutional Isomorphism and Collective Rationality in Organizational Fields. American Sociological Review, 48(2): 147–160.

Dowling, John and Jeffrey Pfeffer (1975). Organizational Legitimacy: Social Values and Organizational Behavior. The Pacific Sociological Review, 18(1): 122–136.

Eesley, Charles and Michael J. Lenox (2006). Firm Responses to Secondary Stakeholder Action. Strategic Management Journal, 27(8): 765–781.

Ekblom, Paul (2011). Deconstructing CPTED… and Reconstructing it for Practice, Knowledge Management and Research. European Journal on Criminal Policy and Research, 17(1): 7-28.

Farrall, Stephen D., Jonathan Jackson, and Emily Gray (2009). Social Order and the Fear of Crime in Contemporary Times. New York, NY: Oxford University Press.

Foster, Sarah, Matthew Knuiman, Lisa Wood, and Billie Giles-Corti (2013). Suburban Neighbourhood Design: Associations with Fear of Crime versus Perceived Crime Risk. Journal of Environmental Psychology, 36: 112–17.

Fung, Archon (2003). Survey Article: Recipes for Public Spheres: Eight Institutional Design Choices and Their Consequences. Journal of Political Philosophy 11(3): 338–367.

Gibson, Chris L., Jihong Zhao, Nicholas P. Lovrich, and Michael J. Gaffney (2002). Social Integration, Individual Perceptions of Collective Efficacy, and Fear of Crime in Three Cities. Justice Quarterly, 19(3): 537–564.

Gill, Martin and Vicky Turbin (1999). Evaluating a ‘Realistic Evaluation’: Evidence from a Study of CCTV. In Painter, Kate and Nick Tilley (Editors), Surveillance of Public Space: CCTV, Street Lighting and Crime Prevention. Crime Prevention Studies 10: 179-199. Monsey, NY: Criminal Justice Press.

Goodman-Delahunty, Jane (2010). Four Ingredients: New Recipes for Procedural Justice in Australian Policing. Policing: A Journal of Policy and Practice 4(4): 403-410.

Groff, Elizabeth, David Weisburd, and Nancy A. Morris (2009). Where the Action is at Places: Examining Spatio-Temporal Patterns of Juvenile Crime at Places Using Trajectory Analysis and GIS. In Weisburd, David, Wim Bernasco, and Gerben J.N. Bruinsma (Editors), Putting Crime in its Place: Units of Analysis in Geographic Criminology, 61-86. New York, NY: Springer.

Hipp, John R. (2016). Collective Efficacy: How Is It Conceptualized, How Is It Measured, and Does It Really Matter for Understanding Perceived Neighborhood Crime and Disorder? Journal of Criminal Justice 46: 32–44.

Hogan, Robert and Carol Mills (1976). Legal Socialization. Human Development (19): 261–276.

Jeffery, C. Ray (1971). Crime Prevention through Environmental Design. Bevery Hills, CA: Sage Publications.

Johnson, Shane D., Rob T. Guerette, and Kate Bowers (2014). Crime Displacement: What We Know, What We Don’t Know, and What it Means for Crime Reduction. Journal of Experimental Criminology 10(4): 549-571.

Levi, Margaret, Audrey Sacks, and Tom Tyler (2009). Conceptuaizing Legitimacy, Measuring Legitimizing Beliefs. American Behavioral Scientist 53(3): 354–375.

Lorenc, Theo, Stephan Clayton, David Neary, Margaret Whitehead, Mark Petticrew, Hilary Thomson, Steven Cummins, Amanda Sowden, and Adrian Renton. (2012). Crime, Fear of Crime, Environment, and Mental Health and Wellbeing: Mapping Review of Theories and Causal Pathways. Health and Place 18(4): 757–765.

Louin-Tapp, June (1976). Psychology and the Law: An Overture. Annual Review of Psychology (27): 359–404.

Louin-Tapp, June (1991). The Geography of Legal Socialization: Scientific and Social Markers. Droit et Société (19): 329–353.

Louin Tapp, June, and Felice J. Levine (1974). Legal Socialization: Strategies for an Ethical Legality. Stanford Law Review 27(1): 1–72.

Markowitz, Fred E., Paul E. Bellair, Allen E. Liska, and Jianhong Liu (2001). Extending Social Disorganization Theory: Modeling the Relationships Between Cohesion, Disorder, and Fear. Criminology 39(2): 293–319.

Mastrofski, Stephen B., Jeffrey Snipes, and Anne E. Supina (1996). Compliance on Demand: The Public’s Response to Specific Police Requests. Journal of Research in Crime and Delinquency 33(3): 269-305.

Mayor’s Office of Criminal Justice (2018). Program Summary of the Mayor’s Action Plan for Neighborhood Safety. New York, NY: City of New York.

McQuarrie, Michael and Nicole P. Marwell (2009). The Missing Organizational Dimension in Urban Sociology. City and Community 8(3): 247-268.

Meyer, John W. and Brian Rowan (1977). Institutionalized Organizations: Formal Structure as Myth and Ceremony. American Journal of Sociology (83): 340–363.

Morenoff, Jeffrey D., Robert J. Sampson, and Stephen W. Raudenbush (2001). Neighborhood Inequality, Collective Efficacy, and the Spatial Dynamics of Urban Violence. Criminology 39(3): 517–558.

Pain, Rachel (2000). Place, Social Relations and the Fear of Crime: A Review. Progress in Human Geography, 24(3): 365–387.

Parsons, Talcott (1960). Structure and Process in Modern Societies. Social Forces 39(1): 84-85.

Poyner, Barry (1993). What Works in Crime Prevention: An Overview of Evaluations. In Clarke, Ronald V. (Editor), Crime Prevention Studies, 1. Monsey, NY: Criminal Justice Press.

Ramsay, Malcolm (1991). The Effect of Better Street Lighting On Crime and Fear: A Review. Crime Prevention Unit Paper Number 29. London: Home Office.

Raphael, Chad and Christopher F. Karpowitz (2013). Good Publicity: The Legitimacy of Public Communication of Deliberation. Political Communication 30(1): 17–41.

Ruef, Martin and W. Richard Scott (1998). A Multidimensional Model of Organizational Legitimacy: Hospital Survival in Changing Institutional Environments. Administrative Science Quarterly 43(4): 877–904.

Sampson, Robert J., Stephen W. Raudenbush, and Felton Earls (1997). Neighborhoods and Violent Crime: A Multilevel Study of Collective Efficacy. Science 277(5328): 918–24.

Saville, Greg and Gerry Cleveland (1998). 2nd Generation CPTED: An Antidote to the Social Y2K Virus of Urban Design. 2nd Annual International CPTED Conference. Orlando, FL.

Schulhofer, Stephen J., Tom R. Tyler, and Aziz Z. Huq (2011). American Policing at a Crossroads: Unsustainable Policies and the Procedural Justice Alternative. Journal of Criminal Law and Criminology 101(2): 335–339.

Sherman, L. W., P. Gartin, and M.E. Buerger (1989). Hot Spots of Predatory Crime: Routine Activities and the Criminology of Place. Criminology 27(1): 27–55.

Skogan, Wesley (1986). Fear of Crime and Neighborhood Change. Crime and Justice 8: 203–29.

Sonpar, Karan, Federica Pazzaglia, and Jurgita Kornijenko (2010). The Paradox and Contraints of Legitimacy. Journal of Business Ethics 95(1): 1–21.

Stafford, Mai, Tarani Chandola, and Michael Marmot (2007). Association between Fear of Crime and Mental Health and Physical Functioning. American Journal of Public Health 97(11): 2076–2081.

Stevenson, Betsey, and Justin Wolfers (2011). Trust in Public Insititutions over the Business Cycle. The American Economic Review: Papers and Proceedings 101(3): 281-287.

Stroebe, Wolfgang, N. Pontus Leander, and Arie W. Kruglanski (2017). Is It a Dangerous World Out There? The Motivational Bases of American Gun Ownership. Personality and Social Psychology Bulletin 43(8): 1071–85.

Suchman, Marc C. (1995). Managing Legitimacy: Strategic and Institutional Approaches. Academy of Management Review 20(3): 571–610.

Sunshine, Jason, and Tom R. Tyler (2003). The Role of Procedural Justice and Legitimacy in Shaping Public Support for Policing. Law and Society Review 37(3): 513-548.

Trinkner, Rick and Ellen S. Cohn (2014). Putting the “Social” Back in Legal Socialization: Procedural Justice, Legitimacy, and Cynicism in Legal and Nonlegal Authorities. Law and Human Behavior 38(6): 602–17.

Tyler, Tom R. (2001). Public Trust and Confidence in Legal Authorities: What do Majority and Minority Group Members Want from the Law and Legal Authorities? Behavioral Sciences & The Law 19(2): 215-235.

Tyler, Tom R. (2006). Why People Obey the Law. New Haven, CT: Yale University Press.

Tyler, Tom R. and Jeffrey A. Fagan (2010). Legitimacy and Cooperation: Why do People Help the Police Fight Crime in Their Communities? In Rice, Stephen K. and Michael D. White (Editors), Race, Ethnicity, and Policing: New and Essential Readings (pp. 84-117). New York, NY: NYU Press.

Tyler, Tom R., and Yuen J. Huo (2002). Trust in the Law: Encouraging Public Cooperation with the Police and Courts. New York, NY: Russell Sage Foundation.

Vermeulen, Floris, Joran Laméris, and Debra Minkoff (2016). Welcome to the Neighbourhood: The Spatial Dimensions of Legitimacy for Voluntary Leisure Organisations. Urban Studies 53(11): 2253–2272.

Weinschenk, Aaron C. and David J. Helpap (2015). Political Trust in the American States. State and Local Government Review 47(1): 26–34.

Weisburd, David, Shawn Bushway, Cynthia Lum, and Sue-Ming Yang (2004). Trajectories of Crime at Places: A Longitudinal Study of Street Segments in the City of Seattle. Criminology, 42(2): 283–321.

Wilkes, Ron (2015). We Trust in Government, Just Not in Yours: Race, Partisanship, and Political Trust, 1958–2012. Social Science Research 49: 356-371.

Wilson, William Julius (2012). The Truly Disadvantaged: The Inner City, the Underclass, and Public Policy. Second Edition. Chicago, IL: University of Chicago Press.

________________

Notes

Funding for this report was provided by the New York City Mayor’s Office of Criminal Justice (MOCJ). Points of view or opinions contained within this document are those of the authors and do not necessarily represent the official position or policies of the City University of New York, John Jay College, or their funding partners.

The authors are grateful to the staff and leadership of the Mayor’s Office of Criminal Justice for their guidance and support during the development of the project. The authors are also grateful for the assistance received from their colleagues at the Research and Evaluation Center who contributed to the report: Nicole Alexander, Rebecca Balletto, Justice Banks, Patricia Cobar, Sheyla Delgado, Richard Espinobarros, Shun Feng, Arlana Henry, Sebastian Hoyos-Torres, Gina Moreno, Rhoda Ramdeen, Jason Szkola, Kathleen Tomberg, and Kevin Wolff.

Report design by Nicole Alexander.

Recommended Citation

Butts, Jeffrey A., John K. Roman, Angela Silletti, Anthony Vega and Wogod Alawlaqi (2019). Measurement Plan and Analytic Strategies for Evaluating the Mayor’s Action Plan for Neighborhood Safety. MAP Evaluation Update 2. New York, NY: Research and Evaluation Center, John Jay College of Criminal Justice, City University of New York.

Copyright
Research and Evaluation Center at John Jay College of Criminal Justice
City University of New York (CUNY)
524 59th Street, Suite 605BMW
New York, NY 10019
http://www.JohnJayREC.nyc