UPDATE 6
by Gina Moreno, Jeffrey A. Butts, and Hans Erickson
INTRODUCTION
As part of an evaluation of the New York City Mayor’s Action Plan for Neighborhood Safety (MAP), researchers from John Jay College of Criminal Justice collaborated with survey specialists from NORC at the University of Chicago to collect data from a probability sample of households in public housing developments in New York City. One group of respondents lived in communities involved in the MAP initiative. A second group was from statistically matched communities not involved in MAP (See Evaluation Update 1). This report describes data from two waves of the survey (2019 and 2020). Researchers measured the perceptions and opinions of residents to identify differences between those living in MAP communities and those from matched comparison communities.
EVALUATING THE MAP INITIATIVE
New York City launched the Mayor’s Action Plan for Neighborhood Safety in 2014, describing it as a targeted and comprehensive approach to public safety in housing developments operated by the New York City Housing Authority (NYCHA). According to the NYC Mayor’s Office of Criminal Justice (MOCJ), housing developments involved in MAP accounted for one-fifth of all violent crimes reported in NYCHA communities during the period preceding MAP. The goal of MAP was to improve public safety in NYCHA communities by supporting the well-being and social capacities of residents, facilitating community empowerment, strengthening connections, and increasing the presence of active public space in and around NYCHA developments.
In 2017, John Jay College’s Research and Evaluation Center (JohnJayREC) began the evaluation of MAP and immediately enlisted the help of researchers from NORC at the University of Chicago. Most of the evaluation relies on administrative data from police and social services, but a partnership with NORC allowed the study to collect survey data directly from residents using rigorous and controlled methods.
Together with MOCJ, researchers from John Jay and NORC designed the survey to measure various perceptions, attitudes, and opinions of people living in public housing. Experts from NORC administered the surveys to a sample of households from more than 30 public housing developments.
Survey items focused on perceptions of community safety, the availability of social supports, and other indicators of community well-being. Drawing on this information, researchers analyzed differences between MAP sites and a matched set of non-MAP sites. Key outcomes included social cohesion (trust in one’s community), belief in government legitimacy, perceptions of safety, and the extent to which residents were willing to engage with government in the interests of the community. Collective efficacy, or neighbors’ capacity to solve problems together, was assessed using two different forms (categorical and dichotomous) to test their comparative utility.
To create a theoretically salient set of questions and scales (i.e., groups of questions), researchers reviewed more than 40 previous studies (Figure 1). Whenever possible, the team preserved the original wording of questions, but it was necessary to adapt some questions for respondents living in dense, urban areas (Figure 2). Previous studies sometimes referred to topics that would be relevant only in smaller cities and suburban areas (e.g., lawn care).
Before beginning data collection, researchers from NORC and JohnJayREC screened the questionnaire with officials from MOCJ and NYCHA. The MAP Resident Survey was then pilot-tested with a small group of residents. Feedback from the pilot group helped to ensure the suitability of language used in the questionnaire and to confirm the accuracy and accessibility of instructions provided for survey respondents.
SAMPLING AND RECRUITMENT
Resident households were sampled from NYCHA housing developments with similar population sizes and demographic characteristics, including the 17 developments involved in MAP (treatment group) and 17 matched developments not involved in MAP (comparison group).(1) To begin the process, NYCHA provided analysts with a de-identified list of more than 80,000 adult residents (ages 18 and older) across all 34 study sites. NORC randomly selected 17,630 of those residents as the initial study sample (see MAP Evaluation Update 4).
1. The MAP initiative is often described as an intervention focused on 15 housing developments, but NYCHA considers three of those developments (Red Hook, Queensbridge, and Van Dyke) as comprising two distinct communities each, and one of those sites is exclusively for older residents (Van Dyke II). It was excluded from the study. Thus, the John Jay College evaluation conceptualizes MAP as an initiative affecting 17 NYCHA communities across the five boroughs of New York City: Bronx (Butler, Castle Hill, Patterson), Brooklyn (Boulevard, Brownsville, Bushwick, Ingersoll, Red Hook East, Red Hook West, Tompkins, Van Dyke), Manhattan (Polo Grounds, Saint Nicholas, Wagner), Queens (Queensbridge South, Queensbridge North), and Staten Island (Stapleton).
Soon thereafter, each sampled participant received an envelope via U.S. Mail with a letter explaining the survey, its purpose, and its sponsorship. Respondents were assured that while NYCHA endorsed the survey, the housing authority was not conducting the survey and would not see the answers of any residents, nor would any resident’s participation or lack of participation in the survey affect their housing status.
Every invitation envelope contained a $2 bill, and the letter described additional incentives for respondents who completed the survey. Respondents could answer the survey by phone or by using a website accessible with a desktop or mobile device after entering their unique log-in credentials. Everyone completing the survey received a $15 prepaid card and a $10 bonus if they completed it within two weeks of receiving the invitation letter. The survey was available in four languages: English, Spanish, Cantonese, and Mandarin. As responses were logged on the survey website, researchers analyzed the incoming data to ensure the integrity of the sample and the quality of responses.
RESPONDENTS
The final respondent pool from both waves of the MAP Resident Survey was just over 3,000, approximately half from MAP communities and half from comparison communities, with few significant differences between the samples of respondents in MAP and non-MAP sites. Only small differences were observed in age, gender, ethnicity, education level, and employment status (Table 1).
Respondents from MAP and non-MAP communities were very similar in age. About two-thirds of all respondents in both groups were between ages 25 and 69. Respondents in 2020 were slightly older than the sample of respondents in 2019, but this was true both for MAP and non-MAP respondents. Female respondents predominated in both survey years, and both MAP and non-MAP respondents were slightly more likely to be female in 2020 than in 2019 (MAP 74% versus 72%; non-MAP 70% versus 67%), which is representative of NYCHA residents overall according to city data.
Some slight differences were observed in self-reported ethnicity, with more Black respondents in the MAP group (47% in 2019 and 49% in 2020) than in the comparison or non-MAP group (32% in 2019 and 35% in 2020). The next most prevalent group in both MAP and non-MAP sites were Latinx residents, comprising more than a third of all respondents in both years, but declining slightly between 2019 and 2020 in accordance with the growing proportions of Black respondents.
About eight in ten respondents reported having earned at least a high school diploma; more than four in ten reported at least some college experience (MAP 44% and non-MAP 45% in 2020). More than a third of all respondents in both years reported being employed either part-time or full-time. Two-thirds of all respondents reported living in their NYCHA development for more than ten years. Few respondents (fewer than 4 percent) were newcomers who lived in NYCHA for less than one year. Among both MAP and non-MAP respondents, the proportion of newcomers grew slightly between 2019 and 2020.
Most respondents reported households of at least three people. This was true for MAP households (54% in 2019, 51% in 2020) as well as non-MAP households (54% in 2019, 52% in 2020). Nearly one in six respondents reported households of five or more, largely due to the presence of children. When household size was reported for adults only (age 18 or older), fewer than one in 25 respondents reported a household of more than four people. The analysis suggests that MAP and non-MAP respondents were very similar, with only slight changes between 2019 and 2020. The results support the comparability of treatment and comparison sites as intended.
RESULTS
Awareness and Engagement
In 2020, researchers asked respondents to indicate their involvement in MAP activities. Before answering, respondents were asked whether they were even aware of MAP. Most respondents were not aware of the initiative, including 78 percent of respondents living in MAP developments and 88 percent of those living in the comparison sites. When asked about the individual components affiliated with MAP, respondents from MAP developments were consistently more likely than those from comparison sites to report participation in each activity. The vast majority of both groups, however, had not participated (Figure 3).
Respondents living in non-MAP sites were always able to access some supports and services affiliated with MAP. Still, researchers wanted to ascertain whether participation was greater in MAP developments. The difference between MAP and non-MAP residents always favored MAP. The difference was statistically significant in eight of the 13 components.
Some differences were smaller than expected. Respondents from MAP sites were more likely to report participating in resident stakeholder meetings (10% vs. 6%) and NStat meetings (12% vs. 7%), but both components were assumed to be exclusive to MAP residents. The fact that some non-MAP residents reported participation could suggest measurement error, or it could reveal that non-MAP residents were invited to attend MAP-related events.
Other differences were expected, including participation in kinship care support groups (5% MAP vs. 3% non-MAP), NYC Summer Youth Employment Program (30% vs. 21%), Police Athletic League’s Playstreets program (8% vs. 5%), Kids in Motion program (5% vs. 3%), access to evening hours at community centers during the summer months (15% vs. 10%), and the use of on-site appointments with outreach specialists from New York City’s social services agency, the Human Resources Administration (11% vs. 7%).
Opinions and Perceptions
MAP was designed to improve the safety and well-being of communities by influencing key mediating variables (MAP Update 2). The evaluation was designed to detect such improvements and, if possible, identify their effects on residents. Survey data were important in this measurement chain, but the study faced difficult asymmetries. The MAP Resident Survey successfully measured key mediating variables, but not until 2019—more than four years after the launch of MAP. Researchers explored associations between mediators and various crime outcomes, but without attribution of causality or the direction of influence.
First, researchers employed factor analysis to identify 10 sets of survey items that were sufficiently correlated with one another to qualify as multi-item scales or factors. Of the 70 potential outcome items measured in the survey, 61 were strong enough to combine into 10 different scales.
Researchers assessed each scale using Cronbach’s alpha coefficient, a statistic used to judge scale reliability. Alpha coefficients range from zero to one. As alpha approaches 1.0, the internal consistency among items is stronger. Values above 0.9 are considered excellent, while those above 0.8 are good. Values above 0.7 are considered acceptable, but values lower than 0.7 are considered not useful. All 10 scales reported in this analysis were above the acceptable threshold (see Figure 1).
Researchers examined scale differences between MAP and non-MAP sites, comparing the 2019 and 2020 survey waves (Table 2). None of the analyses revealed significant differences at the p < .10 level. Statistical significance was assessed using independent samples t-tests and Mann-Whitney U tests. Mann-Whitney tests were added because response distributions were often skewed (clustered at one end of a scale rather than evenly distributed).
Researchers also had to address “missing” or “nonattitude” responses (i.e., “I don’t know” or “prefer not to answer”). In the 2020 survey wave, half the items had missing values above an acceptable threshold of 10 percent. Such responses may reflect a true absence of opinion, confusion about a question, or simply random choices by respondents eager to reach the end of the survey.
Association with Reported Crime Outcomes
Finally, researchers tested the correlation of survey responses with changes in crime before and after the launch of MAP (Table 3). Average responses from both survey waves (i.e., the mean of 2019 and 2020 responses) were tested for their association with changes in crime by comparing the average monthly count of crimes from July 2014 to December 2019 (after MAP) with the average monthly count from January 2010 to June 2014 (before MAP).
Correlations between survey scales and changes in crime outcomes were tested with a nonparametric method when responses were unevenly distributed with values clustering at one end of the scale and items varied in their extent of missing responses. Spearman’s (ρ) coefficient reflects the strength and direction of the correlation between variables.
Consistent with previous findings, the strongest associations were observed in the crime category of misdemeanor offenses against persons. A total of five survey scales were significantly correlated with changes in person misdemeanors, and all correlations were negative—i.e., crime counts tended to fall in developments where NYCHA residents reported more positive perceptions about the very social capacities and community conditions that were the focus of MAP.
CONCLUSION
The analyses in this report are exploratory. The study does not control for all differences between respondent samples, and the two survey waves were administered only one year apart and more than four years after the beginning of MAP. Thus, it is not surprising that little change was observed between the first and second survey. When both waves are viewed in conjunction with changes in crime, however, some results appear to be promising.
At the very least, the findings suggest that MAP focused on appropriate outcomes and mediators. Changes in person-related misdemeanors were significantly correlated with five of the social assets and resident capacities that served as the focus of the MAP initiative. Differences in reported crimes between MAP and non-MAP areas were slightly less distinct in communities where responses to survey scales indicated stronger and healthier conditions. In other words, comparison communities whose residents reported more positive views of the social assets and capacities that were the focus of the MAP initiative tended to have favorable crime changes resembling those seen in MAP communities.
The results suggest that changes in some types of crime might be mediated by gains in community well-being, social cohesion, engagement with government, and citizen trust in the competence of government agencies and actors. As communities become more tightly connected and more supported, they may experience gains in public safety.
Acknowledgments
Funding for this report was provided by the New York City Mayor’s Office of Criminal Justice (MOCJ). All crime data were provided by the New York City Police Department. Points of view or opinions contained within this document are those of the authors and do not necessarily represent the official position or policies of the City University of New York, John Jay College, the Mayor’s Office of Criminal Justice, or the New York Police Department.
The authors are grateful to the staff and leadership of the Mayor’s Office of Criminal Justice for their support during the development of the project and to the members of the MAP Advisory Board for their critical guidance. The authors are also grateful for the assistance received from John Jay colleagues who contributed to the project: Nicole Alexander, Rebecca Balletto, Justice Banks, Patricia Cobar, Sheyla Delgado, Richard Espinobarros, Edda Fransdottir, Rhoda Ramdeen, and Kathy Tomberg.
Recommended Citation
Moreno, Gina, Jeffrey A. Butts, and Hans Erickson (2020). Opinions and Perceptions of Residents in New York City Public Housing: More Findings from Household Surveys in MAP Communities and non-MAP Communities. MAP Evaluation Update 6. New York, NY: Research and Evaluation Center, John Jay College of Criminal Justice, City University of New York.
Copyright
Research and Evaluation Center at John Jay College of Criminal Justice
City University of New York (CUNY)
524 59th Street, Suite 605BMW
New York, NY 10019
www.JohnJayREC.nyc
