by Kwan-Lamar Blount-Hill and Jeffrey A. Butts
August 2015

Acknowledgements
This report was made possible with funds from the New York City Council and the Robert Wood Johnson Foundation of Princeton, NJ. The authors of this report are grateful to Dr. Richard Curtis for his guidance in the design and conduct of RDS surveys, and they are especially grateful to Sheyla Delgado, the project director of the Cure Violence evaluation, for her expertise and management of the RDS study underway at John Jay College and for many of the photographs used in this report.

INTRODUCTION

John Jay College’s Research & Evaluation Center (JohnJayREC) began an evaluation of the Cure Violence model in 2013 with funds from the Robert Wood Johnson Foundation and the City Council of New York. Cure Violence is a violence prevention model that draws upon the concepts of public health (Cure Violence 2015). Developed by Dr. Gary Slutkin and first implemented in Chicago, Illinois, the model was once known as “Chicago CeaseFire.”

The Cure Violence model posits that violence spreads through a community like an infectious epidemic. It begins with a “carrier” of violent tendencies who transfers those tendencies to others (e.g., through violent encounters) which results in even more carriers. As violence spreads, it affects people directly (i.e. physical injuries) and indirectly (i.e. anxiety, trauma, medical costs, and recurring conflict).

The Cure Violence model is designed to interrupt this cycle. First, Cure Violence programs employ staff members who are “credible messengers” — usually individuals who formerly engaged in violence and who still enjoy credibility among neighborhood youth who are at risk for violence. These credible messengers are used to recruit program participants—young people who may currently harbor violence-endorsing norms or values (i.e., the carriers). Staff members work with participants to reduce violence: 1) they encourage participants to adopt new norms that reject violence (treatment); 2) they intervene in active conflicts to reduce violent attacks and the need for retaliation (containment); and, 3) they promote messages of nonviolence and prevention throughout the community (cure).

The Cure Violence theory of change implies that violence is reduced in several steps. Direct intervention leads to changed behavior among carriers of violence, which leads to changes in their attitudes and values, which spread among their social networks. Eventually, broad-scale changes lead to improved community norms and reductions in gun violence.

SAMPLING STRATEGY

Previous evaluations have focused on the first and last steps of the Cure Violence model’s hypothesized sequence of cause and effect (Butts, Bostwick and Porter 2014). Researchers typically collect data about the activities of Cure Violence programs and then skip to the end of the sequence by measuring change in community-level indicators of crime and violence, such as shootings and homicides. The John Jay College evaluation includes a strategy for estimating a critical intermediate stage in the Cure Violence theory of change. The study measures changes in violence-related attitudes and values of young men (age 18-30) in at-risk neighborhoods and compares areas with and without Cure Violence programs.

This requires the study to conduct surveys among a population of hard-to-reach and hard-to-recruit research subjects, which is when “respondent-driven sampling” is most useful (Heckathorn 1997; Salganik and Heckathorn 2004). Respondent-driven sampling (RDS) allows researchers to collect primary data from traditionally hard-to-reach populations. Researchers have used RDS in studies of intravenous drug users, sex workers, immigrants, and men who secretly have sex with men (Rhodes and McCoy 2015; Rotondi 2013; Arfken et al. 2013; Lausevic et al. 2015).

The RDS approach begins when a researcher recruits an initial respondent (the seed) and then encourages that respondent to utilize his or her own social networks to recruit additional respondents, who in turn recruit even more respondents from among their social networks, etc. This recruitment process is repeated until sufficient numbers of respondents (180-200 per neighborhood in the John Jay study) have answered the survey and referred their friends and acquaintances.

RDS is superior to other methods for recruiting hard-to-reach populations because of the restrictions it imposes that increase the representativeness of the sample. In an RDS design, respondents are allowed to recruit only a limited number of additional respondents, usually three (Rhodes and McCoy 2015; Lausevic et al. 2015; Arfken et al. 2013). This prevents the sample from being saturated with any single individual’s social network.

The use of RDS also strengthens the representativeness of the final sample. Ideally, each neighborhood sample is derived from a single seed. In some RDS studies, however, recruitment streams have stalled, requiring additional seeds. RDS samples derived from more than one seed are thought to be slightly less representative of the larger population (Lausevic et al. 2015).

JohnJayREC elected to use RDS in the Cure Violence study because the method has several distinct advantages over more traditional sampling techniques: 1) it is useful for obtaining samples of difficult-to-recruit populations; 2) it is highly cost-effective while ensuring a sufficient degree of representativeness; 3) it largely removes researcher bias from the recruitment process; and 4) it allows analysts to conduct network analysis when traditional sampling techniques may not.

The JohnJayREC evaluation involves surveys of young men in communities throughout New York City. To participate in the survey, respondents must be male, eighteen to thirty years of age, and living within a defined area, either a Cure Violence intervention neighborhood zone or a similar area without Cure Violence. The surveys measure each respondent’s attitudes towards violence, his personal experience with violence and public safety, and his attitudes toward community institutions and neighborhood supports. The items in each survey are designed to estimate changes over time and to test the influence of Cure Violence programs on participant attitudes and community norms. Once a participant completes a survey, he is invited to recruit a maximum of three new respondents.

The John Jay evaluation is designed to reach young men from relatively small areas within specific neighborhoods. Sampling in small areas raises particular concerns for field research. First, one cannot easily tell whether a potential respondent lives in the designated area, as residence is not an observable characteristic. Second, the sampled neighborhoods are small parts of a very large urban area. Many people present in the area during survey administrations are likely to be from other neighborhoods. Third, the sampled population may not carry a valid up-to-date form of identification in order to prove their current residence. Identifying subjects that meet the sample requirements is achieved more efficiently by relying on respondents themselves to recruit their peers.

Without using RDS, the John Jay study would have likely found it difficult to locate respondents of the appropriate residency, to gather them in a central location suitable for survey administration, to secure their participation quickly and to encourage their patience whenever survey administration was delayed. When respondents are recruited by friends and acquaintances, there is often less social distance between the study team and the respondents. Peer recruitment increases the credibility of the study. The positive effect of credibility is a key benefit of the RDS method (Truong et al. 2013).

DATA-COLLECTION PROCESS

Data collection typically requires eight to ten working days. The consistency of recruitment is critical in maintaining the referral chain for each site. Survey sessions begin around 3:00 p.m. and usually conclude by 7:00 p.m. Seeing the research team at the same time and location throughout the data collection period is important as a means of encouraging repondents to recruit from their social networks. Prior to the start of data-collection, the research team scouts each area to find an appropriate place to manage the administration of interviews. Locations are chosen based upon considerations of safety for the research team and survey respondents as well as the opportunity for private conversations between respondents and survey staff.

The first day of data-collection (the “seed day”) begins with the recruitment of one person—i.e. the seed participant. The seed participant is someone who fits the sample criteria, is willing to participate in the survey, and is willing to assist the study team in recruiting three other subjects who fit the criteria and who reside in the defined neighborhood area. Each subject is asked to recruit another three subjects.

The average time required to participate in the study is 10 to 20 minutes. The process starts with a research supervisor screening each subject to ensure that he meets the sample criteria (i.e. males between the ages of 18 and 30 years who live in the selected area). Eligible residency is determined using maps and a list of addresses within the defined area. After screening, each subject is introduced to a survey team member who explains the study in full and obtains verbal consent before initiating the administration of the survey.

At the conclusion of the survey, each respondent is paid $30 cash and given three numbered coupons. The respondent is encouraged to refer up to three friends who fit the study’s criteria in exchange for an additional $10 incentive payment for each new recruit who successfully completes the survey. Using this strategy, the JohnJayREC survey team is able to generate a sample of 180-200 subjects relatively quickly.

REPRESENTATIVENESS

Representativeness is the degree to which a sample mirrors the population being studied with regard to demographics and other characteristics. Put another way, it is whether the sample “looks like” the studied population (Bachman and Schutt 2011). RDS does not guarantee a representative sample and can fail to reach a representative sample for a number of reasons. Respondent-recruiters may have social networks made up only of people just like themselves (homophily) (Truong et al. 2013; Wylie and Jolly 2013). Certain types of participants may not have large social networks and, therefore, have a reduced probability of being recruited into the sample. Certain groups may also be intentionally excluded by respondent-recruiters (Lausevic et al. 2015; McCreesh et al. 2013). Finally, certain groups may be less apt to respond to recruitment entreaties (McCreesh et al. 2013).

The evidence regarding representativeness in RDS studies is encouraging but less than conclusive (Wylie and Jolly 2013). Previous studies demonstrate that it may be possible to obtain representative samples using RDS. Arfken and colleagues (2013), for example, cite a study in which RDS was used to obtain a sample of a university population for which detailed demographic data was already available. The study found that the method was able to generate a sample representative of that population. In a similar study, researchers successfully used RDS to draw a sample from a Ugandan population for which aggregate statistics were already available (McCreesh et al. 2013). The RDS sample in that study was representative in “most respects.”

One way to enhance representativeness is to explain the eligibility criteria in detail to all respondents and potential recruiters (McCreesh et al. 2013). In this way, respondents are better equipped to appreciate the full range of eligible individuals in their social networks and to maximize their recruitment success rate by appealing to as many of their contacts as possible. In the John Jay study, the eligibility criteria were kept simple (age and area of residence) and were explained verbally and in writing to survey respondents.

SELECTION BIAS

One persistent concern for field-based research studies is the threat of selection bias, where researchers—knowingly or unknowingly—select certain individuals for participation in the study and exclude others. RDS mitigates but does not completely eliminate the threat of bias. The initial selection of the seed creates a point where researchers might insert their biases into the sample. However, the sample is self-generating after that point. Thus, having identified the one juncture where researchers can introduce bias, RDS allows efforts to be concentrated at eliminating researcher bias at a very limited point in the study, as opposed to having to police against researcher bias throughout.

In the JohnJayREC study, the initial seed is recruited on the first day of data collection in each neighborhood based purely on chance. As soon as the team arrives and sets up the survey location (typically, a rental car with identifying signs parked next to a pedestrian area), someone from the team approaches every potential seed respondent as they walk by the car. The seed respondent is the first person who meets the selection criteria, agrees to participate in the study, and agrees to recruit additional respondents. This technique reduces researcher selectivity at the only point (seed selection) where it can affect RDS representativeness.

One additional source of bias was the time of day at which each survey site was launched and when the seed respondent was likely chosen: 3 p.m. This time of day was chosen specifically to increase the chances of the seed respondent being unemployed (a risk factor for violence).

The use of respondents as recruiters, of course, could introduce other biases (Wylie and Jolly 2013; McCreesh et al. 2013), but respondent-recruiter bias is likely preferable to researcher bias. The research team has more incentive (if only subconsciously) to choose respondents in a way that could affect the study results. For example, if a researcher wanted to increase the chances of finding a positive effect for Cure Violence, he or she would try to recruit more prosocial respondents at later administrations of the survey. The recruitment efforts of respondents, on the other hand, are only motivated by the study incentive (i.e. cash), and the effects of the incentive are not likely to vary between the first and last survey wave.

If detected, evidence of respondent bias may reveal something useful to researchers. For example, in their study of Ugandan heads of households, McCreesh and colleagues (2013) found that young men and the unmarried tended to be excluded by respondent-recruiters. This revelation brought to light a difference in the researcher’s and subject’s conception of “head of household.” The researchers learned that Ugandan culture was not inclined to regard young, unmarried males as household heads even if they provided for themselves and lived alone. The researchers also found that individuals of a higher socioeconomic status were underrepresented in the sample. This finding had implications for the theoretical underpinnings of RDS, as they hypothesized that the usual RDS incentive may not have the same appeal to those of a higher social class. Neither of these findings would have been notable had they been the result of researcher bias.

NETWORK ANALYSIS

An important benefit of RDS as a sampling technique is that it permits researchers to conduct a limited form of social network analysis when examining the resulting data set. Researchers can examine the nature of relationships between respondents and their referral sources and perhaps discover patterns and distinct characteristics within respondent networks.

This may be especially valuable when studying social phenomena that take place within the context of relationships. For example, Arfken and colleagues (2013) were able to determine that alcohol use was related to the types of social networks inhabited by Muslim students. This finding would have been less apparent in more randomized sampling. Similarly, Truong and colleagues (2013) used RDS methods to demonstrate the fleeting nature of relationships between men who have sex with men during international travel—a topic that would be difficult to study using traditional sampling methods.

The Cure Violence model is based on the presumption that violence spreads between individuals through social connections. More importantly, it presumes that norms are disseminated through social networks and are transferred from individuals, to groups, to entire communities. The “infectious” nature of social and behavioral norms presupposes the idea that people are connected and that those connections are influential.

By studying the social networks of young adult males living in high-risk communities, JohnJayREC’s evaluation of Cure Violence may yield important data about the operations of the program and the validity of its theory of change. Through these findings, questions may be answered about how violent individuals cluster within segregated social networks and whether a great degree of familiarity is important for the transfer of norms. Such inquiries are only possible when social networks are identified by researchers.

CONCLUSION

Respondent-driven sampling is both appropriate and useful for JohnJayREC’s evaluation of Cure Violence. It provides a method of sampling a population that—while perhaps not hidden—is difficult to recruit for research studies. RDS allows the research team to study Cure Violence in a cost-effective manner while maintaining representativeness in the sample and reducing the negative impacts of researcher bias. Finally, RDS provides an opportunity for the analysis of social networks, which may be crucial for fully understanding the social mechanisms at work in the Cure Violence model.

_______________________

Copyright by the Research & Evaluation Center, John Jay College of Criminal Justice, City University of New York (CUNY).

524 59th Street
New York, NY 10019
www.johnjayrec.nyc
Published online August 2015

Any views expressed here are those of the author and do not necessarily reflect the policies of John Jay College, the City University of New York, or its funders.

________________________

REFERENCES

Arfken, Cynthia L., Sameera Ahmed, and Wahiba Abu-Ras (2013). Respondent-driven sampling of Muslim undergraduate U.S. college students and alcohol use: Pilot study. Social Psychiatry & Psychiatric Epidemiology, 48, 945-953.

Bachman, Ronet, and Russell K. Schutt (2011). The Practice of Research in Criminology and Criminal Justice, 4th Ed. Los Angeles, CA: SAGE Publications, Inc.

Butts, Jeffrey with Lindsay Bostwick and Jeremy Porter (2014). Denormalizing Violence: Evaluation Framework for a Public Health Model of Violence Prevention. New York, NY: Research & Evaluation Center, John Jay College of Criminal Justice, City University of New York.

Fox, Andrew M., Charles M. Katz, David E. Choate and E. C. Hedberg (2014). Evaluation of the Phoenix TRUCE Project: A replication of Chicago CeaseFire. Justice Quarterly, 32 (1), 85-115.

Heckathorn, Douglas D. (1997). Respondent-driven sampling: A new approach to the study of hidden populations. Social Problems, 44 (2), 174-199.

Lausevic, Dragan, Senad Begic, Boban Mugosa, Natasa Terzic, Zoran Vratnica, Itana Labovic, and Ivana Bozicevic (2015). Prevalence of HIV and other infections and correlates of needle and syringe sharing among people who inject drugs in Podgorica, Montenegro: A respondent-driven sampling survey. Harm Reduction Journal, 12 (2), 1-7.

McCreesh, Nicky, Andrew Copas, Janet Seeley, Lisa G. Johnston, Pam Sonnenberg, Richard J. Hayes, Simon D. W. Frost, and Richard G. White (2013). Respondent driven sampling: Determinants of recruitment and a method to improve point estimation. PLoS ONE, 8 (10), 1-10.

McCreesh, Nicky, Matilda Nadagire Tarsh, Janet Seeley, Joseph Katongole, and Richard G. White (2013). Community understanding of respondent-driven sampling in a medical research setting in Uganda: Importance for the use of RDS for public health research. International Journal of Social Research Methodology, 16 (4), 269-284.

Rhodes, Scott D., and Thomas P. McCoy (2015). Condom use among immigrant Latino sexual minorities: Multilevel analysis after respondent-driven sampling. AIDS Education and Prevention, 27 (1), 27-43.

Salganik, Matthew J. and Douglas D. Heckathorn (2004). Sampling and estimation in hidden populations using respondent-driven sampling. Sociological Methodology, 34, 193-239.

Rotondi, Michael A. (2013). Towards the estimation of effect measures in studies using respondent-driven sampling. Journal of Urban Health: Bulletin of the New York Academy of Medicne, 91 (3), 592-597.

Truong, Hong-Ha M., Michael Grasso, Yea-Hung Chen, Timothy A. Kellogg, Tyler Robertson, Alberto Curotto, Wayne T. Steward, and Willi McFarland (2013). Balancing theory and practice in respondent-driven sampling: A case study of innovations developed to overcome recruitment challenges. PLoS ONE, 8 (8), 1-8.

Wylie, John L., and Ann M. Jolly (2013). Understanding recruitment: Outcomes associated with alternate methods for seed selection in respondent driven sampling. BMC Medical Research Methodology, 13 (93), 1-11.

__________________________________________________
Recommended Citation:
Kwan-Lamar Blount-Hill and Jeffrey A. Butts (2015). Respondent-Driven Sampling: Evaluating the Effects of the Cure Violence Model with Neighborhood Surveys. New York, NY: Research & Evaluation Center, John Jay College of Criminal Justice.