Sections
Start — Introduction — Research Methods — Development of the Program — Participants and Program Components — Program Effects — Discussion and Conclusion — References
Research Methods
In addition to answering core research questions through quantitative analyses, the team contextualized the study findings with qualitative data collected through interviews with participants, program staff, participants’ family and friends, and Bronx DA staff, as well as intermittent observations of program operations. Multiple data sources provided insight into participants’ experiences with the BOGAP program and various other outcomes.
Qualitative Data
Researchers worked with the Osborne Association to invite eligible program staff to a series of three interview waves in Fall 2023, Fall 2024, and Summer 2025. Staff interviews focused on individuals who worked directly with participants, including credible messengers who delivered the program’s daily curriculum and others involved in the program’s routine operations. Before conducting interviews, the research team provided staff with an overview of the interview plan, including its voluntary nature. Three waves of interviews were conducted by an experienced qualitative researcher, focusing on the program’s implementation and evolution over time, factors influencing participant engagement, and perceptions of the most helpful and most challenging program features.
Researchers conducted in-person interviews with BOGAP participants at the Bronx Osborne office in the Summer and Fall of 2025. Participants were recruited by program staff from the pool of individuals who were active in BOGAP at the time. Respondents were selected from the most advanced participant cohorts to ensure they had sufficient exposure to the program, could reflect on the multiple phases of operations, and could offer in-depth perspectives. Thirteen participants were interviewed over three days. Each interviewee was compensated for their time. A semi-structured interview protocol provided consistency, while allowing sufficient flexibility for follow-up questions to capture each participant’s experience in depth. The interviewer focused on each respondent’s daily involvement in the program, relationships with staff and peers, changes in work-related activities, and personal commitments required to attend. The discussions explored BOGAP’s strengths, challenges, and areas for improvement, as well as each participant’s perceptions of the program’s effect on their job readiness, future planning, and community experience.
Researchers also planned to conduct group interviews or focus groups with individuals closely connected to BOGAP participants, such as family members, partners, and close friends. However, due to a low response rate, the study conducted interviews with only two loved ones of BOGAP participants. Still, the information from those interviews provided helpful context. Conversations focused on their perceptions of BOGAP, its effects on participants and the surrounding community, and their recommendations for improving the program.
The research team conducted additional interviews with several staff members from the Bronx DA’s Office. Researchers gained insight into the program’s origins and early development, including participant recruitment and eligibility, case review processes, and the evolution of program operations. Interviewers explored staff perceptions of the program’s role relative to other District Attorney initiatives. Discussions with staff allowed researchers to ascertain the District Attorney’s plans to sustain the program in light of federal funding cuts following the 2024 U.S. national election.
Researchers attended several court sessions involving young men participating in the BOGAP program. Court hearings allowed program staff and participants to check in with the judges who heard the original firearm cases. Court appearances continued for participants during their time in the program, during the post-program “cooling off” period, and at the time of their graduation from BOGAP. Researchers’ observations of court appearances informed the study’s documentation of participant behavior, staff-participant interactions, overall courtroom dynamics, and the wide array of program practices.
Most interviews with representatives from the Bronx DA’s Office, BOGAP staff, program participants, and their friends and family were audio-recorded with permission and subsequently transcribed using digital transcription software. Transcripts were reviewed and validated against the audio files before being analyzed. Handwritten notes were typed, cleaned, and prepared for analysis. The research team conducted thematic analyses of all qualitative data, organizing responses by topic and applying a structured coding framework to identify recurring patterns and differences across interviews. Codes were grouped into broader analytic categories and refined iteratively to maintain clarity and uniformity. Interview data provided greater context for quantitative findings and helped to surface perceptions and experiences that structured measures are often unable to reflect.
Interview findings are presented thematically in project reports using aggregated, de-identified responses, with illustrative quotations wherever helpful. To protect confidentiality, quotes are never attributed to individuals. Highlighted results are selected to reflect the views and perspectives of respondents rather than to support specific causal assertions or broadly generalizable conclusions. (For more information about the qualitative data methods and analyses, see Appendix A. To review interview instruments, see Appendix B.)
Survey Data
Researchers invited BOGAP participants to complete a series of online surveys, with compensation for each valid response. Surveys were administered at five points during BOGAP, each linked to a particular program phase identified by staff. This enabled the research team to capture participants’ experiences at various stages, including the 90-day cooling-off period following programming. Staff announced the first three surveys during regular program hours. Researchers sent invitations directly to participants for surveys four and five, using contact information collected at the end of the first three surveys. Osborne staff assigned an anonymous identifier to each participant, enabling researchers to confirm enrollment in BOGAP, track individual responses across surveys, and link to program records. Researchers were able to analyze individual response patterns while maintaining confidentiality.
Participant surveys collected basic information, such as housing stability and age, along with questions regarding their attitudes toward health and well-being, violence, conflict resolution, employment readiness, perceptions of neighborhood safety, and social supports within and outside the Osborne Association. Participants responded to items on a five-point scale from “agree strongly” to “disagree strongly.” The final Wave 5 survey included an additional section addressing participant experiences with BOGAP. Survey items were conceptualized as ten constructs or factors.
The study plan originally included survey data from cohorts five through ten (July 2023 to 2025), but changes in the evaluation timeline narrowed the effort to cohorts five through eight (Figure 1). The first four cohorts were already underway when the evaluation began, and the study was unable to follow their program involvement from entry to completion. Cohorts nine and ten were unable to complete all surveys due to the unexpected changes in BOGAP’s funding and program scheduling, necessitating adjustments to the study timeline.
The first step in the survey analysis process was checking for missing and out-of-range values, dropping duplicate and invalid survey responses, and cross-validating demographic responses against program records. Confirmatory factor analyses assessed whether items within the original constructs were statistically coherent and measured the same underlying ideas. Items that did not load suitably on a common factor were dropped from the analysis to ensure each factor measured a single idea.
Reliability was assessed using Cronbach’s Alpha coefficient. Scores for each factor (the average of the individual survey item scores within each factor) indicated average participant beliefs for each construct. These were used in the data analysis to provide descriptive context about the program and as part of a series of bivariate and multivariate analyses measuring four program outcomes. To test whether each outcome changed over time, researchers estimated ordered logistic regression models including an interaction term for wave and cohort, adjusted for age, number of children, education, housing stability, and employment status. (For information about the survey construction, methods, and analyses, see Appendix C and Appendix D. The survey instrument is available in Appendix E.)
Administrative Data
Researchers collected an assortment of administrative data records with formally executed data-sharing agreements (DSAs) that were essential to the evaluation.
Records were obtained from three sources:
- Osborne Association: Data about individuals’ program participation.
- NYS Division of Criminal Justice Services (DCJS): Recidivism (rearrest) data for the full BOGAP study sample, as well as a pool of anonymized records from which researchers statistically matched the best possible comparison group.
- Bronx District Attorney’s Office (Bronx DA): Court processing records for all individuals meeting basic criteria for referral to BOGAP.
Researchers cleaned, standardized, and merged all data from program records, DA case files, DCJS criminal history records, and combined them with participant surveys to create a participant-level dataset linking demographic, program, and criminal justice information. Analyses examined patterns of enrollment, program completion, and service engagement, and explored how participant characteristics related to outcomes, including changes in attitudes and beliefs, and eventual recidivism. Recidivism was assessed by comparing BOGAP participants who completed the program with a statistically matched group of non-participants selected to resemble the treated sample on key characteristics, including age, prior criminal history, and gun possession charges. The recidivism follow-up period varied by individual, depending on when they exited BOGAP or when their case was resolved. The median time period was at least 18 months. The strategy provided a high-level view of who participated in BOGAP, what factors may be associated with successful outcomes, and the extent to which program involvement related to other interactions with the justice system. (For additional information about administrative data methods and preparation, see Appendix F.)
NEXT SECTION: DEVELOPMENT OF THE PROGRAM