The Graduate Research Fellowships of the Evidence Generation initiative are designed to support the evaluation strategies of justice agencies in New York City. Research Fellows work in teams to consult with nonprofit community-based organizations to advance their efforts to generate high-quality evaluation outcomes and to produce evidence of their effectiveness. Research Fellows provide agencies with guidance and training in applied evaluation skills and they apply these skills in building analytic capacities. Graduate Fellows work directly with agency staff, participate in workshops and meetings, and assist in the formulation of each agency’s evaluation strategy. The purpose of the Evidence Generation initiative is not to perform external evaluations of the agencies, but to help them grow the skill set necessary to develop their own evidence base.
Types of Fellowships
Lewis Rudin Fellowships in Applied Justice Research have been part of the Evidence Generation initiative since 2010. Funding is provided by Susan and Jack Rudin of New York City, the Lewis and Rachel Rudin Foundation, Inc. and the May and Samuel Rudin Family Foundation, Inc. Graduate students selected as Lewis Rudin Fellows in Applied Justice Research work with the staff of the Research and Evaluation Center at John Jay College and John Jay faculty advisors to improve the operations and effectiveness of nonprofit community-based agencies in the New York City youth justice system.
Pinkerton Graduate Research Fellowships were a component of the Evidence Generation Initiative from 2012 to 2015. Pinkerton Fellows contributed to the development of the overall initiative by supporting the research capacity of participating nonprofit community-based organizations.
Affiliated Agencies
Justice agencies throughout the New York City area are affiliated with the Evidence Generation initiative. Current affiliates include:
Abraham House, which provides a holistic, family-based program of services to offenders and their families, and other families who struggle to cope with poverty, violence, crime, and lack of essential services.
BronxConnect, a faith-based, community-based program offering alternatives to detention and incarceration that connect court-involved youth (ages 12-19) with positive resources in the local community through mentoring relationships to prevent recidivism and address youth-initiated goals in education and employment.
Brooklyn Defender Services represents clients in a wide variety of cases, including serious felonies and misdemeanor cases as well as Family Court matters. Attorneys at Brooklyn Defender Services focus on the individual client’s needs, addressing the case from the standpoint of how that case will affect the client’s life.
The Center for Community Alternatives (CCA) is a leader in the field of community-based alternatives to incarceration. Its mission is to promote reintegrative justice and a reduced reliance on incarceration through advocacy, services and public policy development in pursuit of civil and human rights.
Children’s Aid Society, the internationally renowned and historic agency that serves children, youth and their families. The Children’s Aid Society coordinates several programs geared towards adolescents’ emotional, educational and physical well-being.
Community Connections for Youth is a New-York based nonprofit organization, located in the Bronx. The mission of CCFY is to mobilize indigenous faith and neighborhood organizations to develop effective community-based alternative-to-incarceration programs for young people.
The DOME Project, which assists young people who are economically, socially, and academically challenged to focus on their education as a means to success. The project helps young people to develop a healthy, positive sense of themselves as individuals who can thrive in society.
exalt supports the success of youth ages 15-19 who’ve been involved in the criminal justice system by inspiring youth at a critical crossroads to believe in their worth, from the first steps in contemplation through the journey to create lasting behavioral change. Through structured classes for tangible skill development, individualized support to navigate the education and justice systems, placement in paid internships, and an alumni network exalt equips youth with the tools and experience to avoid further criminal justice system involvement.
The Fortune Society works to create a world where currently or formerly incarcerated people can become positive, contributing members of society. We do this through community education, advocacy, and a one-stop model of service provision based on over forty years of experience assisting people with criminal records.
Good Shepherd Services, a leading youth development, education and family service agency that serves over 20,000 program participants a year by giving vulnerable youth in New York City the opportunity to take ownership of their future, making a difference today and for the next generation.
Harlem Commonwealth Council’s promotes local economic development and self-sufficiency by investing in the human capital residing in Harlem and by offering programs that give community residents the ability to succeed in their educational and small business pursuits.
Hour Children help incarcerated and formerly incarcerated women and their children to successfully rejoin the community, avoid further justice involvement, reunify families, and build healthy, independent and secure lives.
The Osborne Association offers opportunities for individuals who have been in conflict with the law to transform their lives through innovative, effective, and replicable programs that serve the community by reducing crime and its human and economic costs. We offer opportunities for reform and rehabilitation through public education, advocacy, and alternatives to incarceration that respect the dignity of people and honor their capacity to change.
The Police Athletic League, together with NYPD and the law enforcement community, supports and inspires New York City youth to realize their full individual potential as productive members of society.
Youth Advocate Programs, Inc. engages human service systems so that they rely less on institutional care and invest more in supporting families and neighborhoods. Working with child welfare, juvenile justice, behavioral health, disability and education systems, YAP seeks to capitalize on the strength and resources of families and communities, including identifying and engaging the informal natural helpers that are found in every community to support the highest risk youth and families.
What it Means to be an Affiliated Agency
The Evidence Generation initiative helps agencies to identify the operational or managerial challenges that may limit their ability to participate in rigorous evaluation studies. The Graduate Research Fellows from Evidence Generation work collaboratively with agency staff to build and implement the analytic resources necessary for evaluation efforts. Typically, this includes creating or refining an agency’s theory of change and a conceptual framework or logic model that can be used to specify a program’s key components and their relationship to desired outcomes at the client, family, and community level.
Next, the Fellows focus on the complex challenges each agency faces in developing accurate but flexible data collection methods to support future evaluation efforts. Depending on the program involved, agencies may work to improve their access to administrative information systems, client surveys, or even interviews when those are necessary to collect the client-specific and often subjective data elements required to test the long-term impact of an intervention.
Finally, the Graduate Research Fellows cooperate with agency staff to develop a work plan for implementing whatever research designs each agency chooses to use in evaluating the effectiveness of existing or future intervention programs. Again, the goal of the initiative is not to deliver a finished outcome evaluation. Rather, Fellows help each agency to develop the tools and skills needed to participate in evaluations. As in all human services systems, the most desirable research designs involve randomization and experimental comparison strategies, but Fellows also encourage agencies prepare for some of the reasonable alternatives to randomization, including the wide variety of quasi-experimental evaluation designs.
During an agency’s first year of affiliation with the Evidence Generation initiative, the Graduate Research Fellows help agency staff to create the basic building blocks of effective evaluation designs. These materials may include research summaries, measurement guides, and descriptions of agency programs and objectives. By the end of the first year, each agency should have an individualized set of recommendations and next steps that staff may choose to pursue in subsequent years. For example, they may assist agencies that wish to begin implementing new data collection systems. They may help affiliates to create practical measures for key program activities and outcomes, or they might conduct additional research on programming principles that are consistent with the organization’s theory of change. Materials produced from these projects are designed to be useful as organizations modify program elements to incorporate practices based on stronger evidence. Revised evaluation documents are distributed as they are produced.
or more information, please contact:
Jeffrey A. Butts, Director of the Research and Evaluation Center
jbutts [at] jjay.cuny.edu