Recidivism is an imperfect, often harmful way to judge the effectiveness of community-based justice interventions. It is also unavoidable. Using recidivism data to assess the failure of individuals and the ineffectiveness of services is a crude, but established practice that cannot be dismissed easily. It is possible, however, to diminish the harm caused by simplistic recidivism comparisons. Researchers can measure recidivism outcomes and provide the results to community agencies privately and confidentially, much like annual medical checkups.

An independent research partner is a key ingredient for such an effort. Few community-based service providers can measure client recidivism routinely. They lack the necessary resources (e.g., staff, funding, and analytic expertise). They must be assisted by a research team that reviews recidivism outcomes periodically as a confidential part of an agency’s quality assurance process.

Such a project achieves two key goals. First, it provides service agencies with reliable and valid indicators of recidivism outcomes among their own clients. Second, regular analyses from an array of community agencies could enable justice officials to monitor overall system effectiveness while avoiding the costs and delays involved in the sort of evaluations typically conducted by academics and outside researchers.

In collaboration with the New York State Division of Criminal Justice Services (DCJS), the Research and Evaluation Center at John Jay College of Criminal Justice (JohnJayREC) developed a protocol for conducting follow-up studies of client recidivism for community-based agencies in New York City (NY.ROCS). Agencies were not publicly identified and each analysis adhered to strict data-security safeguards that protected client identities. Project reports provided de-identified recidivism data that agency affiliates used to check their own effectiveness.

Reports contain detailed information without identifying agencies or their clients: Agency B Agency C

The process begins when an agency sends JohnJayREC a list of former clients with identifying information (name, sex, race, date of birth, etc.). The clients are usually those served by the agency two to five years in the past (in order to allow sufficient follow-up time). Researchers work with state or local officials to request a criminal history search for each name on the list. Criminal history records typically include arrests and convictions. All researchers having any contact with the data are trained in relevant data security and privacy procedures.

The results of the client list and records search are combined in a single database by the public agency. Information from criminal history records is appended to the client information supplied by the service provider. All identifying information is stripped or anonymized before the combined data file is returned to John Jay College.

Researchers at JohnJayREC analyze the data to examine recidivism across different client characteristics and levels of program participation. The submitting agency receives a written report about the analysis, but the report never names the agency. Pseudonyms are used instead, such as Agency A, Agency B, etc. Reports include a brief, generic description of the programming offered by each affiliate, the types of clients it serves, and common referral sources, excluding any specific information that could reveal the identity of an agency. The goal is to supply enough information to provide context for comparing reports while preserving the confidentiality of all submitting agencies. Participating agencies know which report is theirs, but they do not know the identity of other participating organizations.

These procedures provide participating agencies with previously unavailable, detailed information about client recidivism. Agencies are encouraged to maintain their affiliation with the project and to review their outcomes annually.

Future Growth

JohnJayREC plans to grow the scope and diversity of ROCS projects. As the initiative involves more agencies, justice systems could build longitudinal evidence on key outcomes of interest. We invite potential affiliates to contact us in order to discuss the feasibility of participating in recidivism checkups.

Email: mailbox@jjay.cuny.edu

ROCS: Concept Paper