Reclaiming Futures and Organizing Justice for Drug-Using Youth

button_opendoc

Butts, Jeffrey A., Kathleen Tomberg, Jennifer Peirce, Douglas N. Evans and Angela Irvine

SUMMARY

Reclaiming Futures is an organizational change initiative that supports coordinated and individualized responses for justice-involved youth with problematic substance use issues. The initiative is managed by Portland State University’s Regional Research Institute and Graduate School of Social Work in Portland, Oregon. It began in 2001 by working with 10 communities across the United States. Fifteen years later, more than 40 jurisdictions have already implemented, or are currently implementing, the Reclaiming Futures approach.

logo_PSU
First funded by the Robert Wood Johnson Foundation (RWJF), the initiative targets six stages of the youth justice system: screening, assessment, service coordination, initiation of services, engagement of families and youth, and transitioning to community support. Reclaiming Futures is not a treatment program, although the quality of substance abuse treatment is relevant. It is a strategy for improving the focus and coordination of interventions for justice-involved youth with substance abuse issues. As such, its effectiveness cannot be evaluated solely by measuring youth outcomes like recidivism and renewed drug use. Recent research, however, suggests that it may have positive effects on those outcomes as well.

cover_changingsystems2007In the first evaluation of Reclaiming Futures, a research team from the Urban Institute and the University of Chicago estimated the initiative’s impact in the first 10 sites by conducting surveys of system actors and their community partners (Butts and Roman 2007). The study’s questionnaire measured perceptions of juvenile justice and substance abuse treatment systems on three major dimensions (administration, collaboration, and service quality).

In 2015, the Research & Evaluation Center at John Jay College of Criminal Justice administered the same questionnaire in 24 communities implementing Reclaiming Futures. The study compares the perceptions of people working in Reclaiming Futures communities today with those of similar colleagues from nearly ten years ago. Nine of 24 sites in this study participated in the 2007 study as well, but the respondents in 2015 were not the same as those surveyed in the earlier evaluation. Thus, the study compares similar but distinct samples of youth service professionals at two different points in time. Nearly half the invited respondents (49%) completed surveys in 2015 (N=128).

When researchers isolated findings from the nine sites that participated in both the original 2007 evaluation and the most recent survey, the data suggest that communities with the strongest engagement in Reclaiming Futures tend to have more positive perceptions of their youth justice and substance abuse treatment systems, including key facets of administration, collaboration, and overall system quality. In communities where the original survey scores increased significantly during the early years of Reclaiming Futures, improvements were sustained through 2015. Thus, robust implementation of Reclaiming Futures may be associated with lasting improvements in system operations.

INTRODUCTION

Reclaiming Futures assumes that positive youth outcomes are achieved when service delivery systems are closely coordinated and provide just the right amount of individualized help with the least possible amount of coercion. Reclaiming Futures is designed to improve outcomes for justice-involved youth, but treatment is only a part of the model. Some drug-using young people need evidence-based treatments that focus on substance abuse. Many, however, simply need positive resources and supports, whether from professional agencies, community partners, or families.

Unlike many initiatives in the substance abuse field, Reclaiming Futures was not designed to deliver a specific treatment program. Rather, it was a strategy for implementing organizational changes that improve the capacity of the justice system to respond effectively to youth involved with alcohol and other drugs, regardless of the severity and urgency of their substance use.

In the first phase of Reclaiming Futures, 10 participating communities worked to change how their organizational networks responded to drug-involved youth in the justice system. Multidisciplinary teams collaborated to create coordinated efforts from what were often disparate collections of autonomous provider agencies. Each site developed its own goals and strategies, but all sites relied on judicial leadership, court/community collaborations, inter-organizational performance management, enhanced treatment quality, and agency partnerships.

Reclaiming Futures continued to expand after its initial phase. RWJF provided funding for the National Program Office to work with four new communities. Nine other sites received technical assistance and training from the National Program Office with funding from the federal Office of Juvenile Justice and Delinquency Prevention (OJJDP) and the Center for Substance Abuse Treatment (CSAT). In 2012, OJJDP funded three additional sites to work with the National Program Office. The Kate B. Reynolds Charitable Trust and RWJF cooperated to support Reclaiming Futures in six North Carolina counties (including one of the counties previously funded by OJJDP). Another eight North Carolina counties received supplementary funding from the Governor’s Crime Commission, the North Carolina Department of Public Safety, and the Duke Endowment. More sites launched in 2014 with funding from the Conrad Hilton Foundation and in 2015 with support from RECLAIM Ohio and the Ohio Department of Youth Services.

By 2016, there were 42 jurisdictions at various stages of implementing the Reclaiming Futures model, ranging from brand new sites to longstanding and deeply established projects, to legacy communities that were no longer in direct contact with the National Program Office. The 10 initial sites were free to continue implementing the Reclaiming Futures approach after 2008, but they were not required to remain in contact with the National Program Office of Reclaiming Futures. As a system reform strategy, there is no distinct “end point” to Reclaiming Futures. The leaders of the initiative hope that the approach becomes “normalized” in each community, even if the Reclaiming Futures brand name becomes less prominent and perhaps forgotten by local practitioners.

Reclaiming Futures Sites

sitetable_medium

POLICY CONTEXT

Youth who commit crimes, including illegal drug use, are adolescents. They have lower impulse control and are not yet proficient in making rational decisions that account for long-term consequences (Steinberg 2009). With support and continued development, however, adolescents are more likely to change their behavior than are adult offenders (Feld 2013). Relatively few juvenile offenders become persistent offenders into adulthood (Moffitt 2006). Until there is a perfect way to identify those youth who are most likely to continue offending after adolescence, the justice system must weigh the risks of intervening versus not intervening.

For decades, juvenile justice policy been designed to limit the legal coercion used to intervene with adolescents because it could increase their criminal involvement and actually produce worse outcomes than would less formal and less coercive approaches (President’s Commission on Law Enforcement and Administration of Justice 1967). For these reasons, the juvenile justice system has always emphasized “diversion,” or the practice of handling youthful offenders outside of the formal justice system whenever possible (Whitehead and Lab 2001; Zimring 2000). Providing services and supports for youth without involving them in the formal justice system reduces the risk of negative consequences while hopefully lowering recidivism and lessening the stigma associated with justice involvement.

The goals of diversion are to:

  • Reduce juvenile recidivism by decreasing youth contact with the juvenile justice system, as contact with the system tends to result in higher rates of re-arrest.
  • Ensure that minimally offending juveniles avoid system involvement, thus reducing the stigma associated with juvenile court adjudications and legal records of delinquency.
  • Provide alternatives to formal processing that give decision-makers viable options for handling delinquent youth in their own communities and that reserve out-of-home placement (e.g., detention, secure and residential facilities, group homes) for the relatively small number of high-risk youth.
  • Offer treatment programs for youth that attend to their individual risks and needs.
  • Minimize the costs associated with repeat juvenile offending, including educational failures, unemployment, and behavioral health issues.
  • Increase the participation of family and community members in youth services.

Any intervention that prevents more formal or coercive processing may be considered “diversion.” Diversion may begin when law enforcement responds to young offenders without resorting to arrest. A police officer may give youth a warning and possibly an escort home without making an arrest. Prosecutors may divert youth by withholding formal charges in exchange for their agreement to participate in a program of informal supervision and services. Judges use diversion when they refer juveniles to community-based treatment programs without adjudication or court orders.

Diversion has been part of juvenile justice since the first separate juvenile court opened in Chicago in 1899 (Feld 2013). The separate juvenile court itself, in fact, is a form of diversion. Juvenile courts prevent youth from entering the criminal (adult) justice system, giving them a chance to avoid additional justice involvement and the potentially lifelong burden of a criminal record.

Diversion, however, is not free of risk. Without the authority of court orders and the resources that formal systems provide, youth-serving agencies may be poorly managed. Youth and families may resist participating in services altogether. It may be difficult to track service participation for evaluation purposes. Keeping youth away from unnecessary coercion and stigma is an appealing notion, but informality increases an array of management challenges that may impede the effectiveness of interventions. This is especially true for youth entering the juvenile justice system with serious drug problems.

Youth Justice and Substance Abuse

Improving justice interventions for youth affected by problematic drug and alcohol use is more complicated than one might think. First, most of the young people involved in the justice system have some experience with alcohol and other drugs, but very few (perhaps one in ten) could be described as dependent or addicted. Intervention programs designed around an addiction model are not appropriate for the majority of young offenders, but treatment providers may struggle to find an effective alternative.

Second, over-intervening to prevent all drug use could end up causing more harm than drug use itself. As mentioned previously, formal intervention by the legal system comes with the risk of negative consequences. Being arrested, labeled as an offender, and forced to comply with court-imposed treatment can reinforce a young person’s anti-social attitudes, resulting in more rather than less offending (Wiley and Esbensen 2016). Justice officials must identify the actual risks presented by a young person’s drug use and not simply respond to its illegality.

Substance abuse treatment programs include a wide range of interventions – ranging from medications, therapy and counseling with individuals and families, to life-skills training, basic health supports, and spiritually oriented activities. These take place in settings as diverse as schools, outdoor camps, and locked facilities. Research studies examining the outcomes of different treatment methods employ different study designs, follow-up periods, and definitions of success, all of which make clear comparisons quite challenging. Tailoring treatment plans to a youth’s individual circumstance is essential to avoid inappropriate or excessive treatment.

Debates about drug policy seem endless in the U.S., in part because responses to drug abuse reflect deep political and cultural differences about the nature, severity, and dangers of substance use. Conventional opinion assumes that any consumption of illegal substances is problematic and that the goal of the justice system is to eliminate all illegal drug use—which for adolescents includes alcohol. This assumption is reflected in evaluation studies that define the “success” of an intervention as total abstinence. Yet, many researchers would suggest more nuanced indicators, such as general health and reductions in the severity and risks presented by an individual’s drug use (Fischer et al. 2015).

Globally, there are growing doubts about prohibition-oriented laws and other tactics of the “war on drugs,” given that they have not meaningfully reduced production, trafficking, or consumption of illicit substances and have instead created unintended costs and consequences (Rolles et al. 2012). Some researchers suggest that drug policy should be evaluated with a focus on human rights, health, and community well-being, rather than simply the interdiction of illegal substances (ICSDP 2016). In this context, it becomes more difficult to evaluate the effectiveness of youth justice interventions by measuring drug use alone.

The National Institute on Drug Abuse (NIDA) proposed a set of “Principles for Adolescent Substance Use Disorder Treatment” that reflect mainstream political thinking (NIDA 2014). The principles focus on proactive treatment; disorders should be addressed promptly and medical staff should ask adolescents about any substance use during routine visits. When needed, treatment should involve individualized and holistic approaches, include family, address mental health conditions not related to drug use, and be conscious of previous violence or abuse. The NIDA principles reinforce the prominence of behavioral therapies, continuity of care, and testing for diseases that are transmitted through injection drug use, such as HIV and Hepatitis B and C.

The NIDA principles view substance abuse as a health problem, and rightly so. The principles are virtually silent, however, on the iatrogenic consequences of relying on legal coercion to address health problems. In the “frequently asked questions” segment of the principles website, NIDA laments that justice involvement is an “unfortunate” reality for youthful drug users, but then it advises readers that justice involvement presents a “valuable opportunity” for intervention. The principles never acknowledge the harmful social and legal effects of using the justice system to intervene in youth drug use. The only outcome of interest is substance use itself.

The NIDA principles encourage drug treatment advocates to take a very aggressive view. Treatment may benefit adolescents even with “non-addictive” levels of use, and legal coercion is welcomed because it may ensure that adolescents continue and complete drug treatment programs. The agency’s principles are quite broad, naming treatments as “evidence-based” as long as research has identified drug-related benefits for particular subjects in certain programs and contexts. The principles cannot guarantee that all models are effective for all individuals, and they do not consider whether the benefits of treatment may, in some cases, fail to compensate for the harm caused by whatever legally coercive means are used to ensure an individual’s compliance with treatment.

Substance Abuse Treatment Approaches*

treatmenttableb
* Adapted from SAMHSA and DrugAbuse.gov

NIDA includes five behavioral treatments among its recognized approaches: the adolescent community reinforcement approach, cognitive behavioral therapy, contingency management (offering positive incentives and rewards), motivational enhancement therapy, and twelve step therapy. The second category, family treatments, includes brief strategic family therapy, family behavior therapy, functional family therapy, multi-dimensional family therapy, and multi-systemic therapy. Medications are becoming more widespread in substance abuse treatment programs, although NIDA notes that the FDA has not approved any of these for cannabis, cocaine, or methamphetamine addictions. The NIDA principles point to evidence supporting other medications: buprenorphine, methadone, and naltrexone for opioids; acamprosate, disulfiram, and naltrexone for alcohol; and burpropion, nicotine replacement, and varnicline for nicotine. On the list for recovery services, NIDA’s evidence-based treatments include assertive continuing care, mutual help groups, peer recovery support services, and recovery high schools.

Like the NIDA principles, drug treatment systems are typically designed for addiction disorders. Yet, most of the adolescents involved in the juvenile justice system cannot be described as addicted. According to the 2014 National Survey on Drug Use and Health (Center for Behavioral Health Statistics and Quality 2015), nine percent of all adolescents (ages 12-17) are current users of illicit drugs (i.e. some use within the past 30 days). Most of these users will not develop substance use disorders. Just five percent of adolescent drug users meet the criteria for substance use disorder and the rate has been declining since 2002 (when it was 9%).

Of course, frequent users of alcohol and other drugs are more likely to have contact with the justice system and more likely to develop substance use disorders. Research suggests an association between early onset of substance abuse by adolescents and subsequent patterns of more serious and chronic criminal offenses (Young, Dembo and Henderson 2007). The causal link between substance abuse and crime among juveniles, however, is not a simple one. Drug use may exacerbate a juvenile’s contact with the justice system, but this may be due to the peer associations formed during illegal (thus risky) drug use rather than to the addictive properties of drugs (Butts and Roman 2004).

A large number of people referred to substance abuse treatment in the US are adolescents. In 2007, about 11 percent of all treatment admissions involved people under age 20 (Tanner-Smith, Wilson and Lipsey 2013), and nearly half of adolescents in treatment (45%) were referred by criminal justice authorities. Three-quarters of adolescents in substance abuse treatment are referred for marijuana use (Mutter, Ali and Strashny 2015). Not surprisingly, marijuana is the most commonly used illicit drug among adolescents (7% of all adolescents are current users). The key policy question is whether unchecked marijuana use is harmful enough to offset the harmful effects of using the justice system to coerce youth into treatment. The question becomes even more complicated as a growing number of states change their laws to make marijuana use illegal only for minors.

Balancing Risks

Using the justice system to intervene in adolescent substance use is risky. Young people who are arrested and brought to court may be more likely to grow into adult criminals than similar youth who are kept out of court and allowed to discover for themselves how to be law-abiding and drug-free (Bernburg and Krohn 2003). If drug treatment programs accept too many clients without serious drug problems, their effect on drug-related crime will diminish and the programs may create more harmful effects for youth than the drug use that brought them to the program. Even worse, when serious drug users face the risk of coerced treatment and criminal penalties, they are likely to avoid important public health services (HIV tests, clean needle exchanges, etc.) (Werb et al. 2016).

Treatment effectiveness—even evidence-based treatment—is not guaranteed (Tanner-Smith, Wilson and Lipsey 2013). Family therapy showed the best results across all comparisons, while cognitive behavioral therapy (CBT) and motivational enhancement therapy (MET) showed some positive effects compared with placebos. None of the programs, however, worked every time and for every individual. Research tends to show the strongest treatment results among marijuana users, but it is difficult to determine whether this is due to better treatment, or to the fact that marijuana users find it easier to abstain when faced with coercion and possibly severe legal consequences.

Not all adolescent substance use is abuse, and even when abuse is indicated, it is not always severe enough to justify the collateral risks that come with coercive legal intervention. Youth justice officials must strike a balance between underreacting to potentially burgeoning drug problems and overreacting to adolescent-typical substance use merely because it occurs within the context of other law violations.

For these reasons, the Reclaiming Futures initiative coordinates the youth justice and substance abuse treatment systems. It knits together their efforts to maximize youth well-being by capitalizing on opportunities to prevent serious drug problems while avoiding undue harm from inordinate legal processing.

EVALUATION EVIDENCE

Reclaiming Futures presents serious challenges for evaluation researchers. The initiative’s most relevant outcomes are organizational and not easily tracked by the information systems used in youth justice. If an evaluation wanted to measure individual youth outcomes, they would also have to draw upon inter-organizational data from justice, health, mental health, education, and labor sectors, as well as an array of community organizations and neighborhood activity providers.

Because measuring the full range of client outcomes would be complex and expensive, the first multi-site evaluation of Reclaiming Future measured system-level outcomes indirectly with a survey of actors in youth justice and substance abuse treatment. More recent evaluations have focused on direct measures of individual-level and group-level differences, but on a narrower range of outcomes. All previous studies, however, suggest that Reclaiming Futures has positive effects. Even at the individual level, the economic value of Reclaiming Futures outcomes—such as preventing costly injuries, illnesses, and crimes—appears to outweigh the costs of implementing the initiative.

Urban Institute and University of Chicago

The Urban Institute and Chapin Hall at the University of Chicago collaborated in the first national evaluation of Reclaiming Futures (Butts and Roman 2007; Roman, Butts, and Roman 2011). As part of that evaluation, researchers conducted biannual surveys in each of the first 10 communities participating in Reclaiming Futures. Respondents answered questions about the quality and effectiveness of the juvenile justice and substance abuse treatment systems in their communities. Researchers constructed thirteen indices to represent the quality and effectiveness of local systems. Several indices focused on administration (i.e. access to services, data sharing, systems integration, resource management). Others focused on collaboration (i.e. client information, partner involvement, agency collaboration) and quality (i.e. alcohol and other drugs assessment, treatment effectiveness, targeted treatment, cultural integration, family involvement, and pro-social activities).

Study results showed that most indicators improved during the first three years of Reclaiming Futures implementation, with statistically significant increases in 12 out of the 13 indices. This suggested that Reclaiming Futures was a promising strategy for improving interventions for youth. The strongest results occurred in measures of treatment effectiveness, the use of client information in support of treatment, the use of screening and assessment tools, and overall systems integration. Four of the 10 sites showed significant and linear changes in two indices, while five showed significant and linear changes in between four and six indices. Researchers also analyzed “percentage improvement” in the index scores to account for the fact that each site started from a unique level of quality and effectiveness. Results showed that scores improved between 11 and 51 percent over three waves of data collection.

Although the results were generally positive, the study measured respondents’ subjective impressions of system performance rather than performance itself. The results could be affected by a form of social desirability bias, or a tendency for the people most deeply involved in Reclaiming Futures to give more positive answers. When researchers tested differences in respondent opinions based on proximity to Reclaiming Futures leadership, however, the results did not vary significantly and this appeared to support the validity of the survey approach. At best, however, the evaluation could only characterize Reclaiming Futures as a promising intervention strategy.

Chestnut Health Systems

Beginning in 2009, a group of researchers based at a drug treatment provider in Illinois modeled the effects of Reclaiming Futures in a study of five juvenile drug courts in varying regions of the country (i.e. Pacific-Alaska, Pacific, Rocky Mountain, Southwest, and the Great Lakes) (Dennis et al. 2012). Study outcomes focused on treatment services delivered (e.g., number of days in substance use treatment services, relative costs of treatment services versus hospital, and detention costs) and behavioral measures (e.g., rates of substance abuse and the number of crimes committed). Researchers tracked youth served in five Reclaiming Futures drug courts and compared them with a matched comparison cohort of youth from other juvenile drug courts. The key research question was whether the addition of Reclaiming Futures improved juvenile drug court effectiveness. The study, however, did not measure system change as an outcome. Furthermore, the comparison courts were incredibly well-funded programs that were being operated with federal support and, therefore, may not have been a totally appropriate representation of typical juvenile drug courts.

The evaluation collected data about youth in the Reclaiming Futures juvenile drug courts (N=462) and compared their outcomes with youth from 16 other juvenile drug courts (N= 1,517). Data were drawn mostly from the Global Appraisal of Individual Needs (GAIN), a tool developed and promoted by Chestnut Health Systems, the organization conducting the study. Study measures included treatment involvement (e.g., treatment engagement, positive discharge status), and 12 youth outcomes (e.g., substance use, victimization, emotional problems, and interpersonal problems), as well as subsequent law violations and the likely costs of those violations.

Dennis and his colleagues identified all participants with complete records (including the GAIN data) and constructed a comparison group for youth in Reclaiming Futures drug courts using propensity score matching. Outcomes were compared at intake and at three, six, and 12 months after intake. The study also conducted a limited form of cost-benefit analysis by estimating the costs of service utilization according to the self-reported frequency of those services, the unit cost of services and the costs of crime.

The study found few significant differences in client outcomes between the two sets of juvenile drug courts. Due to differences in grant requirements, the Reclaiming Futures sites relied more on individual-focused, evidence-based approaches (e.g., Seven Challenges), while the comparison courts used more family-oriented programs (e.g., Functional Family Therapy, Multisystemic Therapy). Reclaiming Futures youth received more services overall during the intervention year and committed fewer violent crimes, but the difference was not significant after controlling for individual differences among participants.

University of Arizona

In a more recent study, Korchmaros et al. (2015) evaluated the contributions of Reclaiming Futures in the same five juvenile drug courts. The study compared juvenile drug courts using Reclaiming Futures with juvenile drug courts not using Reclaiming Futures. The analysis focused on individual youth outcomes. It did not assess Reclaiming Futures as a systems-change strategy and it did not rely on systems-change as the main indicator of effectiveness, although it did incorporate the Urban Institute’s systems-change survey items in considering influences on individual outcomes.

The study followed 522 juvenile drug court clients (age 12-18 years) during a five-year period and tracked their substance use patterns (length of use and intensity of substance use) and their involvement in substance use treatment (whether enrolled in Reclaiming Futures services or transferred to other services). The main sources of data for this study were client data (including demographics, treatment participation and drug use) and interviews with juvenile drug court staff and local expert informants. The client data included information from each program’s use of the Global Appraisal of Individual Needs (GAIN) assessment tool. Data collection methods included the compilation of administrative data, web-based surveys of clients, qualitative interviews, observations of drug court team meetings, and site visits.

The results showed that youth in the drug courts using Reclaiming Futures had a higher likelihood of receiving substance abuse treatment than would be typical in juvenile drug courts (using comparative data from a published meta-analysis). The evaluation also found a connection between greater fidelity to the Reclaiming Futures model (particularly in cross-system collaboration and the use of assessments) and better youth outcomes in terms of treatment access.

The study was not without limitations. While courts using the Reclaiming Futures approach tended to have more success in attaching youth to treatment services, there was substantial variation across the sites. The study also compared the Reclaiming Futures courts to other juvenile drug courts, but the client composition of Reclaiming Futures courts may not be typical. Participants were mostly male (74%) and youth of color (65%), but these proportions were somewhat lower than the average among juvenile drug court clients in general.

Researchers from Arizona also conducted a more detailed economic analysis of the same data (Carnevale Associates et al. 2015; McCollister et al. 2015). To determine whether the costs of implementing and operating Reclaiming Futures as part of a juvenile drug court were matched or outweighed by any savings attributable to the program’s results, the research team first calculated the cost of the enhanced program, incorporating direct budget costs (e.g., staff, activity supplies), in-kind costs (e.g., volunteer time), and incremental costs (e.g., additional time and resources expended beyond those required for the existing drug court programs). Adding all costs for all participants, the study then estimated total costs per participant for the average length of program involvement ($38,288).

The study monetized several activities typically associated with adolescents involved in substance use and juvenile justice: criminal offenses, days of mental health or physical health problems, and absences from school or work. Drawing upon existing literature, the authors estimated the cost of each element, with the largest amounts attached to criminal incidents (e.g., $3,900 average for larceny and $12,000 average for car thefts).

Finally, the study team interviewed a sample of juvenile drug court participants at the beginning of the drug court program and again one year later. Each subject provided self-reported information about their recent experiences in the various cost areas. This enabled the study to calculate the average change in each area across the entire sample, which resulted in an estimated cost savings of $122,857 per youth. Most of the savings came from reductions in reports of criminal offending. Using these figures, the research team estimated net savings of $84,569 per youth. Applying the estimate to 139 youths in a program for one year, the savings amount to $11 million.

Of course, this analysis is also far from perfect. It compared costs for a sample of youth before and after treatment in a drug court using the Reclaiming Futures model, but it did not include a comparison group from another drug court or any other intervention program. Thus, it is possible that the same youth would have reported fewer offenses and other problems one year after the first interview, regardless whether they had been in a juvenile drug court program, a juvenile drug court program with Reclaiming Futures, or indeed any program at all.

This is especially problematic when interventions tend to occur soon after an event or series of events that place subjects at a higher risk of being selected for intervention. For example, when youth are more likely to be referred for intervention soon after a period of greater-than-usual offending, they should demonstrate less offending at any future point in time simply due to the statistical artifact known as “regression to the mean.” Combining an increased odds of selection with regression to the mean produces the “selection-regression artifact” (Maltz et al. 1980), a well-known source of error in simple pretest-posttest evaluation designs.

A Promising Approach

Youth justice policymakers need a more definitive study of Reclaiming Futures, but every previous attempt to evaluate the approach has produced at least some encouraging findings. The Urban Institute/University of Chicago study suggested that Reclaiming Futures had positive effects on the organizational networks that operate youth justice and substance abuse treatment systems. Both the Chestnut study and the University of Arizona study found that adding Reclaiming Futures to juvenile drug courts enhanced their effects on youth, perhaps in a way that was cost-beneficial. Researchers should continue to investigate the effects of Reclaiming Futures, and future studies should measure both its organizational and individual benefits for system reform and youth outcomes.

15 YEARS OF RECLAIMING FUTURES

This study is a follow-up to the first evaluation of Reclaiming Futures. It applies the same survey method used by the Urban Institute and University of Chicago researchers and it includes nine of the same sites studied ten years ago. A one-time survey of 24 Reclaiming Futures sites measured the perceptions of people working in the youth justice system and other relevant organizations. Items in the survey asked whether the principles and practices promoted by Reclaiming Futures were apparent in local service systems.

With help from staff in the National Program Office of Reclaiming Futures, the research team built a respondent list for 24 jurisdictions involved in Reclaiming Futures. Nearly half (49%) the respondents completed the new survey (N= 128) during the last few months of 2015. In addition, the study team attended a June 2015 Reclaiming Futures conference and conducted semi-structured interviews with 16 individuals working with relevant agencies in various communities. Interviewees responded to questions about what has changed in Reclaiming Futures communities and how and why the changes happened.

Interviews with Initiative Leaders

Several members of the study team attended the 2015 National Reclaiming Futures conference in San Diego and conducted 16 in-person interviews. Reclaiming Futures project directors, judges, staff members, and consultants with the National Program Office responded to a standard set of questions. Interviews were semi-structured and included questions about the key elements of the Reclaiming Futures approach, the main challenges facing project sites, how Reclaiming Futures evolved over time, and areas in need of improvement. Several themes emerged from the interviews.

Interviewees described Reclaiming Futures as a comprehensive strategy for changing the agencies and systems that serve justice-involved youth, but details differed. One experienced evaluator described Reclaiming Futures as a “systems change model,” while a longtime practitioner described it as explicitly not a model, but rather a set of “principles” put into practice through “key elements.” Another observed that Reclaiming Futures was “not just a piece of the pie; it is the whole pie.”

Most people involved in Reclaiming Futures identified two elements as essential to its success: a) a coordinated approach to assessment, referral, and treatment; and, b) an emphasis on involving community and family members in meaningful and sustained ways. These elements typically require youth-serving agencies to change the way they conduct daily tasks. This includes how they see their roles and “territory,” and how much time they spend interacting with youth and families rather than with bureaucrats and other professionals. In the juvenile justice system, it is easy for a young person to get “lost” among the many agencies and programs using different assessments and varying treatment approaches. Reclaiming Futures encourages local systems to establish standardized assessment processes and to expand the responsibility for interventions to the entire network and not just to one assigned agency or staff member.

Elements of Success

Interviewees tended to agree that Reclaiming Futures is about building new forms of collaboration among agencies and communities. A crucial element of its success is the structure it provides for this collaboration. For example, several people from each site participate in one of the Reclaiming Futures “fellowships,” or cross-site affinity groups that support and inform implementation. These groups include a judicial fellowship, a treatment fellowship, a community fellowship, etc. Fellowship members hold regular conference calls and meetings and attend training sessions in which they exchange experiences and develop a mutual understanding of their roles in Reclaiming Futures.

Of course, changing organizational systems requires more than an occasional meeting. One judge noted that inter-agency relationships only change with consistent leadership and active involvement at the highest levels— i.e. judges, agency directors, and elected officials. Other stakeholders emphasized the importance of real resources to support change, such as reliable funding, standardized assessments, and data systems to monitor service delivery and outcomes.

Interviewees endorsed the emphasis that Reclaiming Futures places on individualized service plans that are based on youth needs and strengths and tied to community resources. Reclaiming Futures does not require a standard approach to youth services. While there may be established principles that service plans should follow in general, successful implementation occurs through an “iterative” process in which effective strategies are adapted to the local context. Reclaiming Futures does not require specific evidence-based treatments for youth because this would imply that a single plan should work in different settings despite local differences.

Professionals involved in Reclaiming Futures told a number of stories about young people they knew who were able to turn their lives around through sustained, community-based support. Effective intervention, in their view, begins with listening to the opinions and needs of young people and their care givers. Listening to youth leads to better treatment decisions. Several sites mentioned the importance of “natural helpers” as a component of the Reclaiming Futures approach. These are community members who volunteer to be mentors and to provide informal supports for youth. They may be recruited from existing community groups, schools, and religious communities. Being creative in identifying non-professional supports for youth seems to result in more sustainable intervention plans.

Interviewees also valued the use of data collection systems to track intervention activities and outcomes. One justice professional commented that the number of youth referred to the local juvenile court system dropped 25 percent in recent years, but the number of juveniles completing substance abuse treatment tripled. Even youth identified as the most likely to end up “deep in the system” showed improved outcomes. Reclaiming Futures encourages systems to track actual service outcomes against expected outcomes for every youth and for every agency in each local network.

Challenges

Professionals involved in Reclaiming Futures initiatives mentioned three distinct categories of challenges: adapting Reclaiming Futures for the local context; securing funding to support sustainability; and building genuinely trusting relationships in systems that are often beset by division and tension. Adapting Reclaiming Futures principles to local circumstances was especially complex, according to some interviewees, because Reclaiming Futures is not an off-the-shelf program.

Some stakeholders wondered whether Reclaiming Futures was even a “model.” To prevent new sites from expecting a “pre-cooked package,” they recommended that the National Program Office should develop an explicit articulation of Reclaiming Futures principles and processes, along with a clear statement that it is not a program or a model. Training materials for new sites should provide more tools to help practitioners move from theory to action, with room for local adaptation.

Adapting Reclaiming Futures to the unique circumstances of each site requires honest discussions about how existing structures might have to change to support youth and families. Interviewees reported some problems in designing reform strategies as Reclaiming Futures does not provide a clear “recipe for change.” Several interviewees noted that, even when bureaucratic organizations are willing to work with one another (which itself is an achievement), they still struggle to determine how to work with one another. Agencies are often reluctant to expose their own inadequacies.

The main obstacle reported by interviewees was not lack of funds, but rather officials’ shifting priorities for funding. Public officials are willing to pay for professional services, but they are hesitant to spend money on agency infrastructure or on improving trust between justice systems and community members. Reclaiming Futures leaders stressed the importance of non-service elements, including data systems, staff training, leadership development, and activities to engage communities and families. These are not one-time expenses; they must be part of an ongoing investment.

Building trusting relationships among agencies and stakeholders was one of the more complex challenges in Reclaiming Futures. One site leader described how local networks are often hindered by negative stereotypes about justice-involved youth. Another program director echoed this idea, noting that some public officials were unwilling to engage in youth justice reforms at least in part because they were incapable of understanding communities different from their own. Others suggested that public officials may not appreciate the deep mistrust and resentment with which some communities view the justice system—especially when these tensions are rooted in racial and economic inequality. In general, officials may be hesitant to invest resources in systems that serve people they see as “other.” Overcoming such resistance requires creative dialogue and leadership.

Areas for Improvement

Interviewees in this study believe that Reclaiming Futures leaders worked hard to engage community stakeholders, treatment providers, and volunteers. This is an ongoing process and cannot be achieved with a single event at the outset of an initiative. Engagement requires clear and consistent messages from key justice officials so that people in divergent roles may begin to build a common language. The Reclaiming Futures approach asks communities to develop tailored strategies to reach and engage youth who have long been marginalized. These relationships do not come easily.

Several interviewees commented that once a Reclaiming Futures effort has been underway for some time, stakeholders should be willing to identify strategies that are not working and cut resources for those activities. Agencies in any local network are inevitably in competition over scarce resources. Some interviewees reported success in overcoming these tensions by cooperating across agencies to prepare joint funding proposals. Others believe that working to build a true continuum of care, in which multiple providers offer diverse services, will help to bring network members together.

Some professionals involved in Reclaiming Futures suggested that inter-organizational data access is a critical component of the initiative and that data must be a focus from the very beginning. Frequently, youth justice systems implement new data protocols only after major reforms are mandated—such as new inter-agency collaborations, assessment tools, and treatment approaches. This creates an absence of information about conditions prior to reform. With comprehensive data collection both before and after organizational reforms, policymakers would be able to assess the effectiveness of reform initiatives.

Evolution of Reclaiming Futures

Reflecting on the history of Reclaiming Futures in their communities, interviewees for this study believed the initiative began to solidify across their agency networks only after people let go of the notion that Reclaiming Futures was a stand-alone “program.” Understanding Reclaiming Futures as a network reform strategy allowed professionals to shift their focus toward coordinating support structures for youth rather than simply treating diagnoses and deficits. Over time, the people leading Reclaiming Future also began to see how trauma and past violence affected young people’s interactions with the justice system, and they could see how important it was for professionals and agencies to avoid aggravating those effects.

Some interviewees with considerable experience leading Reclaiming Futures sites argued that the most important changes sparked by the initiative happened as a result of improved relationships among the broad network of agency stakeholders and community members. Sites that may have initially focused their efforts strictly on juvenile justice and drug treatment soon learned the importance of engaging with schools, health clinics, sports programs, arts organizations, etc. They also embraced a more nuanced view of evidence-based practice. It became obvious that adopting an overly narrow focus on “what works” would tend to exclude important community partners.

In the most experienced Reclaiming Futures sites, network leaders began to claim their own expertise. They knew that their youthful clients would benefit most from living in healthy families and communities, and that supporting such an agenda would end up affecting a much wider group of youth than just those formally participating in justice systems and drug treatment. They began to see their work as a reform effort to improve community and public health.

2015 Survey of Community Networks

To explore the perceptions of current Reclaiming Futures stakeholders, the study team replicated the survey from the original evaluation conducted by the Urban Institute and the University of Chicago (Butts and Roman 2007; Roman, Butts and Roman 2011). The survey was a cost-effective means of assessing system-level dynamics in such a complex and multifaceted initiative, particularly given that implementation occurred over many years with differing levels of intensity.

Researchers identified survey respondents by asking project directors in the 24 sites to submit lists of people involved in the design and delivery of juvenile justice services and substance abuse treatment in their areas. Ideally, each list included a mix of professionals, community activists, and volunteers. Individual respondents typically included judges, probation officers, educators, substance abuse and mental health treatment professionals, community organizers, members of faith-based organizations, and youth advocates.

The study team attempted to confirm that the respondents nominated by project directors were representative of the expert population in each community and not simply people likely to view the juvenile justice system favorably. To do this, researchers reviewed public, online directories of government officials and nonprofit organizations involved in juvenile justice and substance abuse treatment in each site. Key people were contacted and invited to nominate additional respondents in each site. Thus, respondent lists included people in relevant system roles and not merely those directly involved in Reclaiming Futures. The study team combined all names into a complete respondent pool for each community.

Because each list was reviewed and confirmed as complete by local project directors, researchers were able to consider the nominated respondents to be the ideal informants in each community rather than just a few people from a large group of possible informants. In other words, the evaluation team could assume that when 15 people were on a list of ideal respondents in Seattle, and 10 of them responded to the survey, the 10 Seattle respondents could be treated as a sample of ten experts drawn from a population of 15. Statistical tests in the study could then incorporate a “finite population correction” that produces smaller margins of error with limited samples.

Level of Community Engagement

Reclaiming Futures sites vary in their level of engagement with Reclaiming Futures and the National Program Office. To create a proxy measure of engagement, the study team interviewed two members of the national leadership team for Reclaiming Futures who rated the 24 sites on intensity of implementation. Ratings were based on the staff members’ “best guess” about the relative level of engagement in each of the 24 surveyed sites. Staff members based their ratings on each site’s consistency of funding, scope of effort, length of time involved in Reclaiming Futures, frequency of network meetings, use of training resources, interaction with other sites, strength of participation in the Reclaiming Futures Fellowship groups, and contact with the National Program Office.

The staff members assigned all 24 sites a score of 1, 2, or 3, with 1 representing very little engagement and 3 indicating strong and full engagement. Because the 3-point scale turned out to be insufficiently sensitive, the staff members were allowed to assign scores in between the integers using decimal points. Researchers arrayed the scores on a 5-point scale, from 5 (“strong”) to 2 (“weak”) implementation, with the lowest score of 1 (“none”) indicating that a site was either unable to implement Reclaiming Futures or had little to no contact with the National Program Office. Survey respondents were from communities at various levels of engagement (Table 1).

table1b

Results

Surveys were administered via the internet (using SurveyMonkey). After five weeks and several reminder contacts, the study was able to obtain responses from 128 people in the 24 sites, for a successful response rate of 49 percent. The previous evaluation’s response rate was higher (an average of 70% across six administrations), but it also sampled fewer sites (10). Additionally, those sites were in a more intense period of engagement with significant funding, which may have affected the respondents’ willingness to participate.

Survey respondents answered a number of questions about the quality and effectiveness of the juvenile justice and substance abuse treatment systems in their communities. Questions were asked in the form of brief statements, to which respondents indicated whether they strongly disagreed, disagreed, were neutral, agreed, or strongly agreed (there was also a “not applicable” option). Some statements were worded negatively, but all items were coded so that higher scores always indicated more positive opinions. Responses were scored 1 through 5, from strongly negative to strongly positive.

As in the original study, fifty-eight survey items were compiled into 13 multi-question indices or scales across three categories—administration, collaboration, and quality (see the Appendix). Respondents were asked to agree or disagree with statements that loaded onto the indices. Items in each index were scattered throughout the survey and not asked in sequential order. Scores on a particular index were calculated as the numerical average of the answers to all the questions making up that index (Table 2). The indices were statistically reliable, as judged by a series of factor analyses that tested the extent to which each scale represented a single construct as completed in the original study.

table2b

The thirteen indices of systemic change included four indices related to the general concept of Administration, including Access to Services (i.e., the ease of client access to services/treatment), Data Sharing (i.e., the integration and sharing of information systems among agencies), Systems Integration (i.e., interagency coordination of policies and procedures), and Resource Management (i.e., organization, leverage of staff and funding). Three indices measured by the survey were related to the concept of Collaboration, including Client Information (i.e., agencies sharing client information to support treatment planning), Partner Involvement (i.e., the extent of interaction among Reclaiming Futures partner agencies), and Agency Collaboration (i.e., the quality of interagency relationships in the youth services field). Finally, six indices addressed the Quality of substance abuse treatment for youthful offenders, including Targeted Treatment (i.e., the availability of treatments appropriate for specific client groups), Treatment Effectiveness (i.e., the scope and impact of treatment services), AOD Assessment (i.e., the availability and use of effective screening and assessment tools), Family Involvement (i.e., the role of family members in designing and delivering services for youth), Cultural Integration (i.e., cultural competence and responsiveness), and Pro-social Activities (i.e., the use of pro-social activities for youth as a part of substance abuse interventions).

The highest mean score was for the Partner Involvement Index (4.0), which indicates that Reclaiming Futures communities are most positive about engagement and collaboration among their organizational networks. The next highest score (3.9) was for assessments of client use of alcohol and other drugs (AOD Assessment), which became one of the main areas of emphasis for Reclaiming Futures in recent years. The lowest mean scores were for the Access to Services Index (2.7) and the Targeted Treatment Index (2.9). This would suggest that client services always need improvement.

As mentioned above, nine of the 24 sites surveyed in 2015 were also surveyed in the original evaluation (survey data from 2003 to 2006). These were: Anchorage AK, Santa Cruz CA, Cook County IL (Chicago), Southeastern Kentucky, New Hampshire, Montgomery County OH (Dayton), Multnomah County OR (Portland), Rosebud SD (Sovereign Tribal Nation of Sicangu Lakota), and King County WA (Seattle). A comparison of the 2015 index scores in those nine sites with scores from the same sites a decade earlier shows the ranking of indices to be remarkably consistent (Table 3). Only a few of the 13 indices moved even one place in the rankings. The largest change was the order rank for the Systems Integration, which dropped two places between 2006 and 2015.

table3b

Twelve of the 13 index items had higher mean scores in 2015 than in 2003, but most of the increase occurred during the initial years of Reclaiming Futures between 2003 and 2006 (Table 4). Nine index scores declined slightly between 2006 and 2015. Of course, a decline in the index score does not necessarily imply that views on the topic are dramatically worse. The Partner Involvement Index score was slightly lower in 2015, but it still had the highest overall score in all three survey years, which means there was less room for improvement.

table4b

Several index scores suggest that gains from the early years of Reclaiming Futures were sustained in later years. The largest relative change was in Treatment Effectiveness. The mean score for that index rose from 3.1 to 3.6 between 2003 and 2006, and it was still 3.6 in 2015. A similar trend was evident in the Data Sharing Index, which grew 12 percent between 2003 and 2015, almost entirely during the early years of Reclaiming Futures. A number of other scores show sustained improvements from the first three years, including AOD Assessment and Cultural Integration.

Other items that improved in the first few years of Reclaiming Futures appeared to decline after 2006. Systems Integration, for example, jumped from 3.2 to 3.6 between 2003 and 2006, and then dropped to 3.3 in 2015. Resource Management increased from 3.5 to 3.8 in the first three years of the initiative, but fell back to 3.6 in 2015. Index scores for Family Involvement and Prosocial Activities were also down slightly in 2015 after growing substantially between 2003 and 2006.

Positive perceptions of local systems appeared to be related to the strength of Reclaiming Futures implementation (Figure 1). As mentioned above, the study team interviewed two leadership staff from the National Program Office before conducting the survey in 2015. From these interviews, researchers obtained informal ratings of each site’s engagement with the initiative on several dimensions (consistency, communication, funding support, etc.). This allowed the study to explore any differences in the index scores according to the level of each site’s engagement. There were significant associations between levels of engagement and index scores in several of the survey indices.

figure1b

For example, Reclaiming Futures sites that remained strongly engaged reported significantly better Access to Services for clients than did sites with weak engagement or no engagement at all (p < 0.05). Sites with strong engagement also reported better scores on Resource Management, with a statistically significant difference between sites with strong engagement and no engagement or no contact. Several other indices appeared to differ by level of engagement, although not always in a consistent way. Sites with stronger engagement reported significantly better use of client information compared with sites that were either out of contact or unable to implement Reclaiming Futures (p < 0.05). One of the most consistent and linear associations with implementation was observed in the average scores for the AOD Assessment Index. Respondents in communities that engaged more intensively with Reclaiming Futures had more positive opinions of practices related to AOD assessment in their youth justice systems.

DISCUSSION

The results of this study suggest that implementation of Reclaiming Futures has perceptible benefits for youth justice and substance abuse intervention systems, and some of these benefits may persist a decade after initial implementation. In nine of the first ten Reclaiming Futures sites, many of the system quality indices measured by the 2007 evaluation of the initiative appear to have been sustained through 2015. Perceptions of system effectiveness are still generally positive across all active sites, and several key indices are significantly related to the strength of implementation.

The generally positive perception of Reclaiming Futures was supported by the study’s interviews with more than a dozen professionals affiliated with Reclaiming Futures. They noted that Reclaiming Futures provided an impetus for system changes and a flexible structure for making changes. Interviewees appreciated the ability of the Reclaiming Futures approach to accommodate the unique factors in each project site while focusing on building positive, community-based intervention strategies for youth. Several interviewees noted that the work of Reclaiming Futures is inherently collaborative and inter-organizational. While a judge or political leader might be an essential catalyst, the work to implement lasting change requires the sustained efforts of many partners.

This study presents a partial picture of how juvenile justice and substance abuse treatment professionals perceive system functioning. Given the complexity of the Reclaiming Futures model, researchers should continue to study its effectiveness using diverse methods. Researchers could administer surveys to a wider group of respondents, especially youth and families involved in the justice system. Case studies could shed light on the mix of factors that underlie the success of Reclaiming Futures from the perspectives of individual actors in specific agency settings.

As noted by several interviewees in this study, policy issues in youth justice and substance abuse are strongly influenced by larger social dynamics such as racism and poverty. Future research could explore how Reclaiming Futures strategies might better engage these challenges at a local level. This report provides useful information about youth-serving systems involved in the Reclaiming Futures initiative, but many important questions remain.
_____________________

Recommended Citation
Butts, Jeffrey A., Kathleen Tomberg, Jennifer Peirce, Douglas Evans and Angela Irvine (2016). Reclaiming Futures and Organizing Justice for Drug-Using Youth. New York, NY: Research & Evaluation Center, John Jay College of Criminal Justice, City University of New York.

Copyright by the Research & Evaluation Center, John Jay College of Criminal Justice, City University of New York (CUNY).

524 59th Street
New York, NY 10019
www.johnjayrec.nyc
Published online February 2016

Preparation of this report was supported by a grant from the National Program Office of Reclaiming Futures at Portland State University. Its contents were created by the authors without influence from the National Program Office other than to correct factual errors. Any views or opinions expressed in the report are those of the authors alone and do not necessarily reflect the policies of Reclaiming Futures, John Jay College, the City University of New York, or their funders.

The authors are grateful to the local community members who took the time to answer the study’s survey and to the professionals who consented to be interviewed, as well as those who agreed to be video recorded for this report. The authors also appreciate the assistance provided by several colleagues in the Research & Evaluation Center at John Jay College: Marissa Mandala; Megan O’Toole; Pamela Ruiz; and Maggie Schmuhl.
________________________

REFERENCES

Bernburg, Jon G. and Marvin D. Krohn (2003). Labeling, life chances, and adult crime: The direct and indirect effects of official intervention in adolescence on crime in early adulthood. Criminology, 41(4): 1287-1318.

Butts, Jeffrey A. and John Roman. (2004). Juvenile Drug Courts and Teen Substance Abuse. Washington, DC: Urban Institute.

Butts, J. A. and J. Roman (2007). Changing Systems: Outcomes from the RWJF Reclaiming Futures Initiative on Juvenile Justice and Substance Abuse. A Reclaiming Futures National Evaluation Report. Portland, OR: Reclaiming Futures National Program Office, Portland State University.

Carnevale Associates, LLC and University of Arizona – Southwest Institute for Research on Women (2015). Findings from the Economic Analysis of JDC/RF: Policy Implications for Juvenile Drug Courts. Tucson, AZ: University of Arizona.

Center for Behavioral Health Statistics and Quality (CBHSQ) (2015). Behavioral Health Trends in the United States: Results from the 2014 National Survey on Drug Use and Health (HHS Publication No. SMA 15-4927, NSDUH Series H-50). Washington DC: Substance Abuse and Mental Health Services Administration, U.S. Department of Health and Human Services.

Dennis, Michael, Kate Moritz, Rachel Meckley and Nora Jones (2012). Evaluating the Impact of Adding the Reclaiming Futures Approach to Juvenile Treatment Drug Courts. Presentation to the Robert Wood Johnson Foundation. Normal, IL: Chestnut Health Systems.

Feld, Barry C. (2013). Cases and Materials On Juvenile Justice Administration (4th ed.). St. Paul, MN: West Academic Publishing.

Fischer, Benedikt, Yoko Murphy, Katherine Rudzinski, and Donald MacPherson (2015). Illicit drug use and harms, and related interventions and policy in Canada: a narrative review of select key indicators and developments since 2000. International Journal of Drug Policy, 27: 23-35.

International Centre for Science in Drug Policy (ICSDP) (2016). Open Letter: A Call for a Reprioritization of Metrics to Evaluate Illicit Drug Policy. Toronto, ON: Author.

Korchmaros, Josephine D., Sally J. Stevens, Alison R. Greene, Monica Davis and Rachel Chalot. (2015). Meeting treatment needs: overall effectiveness and critical components of juvenile drug court/reclaiming futures programs. Journal of Juvenile Justice, 4(2), 37-56.

Maltz, Michael, Andrew C. Gordon, David McDowall and Richard McCleary (1980). An artifact in pretest-posttest designs: How it can mistakenly make delinquency programs look effective. Evaluation Review, 4(2): 225-240.

McCollister, Kathryn, Pamela Baumer and the University of Arizona – Southwest Institute for Research on Women (2015). Cross-Site Report: Cost Analysis Overview and Results Narrative: Five Site Report – March 2015. Tucson, AZ: University of Arizona.

Moffitt, Terrie E. (2006). Life-course-persistent versus adolescence-limited antisocial behavior. In Cicchetti, Dante and Donald J. Cohen (Editors), Developmental Psychopathology, Vol 3: Risk, Disorder, and Adaptation (2nd ed.), pp. 570-598. Hoboken, NJ: John Wiley & Sons.

Mutter, Ryan, Mir Ali and Alex Strashny (2015). Profile of Adolescent Discharges from Substance Abuse Treatment. The CBHSQ Report: Treatment Episode Data Set. Washington DC: Substance Abuse and Mental Health Services Administration, U.S. Department of Health and Human Services.

National Institute on Drug Abuse (NIDA) (2014). Principles of Adolescent Substance Use Disorder Treatment: A Research-Based Guide. Washington DC: National Institute on Drug Abuse, National Institutes of Health, U.S. Department of Health and Human Services.

President’s Commission on Law Enforcement and Administration of Justice (1967). Task Force Report: Juvenile Delinquency and Youth Crime. Washington, DC: Government Printing Office.

Rolles, Steve, George Murkin, Martin Powell, Danny Kushlick, and Jane Slater (2012). The Alternative World Drug Report: Counting the Costs of the War on Drugs. Count the Costs Initiative. Bristol, UK: Transform Drug Policy Foundation.

Roman, John K., Jeffrey A. Butts, and Caterina Roman (2011). Evaluating systems change in a juvenile justice reform initiative. Children and Youth Services Review, 33: S41-S53.

Steinberg, Laurence (2009). Adolescent development and juvenile justice. Annual Review of Clinical Psychology 5, 459–485.

Tanner-Smith, Emily E., Sandra Jo Wilson and Mark W. Lipsey (2013). The comparative effectiveness of outpatient treatment for adolescent substance abuse: A meta-analysis. Journal of Substance Abuse Treatment, 44(2), 145-158.

Werb, Daniel, A. Kamarulzaman, M.C. Meacham, C. Rafful, B. Fischer, S.A. Strathdee and E. Wood (2016). The effectiveness of compulsory drug treatment: A systematic review. International Journal of Drug Policy, 28: 1-9.

Whitehead, John T. and Steven P. Lab (2001). Juvenile Justice: An Introduction. Cincinnati, OH: Anderson Publications.

Wiley, Stephanie A. and Finn-Aage Esbensen (2016). The effect of police contact: Does official intervention result in deviance amplification? Crime & Delinquency, 62(3): 283-307.

Young, Douglas W., Richard Dembo and Craig E. Henderson (2007). A national survey of substance abuse treatment for juvenile offenders. Journal of Substance Abuse Treatment, 32(3), 255-266.

Zimring, Franklin E. (2000). The common thread: Diversion in juvenile justice. California Law Review, 88(6): 2479-2495.

_________________________________________________________

APPENDIX: ITEMS IN THE SURVEY

ADMINISTRATION INDICES

Access to Services Index (a = .751)

In the past three months, youth-serving agencies in my community . . .

  • Had problems due to a lack of transportation for youth (reverse-coded)
  • Had problems due to poor location of services (e.g., dangerous areas, inaccessible areas) (reverse-coded)
  • Had problems due to waiting lists for services (reverse-coded)
  • Had problems due to reductions in funding (reverse-coded)

Data Sharing Index (a = .835)

In the past three months . . .

  • Youth-serving agencies in my community found it difficult to share information due to legal issues (reverse-coded)
  • Youth-serving agencies in my community found it difficult to share information due to local policies and regulations (reverse-coded)
  • Youth-serving agencies in my community found it difficult to share information due to state policies and regulations (reverse-coded)
  • Youth-serving agencies in my community found it difficult to share information due to federal policies and regulations (reverse-coded)
  • Youth-serving agencies in my community found it difficult to share information due to technological issues (reverse-coded)

Systems Integration Index (a = .780)

In the past three months . . .

  • Youth-serving agencies in my community worked hard to include community-based organizations in the design and delivery of services for adolescent drug users
  • Youth-serving agencies in my community worked hard to make sure that treatment goals for individual youth were consistent across agencies
  • Youth-serving agencies in my community worked hard to include the schools in the design and delivery of services for adolescent drug users
  • Youth-serving agencies in my community worked hard to include the faith community in the design and delivery of services for adolescent drug users

Resource Management Index (a = .816)

In the past three months, youth-serving agencies in my community worked collaboratively to . . .

  • Share resources such as equipment and materials
  • Identify new resources through grant writing and fund raising
  • Use existing funding more efficiently
  • Share staff or relocate staff positions to serve youth better
  • Cross-train staff from different agencies and systems

COLLABORATION INDICES

Client Information Index (a = .810)

In the past three months . . .

  • Youth-serving agencies in my community were effective at sharing information to improve services for youth
  • Youth-serving agencies in my community generally worked hard to provide other agencies with accurate and reliable information
  • Service providers in my community gave regular feedback about youth to their referral sources and case management agencies
  • Service providers in my community got the type of information they needed to connect with youth and engage them in services or treatment
  • Youth-serving agencies in my community provided regular status updates on client progress (e.g., utilization, compliance, terminations)

Partner Involvement Index (a = .876)

In the past three months, the Reclaiming Futures partnership in my community was effective in . . .

  • Recruiting and/or retaining essential partners, both individuals and agencies
  • Sharing decision-making among various partners
  • Sharing information among various partners
  • Gaining access to key local leaders and decision-makers
  • Obtaining cooperation and support from community-based organizations and other nongovernmental organizations

Agency Collaboration Index (a = .809)

In the past three months, youth-serving agencies in my community . . .

  • Were effective at minimizing agency turf issues
  • Tended to be suspicious of each other (reverse-coded)
  • Tended to share the same priorities in serving youth and families
  • Tended to see each other as dependable
  • Were generally respectful to each other

QUALITY INDICES

AOD Assessment Index (a = .760)

In the past three months. . .

  • The drug and alcohol assessments used in my community provided reliable information
  • The drug and alcohol assessments used in my community helped link youth to services that were matched to their individual needs
  • Youth-serving agencies in my community had problems due to a lack of reliable alcohol and drug assessment information (reverse-coded)
  • Youth-serving agencies in my community routinely used standard protocols or instruments to assess youth for drug and alcohol problems

Treatment Effectiveness Index (a = .824

In the past three months . . .

  • The substance abuse treatment needs of youth in my community were adequately met
  • The mental health needs of youth in my community were adequately met
  • Graduated sanctions were used effectively to support treatment goals for youth
  • Youth-serving agencies in my community generally did a good job serving youth
  • Youth-serving agencies in my community were usually able to provide youth with the range of services they needed

Targeted Treatment Index (a = .827)

In the past three months, youth-serving agencies in my community . . .

  • Had enough access to developmentally appropriate services for youth
  • Had enough access to appropriate services for gay and lesbian youth
  • Had enough access to outpatient substance abuse services for youth
  • Had enough access to intensive outpatient substance abuse services for youth
  • Had enough access to inpatient substance abuse services for youth
  • Had enough access to gender-specific services for youth
  • Had problems due to a lack of accessible mental health services (reverse-coded)

Cultural Integration Index (a = .824)

In the past three months, youth-serving agencies in my community . . .

  • Had problems due to a lack of bilingual staff (reverse-coded)
  • Had problems due to a lack of forms and materials in the primary languages spoken by clients and families (reverse-coded)
  • Had problems due to incompatibility between clients and the religious orientation of service providers (reverse-coded)

Family Involvement Index (a = .844)

In the past three months . . .

  • Family input was used to define service and treatment goals for justice-involved youth
  • Youth-serving agencies in my community did a good job involving family members in delivering drug and alcohol treatment services for adolescents
  • Youth-serving agencies in my community did a good job involving family members in developing overall treatment goals for their children and youth
  • Youth-serving agencies in my community did a good job involving family members in developing treatment service plans for their children and youth

Pro-social Activities Index (a = .754)

In the past three months . . .

  • Youth-serving agencies in my community effectively linked youth to pro-social activities (e.g., recreational and cultural activities)
  • Youth-serving agencies in my community had problems with a lack of pro-social activities for youth (e.g., recreational and cultural activities) (reverse-coded)