Performance measurement of drug courts: The state of the art, DM Rubio

Tags: performance, drug courts, drug court, NCSC, measurement, performance measurement, performance measures, drug court participants, National Center for State Courts, SPMS, National Research Advisory Committee, Technical Assistance, NRAC, Bureau of Justice Assistance, National Drug Court Training, state courts, Principal Court Management Consultant, Authors Dawn Marie Rubio, Conference of State Court Administrators, Statewide Performance Measurement Systems, Court Executive Development Program, court operations, accountability measures, PMs, adult drug court, performance challenges, Drug Courts National Research Advisory Committee, trial courts, substance abuse treatment, Court Performance Framework, NPMS, Procedural Satisfaction
Content: Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
About the National Drug Court Training and Technical Assistance Initiative: Statewide Technical Assistance Project
Since 2002, the Bureau of Justice Assistance has awarded funds to the National Center for State Courts to provide technical assistance services to state-level agencies such as the Administrative Office of the Courts and the Alcohol and Drug Abuse Agency to: 1. Enhance the leadership of statewide drug court efforts; 2. Improve coordination and collaboration between the drug court agencies; and 3. Increase the likelihood of the institutionalization of drug courts into the mainstream of court operations.
Pursuant to these awards, the National Center for State Courts has provided technical assistance services to states that included: 1. On-site technical assistance; 2. Off-site technical assistance; and 3. A series of topical publications on integrating drug courts into mainstream court operations. This Statewide Technical Assistance Bulletin is the sixth in the series of publications.
Bureau of Justice Assistance Acknowledgement The National Center for State Courts thanks the Bureau of Justice Assistance Office of Justice Programs U.S. Department of Justice for its financial support of the National Drug Court Training and Technical Assistance Initiative: Statewide Technical Assistance. This document was developed under grant numbers 2005-DC-BX-K163 and 2006-DC-BX-K060. The points of view expressed are those of the authors and do not necessarily represent the official position or policies of the Bureau of Justice Assistance or the National Center for State Courts.
About the Authors Dawn Marie Rubio, J.D. -- served as Project Director and is a Principal Court Management Consultant with the NCSC. Ms. Rubio is an attorney whose expertise is in the areas of family law, juvenile dependency and child protection, domestic relations, domestic violence, drug courts, problem-solving courts and court programs. She is the director of large-scale projects improving court organization, process and operations. Currently, she is the director of the Statewide Drug Court Technical Assistance Initiative funded by the Bureau of Justice Assistance. Ms. Rubio is an advisor to the Conference of Chief Justices/Conference of State Court Administrators Committee on ProblemSolving Courts. Ms. Rubio earned a Bachelor of Science in Psychology and Juris Doctor from the University of Florida. She is a fellow of the Court Executive Development Program of the Institute for Court Management and the recipient of the 2006 NCSC Staff Excellence Award.
Fred Cheesman, Ph.D. -- Senior Court Research Associate with the NCSC, is an expert in evaluation methodology, forecasting, and statistical analysis. Major interests include juvenile justice, problem-solving courts, risk assessment, and sentencing. Since joining the NCSC in 1997, major projects include development of performance measurement systems for drug courts, drug court evaluations, evaluations of community courts, evaluation of risk assessment instruments used in sentencing, and investigations of blended sentencing. Prior to joining NCSC, Dr. Cheesman served on the faculty of the University of Baltimore (UB) with a joint appointment in the criminal justice and public policy divisions, also serving as a research associate with the Schaefer Center for Public Policy. He also spent a year as a visiting professor at Indiana University. Prior to this, he served as a researcher and systems analyst for 15 years with the Ohio Department of Youth Services, where he developed population forecasts for the agency and conducted program evaluations.
William Federspiel, J.D. -- attended the William and Mary Marshall-Wythe School of Law School and served as a Jefferson Scholar at the National Center for State Courts during the period he contributed to this document. He has since graduated and is serving as a law clerk for Chief U.S. District Court Judge Louise Flanagan in the Eastern District of North Carolina. Information Design Neal Kauder -- VisualResearch, Inc. Design & Layout Michael Zanconato -- Mazmedia
i
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
Introduction Performance measurement has come to be considered an essential activity in many government and non-profit agencies because it "...has a common sense logic that is irrefutable, namely that agencies have a greater probability of achieving their goals and objectives if they use performance measures to monitor their progress along these lines and then take follow-up actions as necessary to insure success" (Poister, 2003). Effectively designed and implemented performance measurement systems provide tools for managers to exercise and maintain control over their organizations, as well as a mechanism for governing bodies and funding agencies to hold organizations accountable for producing the intended results. States Initiating Drug Court Performance Measures Implemented without NCSC assistance (at least 1 SPMS) NCSC assisted implementation (at least 1 SPMS) No SPMS implemented Did not respond to the survey Impetus for this general movement toward performance measurement comes from both government and stakeholders in organizations subject to performance measurement. As a result of the Government Performance and Results Act of 1993, all federal agencies report performance information as part of the federal budgeting process, an approach that has also been adopted by many state and local governments. Unlike many governmental reforms, performance measurement appears to be here to stay.
The emergence of drug courts as a reform of courts' traditional practice of treating drug-addicted offenders in a strictly criminal fashion coincided with renewed interest in performance measurement for public organizations. The argument for measuring the performance of drug courts is compelling because they are a recent reform that must compete with existing priorities of the judicial system for a limited amount of resources. This makes it incumbent upon drug courts to demonstrate that the limited resources provided to them are used efficiently and that this expenditure of resources produces the desired outcomes in participants. This Statewide Technical Assistance (TA) Bulletin updates the volume published in 2004 that described the methodology used by the National Center for State Courts (NCSC) to develop Statewide Performance Measurement Systems (SPMSs) for the drug courts of several states (Cheesman, Rubio, and Van Duizend, 2004). This earlier Bulletin also provided descriptions of the SPMSs of the states that received NCSC Statewide Technical Assistance through an award funded by the Bureau of Justice Assistance (BJA). In the current Statewide Technical Assistance Bulletin, we: Describe efforts to measure the performance of trial courts and assess their applicability to the measurement of drug court performance; Describe recent innovations in the area of drug court performance measurement, in particular the National Research Advisory Committee (NRAC) recommendations; Describe additional statewide performance measurement systems for drug courts that have been developed through the Statewide Technical Assistance grant from BJA since the last bulletin was published; Describe the results of a survey of state drug court administrators ( or their equivalents) to assess the states' efforts to develop statewide performance measurement systems for their drug courts; Describe some of the results of a recently completed assessment of drug court performance in Wyoming, using the NRAC measures; Present conclusions about the state of performance measurement of drug courts in the U.S., and offer recommendations to advance performance measurement of drug courts.
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
1
Performance Measurement of Trial Cour ts in General Because all drug courts are trial courts, any discussion of performance measurement of drug courts should be informed by the lessons learned from efforts to measure the performance of conventional trial courts. The NCSC has historically provided leadership to the movement to measure trial court performance. NCSC's initial foray into performance measurement for trial courts took the form of the Trial Court Performance Standards (TCPS; BJA, 1997). As described by Schauffler (2007), drawing upon Casey (1998): Over three years, the Commission on Trial Court Performance Standards engaged the court community in the development of 22 standards requiring 68 measures across five broadly defined areas; access to justice: expedition and timeliness; equality, fairness and integrity; independence and accountability; and public trust and confidence. Conceptually, the TCPS were aimed at evaluating the performance of the court as an organization, not the performance of individual judicial officers per se. The point of reference was those who use the court, and the focus was on how to improve services to the public. The TCPS were first published in 1990 and endorsed by all key national court organizations (Conference of Chief Justices, Conference of State Court Administrators, National Association for Court Management and American Judges Association). (p.119). As Schauffler goes onto relate, the movement toward performance measurement in the state courts lost momentum shortly after it received the key endorsements that it required to establish its legitimacy. He identified several factors that contributed to the inability of state courts to institutionalize performance measurement at that time: 1. The number of proposed measures (68) was too great and the measures appeared complex and seemingly without priority; 2. The courts' information systems were not originally designed to produce the data required for the measures, and manual data collection was too labor intensive; 3. The economic pressure on budget resources diminished as the economy improved, removing some of the impetus to spend dollars strategically using performance measurement; 4. The institutional separation of the judiciary from other branches of government which enabled the courts to avoid the tide of performance measurement that was sweeping many executive branch agencies; and 5. A lack of consistent leadership on this issue.
At the beginning of the second millennium, the conjunction of several forces led to revived interest in performance measurement of courts. First, the souring national economy increased pressure on courts to spend limited budgetary resources wisely. Secondly, the perception within the court community that judicial branch institutions had not been as successful as other public sector organizations in advocating for budget resources grew. Finally, throughout the nineties and to this day the general movement toward performance measurement in the public and non-profit sectors grew in strength as did the state-of-the-art of performance measurement methodology. In particular, the emergence of a new approach to performance measurement, the "balanced scorecard" method (Kaplan and Norton, 1992), influenced the development of performance measures. Originally designed for the business sector, this model was based on the premise that corporations need to look beyond such traditional measures as return on investment, profit and loss, and cash flow so as to get a more balanced picture of performance. The balanced scorecard incorporates four perspectives: The customer perspective, the internal business perspective, the innovation and learning perspective, and the financial perspective. Corporate entities establish goals in each of these domains and then define measures to track their performance against these goals. In response to these forces and to advances in the state-ofthe-art of performance measurement, NCSC revisited the TCPS in a series of national meetings (Ostrom and Hall, 2005). Feedback from these meetings reinforced the notion that the TCPS had incorporated too many measures to be practical. However, TCPS did identify important measurement domains for courts which lent themselves to a simpler, more refined reformulation of the TCPS using the balanced scorecard approach. What emerged were CourTools, a set of ten performance measures designed to evaluate a small set of key functions of the court. Three criteria were used to select potential measures for inclusion into CourTools (Ostrom, 2005): 1. Correspondence to fundamental court values; 2. Balanced perspective on the work of the court; and 3. Feasibility and sustainability.
2
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
Ten CourTools Measures
Measure 1: Access and Fairness Ratings of court users on the court's accessibility, and its treatment of customers in terms of fairness, equality, and respect.
Measure 7: Collection of Monetary Penalties Payments collected and distributed within established timelines, expressed as a percentage of total monetary penalties ordered in specific cases.
Measure 2: Clearance Rates The number of outgoing cases as a percentage of the number of incoming cases. Measure 3: Time to Disposition The percentage of cases disposed within the established time frames. Measure 4: Age of Active Pending Caseload The age of active cases pending before courts, measured as the number of days from filing until the time of measurement. Measure 5: Trial Date Certainty The number of times cases disposed by trial are scheduled for trial.
Measure 8: Effective Use of Jurors Measurement of juror yield (i.e., the number of citizens who report for jury duty as a percentage of those summoned) and juror utilization (i.e., the number of prospective jurors actually used as a percentage of those who reported for jury duty). Measure 9: Employee Satisfaction Ratings of court employees assessing the quality of the work environment and relations between staff and management. Measure 10: Cost per Case The average cost of processing a single case, by case type.1 1 Full definitions and detailed descriptions of these measures are available from the National Center for State Courts at www.courtools.org.
Measure 6: Reliability and Integrity of Case files The percentage of files that can be retrieved within established time standards and that meet established standards for completeness and accuracy of contents.
Giving Courts the Tools to Measure Success
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
3
Refreshingly new is the fact that states and individual courts are actually implementing performance measures. The state of Utah has begun to implement the CourTools measures statewide, proceeding measure by measure. The results of those measurements are published on the state courts's public Web site at www.utcourts.gov/courtools and data for most measures (e.g., clearance rate) are available at the aggregate statewide level as well as at the local jurisdiction level. The state of California is currently pilot testing all ten CourTools measures in four courts, with the intention of building reporting capacity on most of the measures into the new statewide California Case Management System. The state of Arizona has major work underway in its largest superior and municipal courts pilot testing many of the CourTools measures. The Yuma County Superior Court has published results at www.co.yuma.az.us/courts/dashboard.htm while the Maricopa County Superior Court has an internal dashboard and reports its performance results in its annual report at www.superiorcourt.maricopa.gov. Individual courts are also taking up performance measurement. These range from large urban courts like Harris County, Texas, (which includes the city of Houston) and Hennepin County, Minnesota (which includes the city of Minneapolis) to small courts of one to six judges in rural areas. Results of the Fourth District Court in Hennepin County are available at www.mncourts.gov/district/4. Two small rural courts have posted the results of their first round of performance measurement, along with management recommendations for actions to take based on the results (for Lubbock, Texas, see www.co.lubbock.tx.us and for Morrow County, Ohio, see morrowcountycpc.com). The NCSC also seeks to disseminate all such reports on the online community pages of the CourTools Web site at www.courtools.org.
The differences between drug courts and conventional trial courts preclude the direct application of CourTools to drug courts. For example, whereas reducing time to disposition is an admirable goal for conventional courts, the nature of addiction and the realities of substance abuse treatment require extended times to disposition for drug court participants, typically more than a year in a pre-plea drug court. However, NCSC's experience measuring the performance of trial courts in general provides several important lessons that are relevant to an undertaking to measure the performance of drug courts. First of all, the TCPS and CourTools demonstrated that courts of all sizes share common performance challenges and that national-level performance measures can be designed to assist them to address these challenges. CourTools demonstrate the feasibility of a national level approach to solving common and shared performance problems among similar courts. Second, CourTools in particular demonstrated that a "balanced" approach to developing Performance Measures (PMs) is needed to provide a comprehensive picture of trial court performance. It is necessary to use PMs from several different, critical measurement domains. Third, the number of PMs should be small but targeted at critical functions of the court. By keeping the number of measures small, implementation will be more likely and easier, allowing drug courts to focus their generally limited resources on the most useful measures. Fourth, the manner of presentation of the PMs will influence their acceptance and use. Great care was taken to present CourTools in a user-friendly fashion, including abundant use of graphics and computational examples. All of the CourTools are well documented and leave little room for equivocation regarding their measurement. Fifth, courts will use PMs, if lessons three and four above are heeded. The failure of the TCPS and the increasing use of CourTools support this assertion.
4
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
Performance Measurement of Drug Courts National Research Advisory Committee (NRAC) Recommendations The most important development in performance measurement of drug courts since the last Statewide TA Bulletin on Drug Court Performance Measures was written in October 2004 was the development and promulgation of the first set of nationallyrecommended performance measures for Adult Drug Courts. These measures were developed by a leading group of scholars and researchers assembled by the National Drug Court Institute (NDCI), with funding from BJA, that became known as the National Research Advisory Committee (NRAC). This committee met on three separate occasions in the fall of 2004 to create and develop a uniform research plan for drug court data collection and analysis. Dr. Cary Heck, who chaired NRAC, authored a monograph ("Local Drug Court Research: Navigating Performance Measures and Process Evaluations") that summarized the work of NRAC and was published in June 2006. NRAC recommended that adult drug courts adopt four measures of performance: 1. Retention; 2. Sobriety; 3. In-program Recidivism; and 4. Units of Service. Retention is necessary to keep drug court participants in treatment long enough to realize an effect. Research indicates that three months of drug treatment may be the minimal threshold for detecting dose-response effects, six to 12 months may be threshold for clinically meaningful reductions in drug use, and that 12 months of drug treatment appears to be the "median point" on the dose-response curve: i.e., approximately 50% of clients who complete 12 months or more of drug abuse treatment remain abstinent for an additional year following completion of treatment (Marlowe, DeMatteo, and Festinger, 2003). Longer retention not only indicates success in treatment but also predicts future success in the form of lower posttreatment drug use and re-offending (Cissner and Rempel, 2005).
Retention was measured using admissions cohorts as the sampling frame. Overall program retention was the percentage of a particular admissions cohort2 that exited the drug court program, broken down by the type of exit (e.g., graduation, termination, voluntary withdrawal, or death). Sobriety, both during and after drug court participation, is a goal of all drug courts because it fosters offender rehabilitation, public safety, and offender accountability. Two indicators of participant sobriety, both measured during the course of participation, were recommended: (1) average length of continuous sobriety and (2) the average number of failed tests. As the participant proceeds through the program, a trend of decreasing frequency of failed tests should occur. Research has shown that increasing amounts of time between relapses is associated with continued reductions in use. Both the trends and the average of these measures should be useful performance measures. Drug courts are expected to produce low rates of in-program recidivism among drug court participants in comparison to other more traditional interventions for drug offenders such as probation or community-based treatment. The combination of judicial supervision, treatment, and rewards and sanctions that uniquely characterize drug courts are expected to lower recidivism, a finding supported by research (US Government Accountability Office, 2005). Recidivism was defined as the rate at which drug court participants are rearrested during the course of their participation. NRAC also recommended that drug courts measure post-exit recidivism but provide little guidance as to how this should be done. Though recommending arrests as the primary measure of recidivism, NRAC also suggested collecting data on convictions. Treatment services must be delivered in sufficient dosage to drug court participants to be effective (National Institute of Justice, 2006). Units of service are measures of dosage that "can be loosely defined as a measure of those drug court activities that address the needs of drug court clients including, but not limited to, substance abuse treatment" (Heck, 2006). Service units should be based on actual attendance of a drug court participant in one of the recommended or mandated activities. Units of service for outpatient services are measured by counting sessions or episodes. For inpatient services, units of service are measured by the number of days the service was provided.
2 An admissions cohort is a group of individuals who enter a program during a specified time period. Individual courts can define this time period though it is generally defined as a six-month or one-year period.
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
5
Though the NRAC measures were intended to bring some uniformity and standardization to drug court research, their applicability to the ongoing measurement of the performance of drug courts is obvious. For this reason, NCSC actively promoted the incorporation of the NRAC measures in the SPMS that were developed since 2006. The original set of NRAC measures was limited, by design, to a relatively small set of critical measures. However, as we describe in the next section, every state that NCSC has assisted to develop an SPMS since the introduction of the NRAC measures has chosen to add additional measures that examine drug court performance in areas unexamined by NRAC. NCSC Statewide Technical Assistance to Develop SPMSs As described in the first Statewide TA Bulletin on this subject, the Bureau of Justice Assistance (BJA) provided support to NCSC to provide technical assistance to state-level agencies to enhance their drug court programs statewide. This technical assistance has been documented in a series of TA Bulletins published by NCSC and range from strategic planning to needs assessment. Several states enlisted NCSC to assist them with the development of an SPMS. The methodology employed by NCSC to develop SPMSs was described in the earlier TA Bulletin. The only changes in the methodology that have occurred since then are the incorporation of the NRAC measures and the fact that our efforts have been better informed by the experiences of states that have adopted SPMSs. NCSC philosophy for the development of SPMSs is guided by a few important principles. First, we aim for a small number of measures targeting the most critical of drug court processes. Second, performance measures (PMs) are developed from the "bottom-up"- stakeholders tell us what should be measured and how it should be measured. NCSC acts an informed facilitator, offering suggestions and making recommendations for PMs, but the ultimate decision is made by the advisory committee convened by the state-level agency responsible for drug courts. Third, PMs are well-documented. Detailed "specification" sheets are written for each PM, documenting data sources, calculations, and interpretation, and leaving little equivocation about the gritty details of the PM.
At the time of the publication of the first Statewide TA Bulletin, NCSC had assisted four states (Tennessee, Missouri, Vermont, and Wyoming) to develop SPMSs, all of which were developed without the benefit of the NRAC recommendations. Since then NCSC has assisted another eight states to develop or enhance SPMSs3, for a total of 11 states and the Commonwealth of Puerto Rico that have received Statewide TA from NCSC to develop or enhance SPMSs. Table 1 shows the states and the measures that each has chosen to incorporate in their SPMS. It can be seen that the PMs naturally fall into several measurement domains: NRAC Core Measures NRAC Recommended Measures Accountability Social Functioning Processing Interaction with Other Agencies Cost and Cost Avoidance Compliance with quality standards In the following, we briefly discuss the PMs contained in these domains. 3 Hawaii received technical assistance from NCSC through a contract and not through funding from BJA. This state developed an SPMS independently, using the methodology described in Cheesman, Rubio, and Van Duizend (2004). Later, NCSC provided assistance enhancing Hawaii's SPMS.
6
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
Table 1: Adult Drug Cour t Per formance Measurement Systems Developed with BJA Technical Assistance
Performance Measure AR FL HI KY MO NV PA PR TN VT WY
NRAC Core Measures
Retention Rate
In-Program Recidivism


Sobriety--% Positive Drug Tests

Sobriety--% Longest Continuous


Units of Service


NRAC Recommended Measures
Post-Exit Recidivism
Time-in-Program


Accountability
Hours of Community Service Performed


Financial Obligations--Amount Collected

Financial Obligations--Compliance

Financial Obligations--Child Support


Social Functioning
Change in Driver's License Status



Change in Driver's License Readiness (DUI Courts)

Change in Educational Status


% Earning GED or HS Diploma


% Pursuing Post-secondary education

% Completing or Actively Pursuing Education or Vocational Training

Change in Vocational Status

Days Employed While Participating
Employment Status Two Years After Exit
Change in Housing Status



Change in Living Situation

Births of Drug-Free Babies



Change in Family Functioning
Child Custody Status

Child Visitation Status

Contact with Family

Processing
% Referrals Admitted
% Referrals Found Appropriate for Drug Court

% Appropriate Referrals Admitted

Timeliness--Days between Arrest and Admission
Timeliness--Days Between Referral and Eligibility Assessment

Timeliness--Days Between Eligibility Assessment and Staffing

Timeliness--Days Between Staffing and First court appearance

Timeliness--Days Between Eligibility Assessment and Admission

Timeliness--Days Between Referral and Admission

Timeliness--Days Between Admission and Treatment Entry


Timeliness--Days Between Treatment Referral and Treatment Entry
Number of Days Continuously Monitored (DUI Courts Only)
Number of Drug Tests Administered
% Suspected Positive Drug Tests
Number of Alcohol Tests Administered
% Positive Alcohol Tests
% Suspected Positive Alcohol Tests
Number of Sanctions Imposed



Time Between Precipitating Event and Sanction
Number of Incentives Granted


Number of Judicial Status Hearings


Number of Drug Court Case Manager/Probation Officer Contacts per Participant

Number of Activities Planned per Drug Court Coordinator
Number of External Contacts per Drug Court Coordinator
Number of Significant Others Served

Number of Program Violations

Number of Times Admitted to Jail and/or Prison (pre-, during participation, post-)

Amount of Time in Jail and/or Prison (pre-, during participation, post-)

Graduation Rate


Reason for Termination

Access/Fairness

Interaction With Other Agencies

Cost and Cost Avoidance
Compliance with Quality Standards
Performance Measure AR FL HI KY MO NV PA PR TN VT WY
Number of drug tests per person per month per phase Aspirational Recommended as a "best practice" but not required
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
7
NRAC Core and Recommended Measures While the NRAC measures were well-described in Heck's monograph, NCSC had to work out the operational details on many of these measures, as will be related. Every state chose to include a measure of retention in their SPMS. Tennessee and Vermont, having developed their SPMS before the advent of the NRAC measures, used well-known formulae to calculate retention and graduation rates:
Every state except Missouri incorporated a measure of sobriety in their SPMS. In every instance, sobriety was measured during the course of participation and not post-exit, given the difficulties of measuring the latter. All of the states measuring sobriety used "percent of failed tests" as an indicator. All of the post-NRAC SPMSs also incorporated "period of longest continuous sobriety" as an indicator, excepting Puerto Rico's measures. NCSC discovered that measuring the latter variable can be complex when analyzing SPMS data from
=( + )ч( ) Retention % Rate
Total number of graduates since program's
To t a l number currently
Total number of admissions to program since
inception
enrolled
program's inception
=( )ч( + ) Graduation % Rate
Total number of graduates since program's
To t a l number of
Total number of terminations, both measured since
inception
graduates
program's inception
The remaining states measured retention by tracking admissions cohorts until every member of the cohort had exited in some fashion, as recommended by NRAC. The only other variation among the states that chose to measure retention in this fashion was the length of time that was used to define an admissions cohort (usually all admissions to a drug court program during a six-month period) and the manner in which they delineated types of exits from drug courts. For example, some states included voluntary withdrawals as an exit type while others included this category with terminations. Other than retention, NCSC recommends using "exit" cohorts4 to measure all of the other PMs. While there would be advantages to using admissions cohorts to measure everything, as a practical matter, this approach would require drug courts to track admissions cohort members for extended periods of time in order to measure an array of variables and it would require a long wait-period before complete results would be available. Using exit cohorts to measure the PMs requires that the results be disaggregated by type of exit, (e.g., graduations and terminations) so as to make the results interpretable. For example, we might reasonably expect the post-exit recidivism rates of graduates and terminations to differ. 4 An exit cohort is a group of individuals who exit a program during a specified time period. Individual courts can define this time period though it is generally defined as a 6-month or one year period. Shorter time periods for exit cohorts such as three-month exit cohorts provide feed back in a more timely fashion but are also more resource intensive.
Wyoming where we found that participants on bench warrant status for extended periods of time could skew this calculation. These individuals often remained on the rolls of a given drug court but since they were on the run and not being tested, yielding periods of longest continuous sobriety that were artifactually long. It is thus necessary to disallow time on bench warrant status from any calculations of period of longest continuous sobriety. Hawaii included other PMs that measured sobriety, in particular the results of alcohol testing. Vermont and Missouri chose not to measure in-program recidivism. Only new offenses (i.e., offenses that occurred after admission to the drug court program) were counted by every state that measured in-program recidivism. There was variation among the states regarding how this construct was measured, as can be seen in Table 2. Note that most states used convictions as their measure of in-program recidivism. Kentucky disaggregated in-program recidivism by the phase of the drug court program during which the recidivism occurred. Units of service were measured by all of the post-NRAC states but none of the pre-NRAC states. Typically, a distinction was made between addiction-related and ancillary services, the latter being non-addiction related services that addressed participants' other criminogenic needs (e.g., education or employment). Several states, such as Wyoming and Pennsylvania, were able to incorporate pre-existing systems for counting
8
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
Table 2: Measurement of Adult Drug Cour t In-Program Recidivism by State
State Arkansas Florida Hawaii Kentucky Nevada Pennsylvania Puerto Rico Tennessee Wyoming
Method of Measuring
In-Program Recidivism
Arrest Charge Conviction










Types of Offense Felony or DWI offense Felony and misdemeanor Felony and misdemeanor Jailable offenses, higher felonies, and misdemeanors; excluding violations Felony, gross misdemeanor, misdemeanor, and DUI; excluding traffic offenses Felony, misdemeanor, DUI; excluding traffic offenses Felony and misdemeanor, administrative traffic offenses excluded Felony and misdemeanor Felony and misdemeanor
units of service into their SPMS. NCSC also discovered that many states could not report units of service for ancillary services because they often referred participants to these services but did not keep data regarding their participation. Consequently, most states chose to count referrals rather than actual number of units of service received for ancillary services. Every state except Wyoming decided to include at least one measure of post-exit recidivism in their SPMS, as shown in Table 3. Most states chose to measure recidivism by convictions; only Tennessee used arrests while Missouri and Vermont used charges. Most states counted both felonies and misdemeanors but excluded traffic offenses other than DWI. Only Missouri and Arkansas counted felonies exclusively, though Arkansas also counted DWIs. Kentucky also added the requirement that the offense be "jailable." Tracking periods ranged from one to five years.
Some states chose to disaggregate post-exit recidivism according to other criteria as well as type of exit. Tennessee disaggregated by whether the post-exit offense was a felony or misdemeanor. Arkansas disaggregated post-exit recidivism by whether the conviction for the post-exit offense resulted in placement in (1) Arkansas Department of Corrections, (2) Arkansas Department of Community Corrections, or (3) some other disposition. Puerto Rico disaggregated by type of offense (Drug/DUI or Non-Drug). Pennsylvania disaggregated by the age of the participant (18-25 or over 25 years of age) and type of offense (Drug/DUI or Non-Drug), though their measure is currently aspirational and has not been implemented. Some states also measured time-in-program, measured from admission to exit. States including time-in-program in their SPMS were Tennessee, Hawaii, Puerto Rico, Pennsylvania, and Kentucky.
Table 3: Measurement of Adult Drug Cour t Post-Exit Recidivism by State
State Arkansas Florida Hawaii Kentucky Missouri Nevada Pennsylvania Puerto Rico Tennessee Wyoming
Method of Measuring
Post-Exit Recidivism
Arrest Charge Conviction












Types of Offense
Tracking Period (in Years)
Felony or DWI offense
1&2
Felony and misdemeanor
1&2
Felony and misdemeanor
3
Jailable offenses, higher felonies, and misdemeanors; excluding violations
1, 2 & 3
Felony
2&5
Felony, gross misdemeanor, misdemeanor, and DUI; excluding traffic offenses 1 & 2
Felony, misdemeanor, DUI; excluding traffic offenses
2
Felony and misdemeanor, administrative traffic offenses excluded
1, 2, 3, 4 & 5
Felony and misdemeanor
1&2
Felony and misdemeanor
2
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
9
Accountability Measures Most states incorporated at least one measure of participant accountability in their SPMS, excepting Wyoming, Puerto Rico, and Florida. Most states used the total amount of financial obligations collected as a performance measure in this category. Two states (Hawaii and Arkansas) measured whether participants had been compliant with their financial obligations while participating. States varied with regards to what was included among "financial obligations." For example, some included child support payments among financial obligations while others did not. Several states also opted to include hours of community service performed among the performance measures in this category. Social Functioning Florida and Wyoming were the only two states that did not include at least one social functioning PM in their SPMS.5 All of the remaining states included a PM that measured change in vocational status. Some chose to measure simply whether the participant became employed during participation while others went beyond this to assess changes in the quality of employment. Most states also attempted to measure educational gains made by participants during the course of participation. Several states measured improvement in driver's license status and several change in housing status. Three states measured birth of drug-free babies (Tennessee, Missouri, and Kentucky). Three states (Tennessee, Nevada, and Kentucky) measured changes in family functioning, including child custody, child visitation, and contact with family. 5 For the most part, the SPMSs for Florida and Wyoming were restricted to the NRAC measures, though Florida's SPMS included a few additional measures.
Processing States measured a wide array of processing PMs, including measures of timeliness of processing and drug court operations. Both Hawaii and Puerto Rico incorporated PMs that measured the referral process. Most states measured timeliness of processing in some fashion, though there was much variation in the time intervals that were measured. The most commonly measured time interval was from admission to treatment entry. Hawaii had the most comprehensive set of measures of timeliness, measuring the intervals from referral to eligibility determination, eligibility determination to staffing, and staffing to first court appearance. Most of the post-NRAC states measured incentives granted and sanctions administered as well as the number of status hearings attended. Hawaii and Puerto Rico included PMs that measured drug court coordinator activities. Tennessee included additional recidivism PMs that measured jail and prison time. Exclusively, Kentucky's PMs included a measure of Access/Fairness that compares characteristics (demographic, offense-related) of referrals to drug court with the characteristics of admissions and of admissions with exits. Interaction with Other Agencies Tennessee was the only state that chose to include a measure of the amount of interaction between drug courts and other agencies. This simple measure only accumulated contacts but it does reflect compliance with Key Component #1, "drug courts integrate alcohol and other drug treatment services with justice system case processing" (The National Association of Drug Court Professionals Drug Court Standards Committee, 1997).
10
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
Table 4: Measurement of Juvenile Drug Cour t Recidivism by State
Definition of Recidivism
In-Program and Post-Exit Recidivism 1) Felony drug law violations resulting in adjudication 2) Felony non-drug law violations resulting in adjudication 3) Misdemeanor drug law violations resulting in adjudication 4) Misdemeanor non-drug law violations resulting in adjudication 5) Status offenses resulting in adjudication
Post-Exit Recidivism:
Participant Age 17 or Younger: Referrals for delinquent conduct substantiated through informal adjustment or sustained formally Participant Age Greater than 17: Findings of guilt for felonies, misdemeanors, birth of drug-exposed babies within two years of graduation
In-Program
A new petition or criminal complaint is filed between admission and exit, excluding filings for traffic offenses other than DUI.
Post-Program
A new petition or criminal complaint is filed between admission
A new petition or criminal complaint is filed after exit,
excluding filings for traffic offenses other than DUI while the juvenile is under 18.
For juveniles that age out of the juvenile system within two years of program exit and who did not
recidivate post-program as juveniles, the adult definition of post-program recidivism applies.
State Tracking Period Post-Exit Recidivism Hawaii 3 Years Missouri 6, 12 & 18 Months Nevada 1 & 2 Years
Cost and Cost Avoidance Although many states have acknowledged the potential value of a PM of this type, only Missouri chose to include such a measure explicitly in their PM system although it was eventually tabled because of the difficulty of measurement. Compliance with Quality Standards All three of the pre-NRAC states incorporated a PM that required an assessment of drug courts with respect to their level of compliance with the 10 key components of adult drug courts (The National Association of Drug Court Professionals Drug Court Standards Committee, 1997). Essentially, this measure requires an audit of drug court operations to assess level of compliance with the key components.
Juvenile Drug Court Measures NCSC assisted three states (Missouri, Hawaii, and Nevada) to develop PMs for their juvenile drug courts. Naturally, the definition of recidivism will differ somewhat for juveniles compared to adults. Table 4 shows how the three states defined juvenile recidivism. All states attempting to measure juvenile recidivism face a challenge due to the bifurcated nature of tracking juvenile recidivism. If a juvenile remains younger than the age of majority during the post-exit tracking period, recidivism data for both in-program and post-exit recidivism should be collected from juvenile justice data sources. If, on the other hand, a juvenile reaches the age of majority during participation, both juvenile and adult criminal justice data sources should be consulted for any instances of in-program recidivism but only adult criminal justice data sources need be consulted for any instances of post-exit recidivism. Once a juvenile reaches the age of majority, the definitions of adult recidivism for the respective states apply. Finally, if the juvenile reaches the age of majority during the tracking period, only juvenile justice data sources should be consulted for any instances of in-program recidivism but both juvenile and adult criminal justice data sources should be consulted for any instances of post-exit recidivism.
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
11
Table 5: Measurement of Family Drug Cour t Recidivism by State
Definition of Recidivism Tracking Period Post-Exit Recidivism
In-Program
1) Subsequent (to initial removal) removals of children that occurred while the participant was under drug court supervision 2) Drug or alcohol offenses that occurred while under drug court supervision that ultimately resulted in a conviction 3) Family Offenses that occurred while the participant was under drug court supervision that ultimately resulted in a conviction 4) Birth of a drug-positive baby while under drug court supervision if the baby was also conceived while the participant was under drug court supervision
Hawaii 3 Years
Post-Exit Recidivism:
1) Subsequent (to initial removal) removals of children that occurred post-exit 2) Felony drug or alcohol offenses that occurred post-exit that ultimately resulted in a conviction drug-exposed babies within two years of graduation 3) Misdemeanor drug or alcohol offenses that occurred post-exit that ultimately resulted in a conviction
Post-Exit Recidivism
1) Substantiated hotline report on drug court participant/parent 2) Sustained allegation of abuse or neglect 3) Birth of drug-exposed babies 4) Findings of guilt for drug-related offenses or offenses against the family or alcohol/drug-related offenses
Missouri 6, 12 & 18 Months
In-Program
A substantiated allegation of child abuse and/or neglect or the birth of a drug-positive baby (only if the birth occurred during participation)
Nevada 1 & 2 Years
Post-Exit Recidivism:
A substantiated allegation of child abuse and/or neglect or the birth of a drug-positive baby (only if the birth occurred during participation)
Post-Exit Recidivism
1) Substantiated report of abuse or neglect 2) Petitions filed in Family Court
Vermont 1 & 2 Years
None of the measures of juvenile recidivism were based on arrests. Rather, they were based on referrals to juvenile court substantiated through informal adjustment or sustained formally in Missouri or adjudications for law violations in Hawaii, both of which correspond to convictions in an adult criminal court. In Nevada, recidivism was based on the filing of a new petition or criminal complaint, which would correspond to the filing of a charge in the adult criminal justice system.
Other than recidivism, many of the PMs selected for adult drug courts are more or less applicable to juvenile drug courts, with some exceptions particularly in the accountability, social functioning, and compliance with quality standards categories. In the case of the latter, compliance with the "16 Strategies for Juvenile Drug Courts" (BJA, 2003), as opposed to the 10 Key components for adult drug courts is desired. The juvenile accountability measures include the number and duration of "Alternative Care Placements." Social functioning measures include educational gains for juveniles, as was the case for adults, but these are often expressed in terms of grade-level advancement.
12
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
Family Drug Court Measures NCSC assisted four states (Missouri, Vermont, Hawaii, and Nevada) to develop PMs for their family drug courts. Naturally, the definition of recidivism will differ from that used with juvenile and adult drug courts, as can be seen in Table 5. In addition to convictions for new offenses, recidivism measures also include substantiated hotline reports, substantiated allegations of child abuse/neglect, and birth of drug-exposed babies. "Safety and Permanency" measures are appropriate for Family Drug Courts. Four states (Missouri, Vermont, Hawaii, and Nevada) chose to incorporate such measures for their family drug courts. The measures included by Missouri and Vermont are: 1. Filings for Termination of Parental Rights (TPR); 2. Establishment of paternity and support; 3. Percentage of children who are transferred among one, two, three, or more placements while under court jurisdiction; 4. Percentage of children who reach legal permanency (by reunification, guardianship, adoption, planned permanent living arrangement, or other legal categories that correspond to ASFA)6 within 6, 12, 18, and 24 months from removal; 5. Percentage of children who re-enter foster care pursuant to court order within 12 and 24 months of being returned to their families; 6. Percentage of children who do not have a subsequent petition of maltreatment filed during program participation; and 7. Percentage of children who are the subject of additional substantiated findings of maltreatment within 12 months of graduation. Hawaii measured: 1. Time from removal to family supervision; 2. Time from removal to reunification or alternative permanency decision; 3. Time from admission to reunification or alternative permanency decision; and 4. Percentage of children who achieve reunification or alternative permanency decision.
Nevada measured the change in the number of children reunified with parent. This PM was the percentage of children that were removed from the home at the time the participant was admitted to drug court that were re-united with their parent by the time of exit from drug court. Other than recidivism and safety and permanency measures, many of the PMs selected for adult drug courts are more or less applicable to family drug courts, with some exceptions particularly in the accountability category. In that category, Missouri and Vermont measured the number of alternative care placements (for children of the participant), measuring the number of times this occurred and the number of days the children spent in alternative care placements (e.g., foster care). They also both measured the public assistance status of participants. Domestic Violence Drug Court Measures NCSC assisted Hawaii to develop PMs for their domestic violence drug court. In­program recidivism was measured by the occurrence of: New Temporary Restraining Order (TRO) or TRO violation Drug or alcohol offenses that occurred while the participant was under drug court supervision that ultimately resulted in a conviction Family offenses that occurred while the participant was under drug court supervision that ultimately resulted in a conviction Post-exit recidivism was measured by the occurrence of: New Temporary Restraining Order (TRO) or TRO violation Felony drug or alcohol offenses that occurred post-exit that ultimately resulted in a conviction Family offenses that occurred post-exit that ultimately resulted in a conviction
6 Adoption and Safe Families Act of 1997.
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
13
National Survey of SPMSs While NCSC is intimately familiar with the SPMSs of states that received Statewide TA from NCSC for their development, other states have developed SPMSs independently. NCSC developed a survey to gather some fundamental information about these SPMSs. The survey was administered to all statewide drug court coordinators present at their semiannual meeting in Burlington VT during September of 2007, inquiring about their use of SPMSs and e-mailed to all other statewide drug court coordinators (or other individuals identified as serving in this function) who were not present at this meeting. The survey consisted of seven questions designed to determine whether a state uses a statewide performance measurement system (SPMS) for any of its drug courts. Four different types of drug courts were included in the questionnaire: adult, juvenile, family and DUI. It also inquired whether a state's SPMS included the NRAC measures and whether the state provided training and support to SPMS users. The survey also sought to determine how the states are using SPMS data through a series of questions dealing with various reporting procedures. Use of SPMSs among the States Forty-five states completed the SPMS surveys. Twenty-six of the 45 states (58%) that responded to the survey reported using at least one type of SPMS (adult, juvenile, family and/or DWI). As Figure 1 indicates, half of the states (25) reported adopting an SPMS for their adult drug courts, while only 20 and 15 states reported adopting an SPMS for their juvenile and family drug courts, respectively. Eleven states reported adopting an SPMS for their DUI courts.
Figure 1 also shows that several states that do not currently have an SPMS reported plans to adopt one. More states (9) reported that they planned to adopt an SPMS for their juvenile drug courts than for any other type of court. Seven states reported plans to adopt an adult drug court SPMS, seven reported plans to adopt an SPMS for their family drug courts, and six for their DUI drug courts. Alaska reported the development of the earliest SPMS in 1999. Most states reported adopting their adult drug court SPMS within the last four years: four states adopted their adult drug court SPMS in 2004; four states in 2005; two states in 2006; six states in 2007; and two states in 2008. Only five states (Alaska, Idaho, Maine, New Hampshire, and Oklahoma) reported adopting their adult drug court SPMS earlier than 2004. This trend of recent SPMS adoption was generally replicated across the other drug court types as well.
Figure 1: SPMS Measures by Cour t Type $PVSU5ZQF
"EVMU


+VWFOJMF


'BNJMZ

%6*


/VNCFSPG4UBUFT States with SPMS States Planning s SPMS
14
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
The majority of states with an SPMS in place use at least some of the NRAC performance measures. Sixteen states report using all of the NRAC measures and another three states report partially adopting them so that approximately 73% of states using an SPMS use the NRAC measures. Still, seven states with an SPMS do not use the NRAC measures and another 19 do not have an SPMS at all, so that the measures, or some portion of them, are used in more than 40% of the responding states. Fourteen states7 reported that they provide some training and support to users of their SPMS, while eight states reported that they did not. Consequently, a little more than half of states with an SPMS provide training to users and stakeholders. Overwhelmingly, the states that reported having an SPMS reported that they provide PM data to a central office. Twenty-one of the twenty-six states using an SPMS for some of their drug courts report the data they collect to a central agency.8 Most states report data to the State Court Administrator's Office (9). Other agencies to which states reported data included: agencies overseeing drug/therapeutic courts (3); state health or substance abuse offices (3); a statewide drug court database (1); criminal justice agencies (3); and the state supreme courts (2). States varied on the frequency of reporting performance measure data to their respective state agencies. As shown in Figure 2, most states reported data either annually (7) or quarterly (6). About 15% (4) of the states using an SPMS report data monthly. Two states report data on a daily basis and two other states report on a semi-annual basis. 7 Four states that adopted an SPMS at the end of 2007 or in 2008 (Nevada, Arkansas, Pennsylvania, and Kentucky) had not yet worked out the details of implementation, including those related to training and support and are not included in these statistics. 8 Four states that adopted an SPMS at the end of 2007 or in 2008 (Nevada, Arkansas, Pennsylvania, and Kentucky) had not yet worked out the details of implementation, including those related to frequency and means of reporting PM data and are not included in these statistics. Only one state using an SPMS responded that they did not report PM data to a central agency, Alabama.
Figure 2: Frequency of Reporting SPMS Data to State Agency
"OOVBMMZ

4FNJ"OOVBMMZ

2VBSUFSMZ

.POUIMZ

%BJMZ

/VNCFSPG4UBUFT
States also varied in terms of how the drug courts reported their SPMS data. An equal number of states (8) reported using a web system, a statewide MIS system, and local databases. Five states also reported using paper reporting.9 In terms of using PM data, most states (14) with an SPMS issue a report based on the data they collect. Another two states plan to issue reports using the SPMS data in the future. However, the kinds of reports issued vary widely from state-to-state. Most states seem to produce formal reports of some kind on a regular basis. These states use the reports to fulfill statutory mandates and to otherwise educate legislators and other stakeholders. Some states use the data to assess system needs and to improve the programs. On the other hand, a handful of states only produce reports at the request of stakeholders and do not seem to have a formal use for the reports. 9 Several states reported multiple using methods of reporting their data to a central office.
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
15
Performance Measures in Action: The Wyoming Evaluation In 2007, NCSC completed an analysis of performance measurement data collected by the state of Wyoming (Rubio, Cheesman, Maggard, Durkin, and Kauder, 2007). To the best of our knowledge, this analysis was the first to be conducted of the NRAC measures. Data on all of the NRAC measures for every person that participated in drug court between January 1, 2005 and June 30, 2005 were collected. The analysis produced a number of interesting findings that led to recommendations about program structure and operation. This study demonstrates how performance measurement data, when carefully analyzed, can produce information that can be used to improve drug court performance. Retention Two aspects of retention were measured, both based on the cohort admitted to the drug courts from January 1 to June 30, 2005 to ensure uniform measurement of performance: (1) Graduation and termination rates and (2) the Length-of-Stay (LOS) in program for graduates and terminations. High graduation rates (60%+) and low termination rates are desired. Approximately half (48%) of the adult admissions cohort have graduated while another 36% have been terminated or have withdrawn or absconded, leaving about 16% still active. For juveniles, the percentages are 45%, 25%, and 30% respectively. Adult Drug Court graduates spent an average 402 days in the program while terminations spent 38% less time in the program (249 days). Juvenile Drug Court graduates spent an average 359 days in the program while terminations spent 28% less time in the program (260 days).
Sobriety Two aspects of sobriety were examined, both measured for all drug court participants during the period January 1, 2005-June 30, 2005 who have since exited drug courts: (1) Percentage of drug tests failed and (2) period of longest continuous sobriety. Research has shown that increasing amounts of time between relapses is associated with continued reductions in use. The average number of days of continuous sobriety was 318 and 220 days for adults and juveniles, respectively, and in both cases, graduates experienced longer periods of continuous sobriety than terminations. In addition, the average percentage of positive drug tests was 2.1% and 8.2% for adults and juveniles, respectively. In both cases, graduates reported a smaller percentage of positive drug tests than terminations. In-Program Recidivism In-program arrests were reported. The in-program recidivism rate for adult participants that graduated was 14% while that for terminations was 28%. Juvenile rates were higher, 28% and 51%, respectively. Note that the in-program recidivism rates for adult terminations and juvenile graduates are identical. Across the board, juvenile and adult, as well as graduate and termination, the re-arrest offense was almost certainly a misdemeanor. However, though the majority of adult graduates rearrested while participating were rearrested for misdemeanors, they were twice as likely to be rearrested for a felony as any other group (39%). Both adult graduates and terminations were most likely to be rearrested for a person-related offense while participating, while juvenile graduates and terminations were overwhelmingly rearrested for drug use/possession.
16
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
Units of Service Units of service were measured for two types of services: treatment (i.e., addiction-related services including inpatient days and outpatient sessions) and ancillary which are non-addiction-related services that address other "criminogenic" needs of drug court participants (e.g., vocational training and medical treatment). The number of referrals for ancillary services was reported. Programs were instructed to report units of service according to the conventions used by drug court coordinators in their quarterly reports to the Department of Health/Substance Abuse Division (DOH/SAD). Approximately 37% and 22% of adult and juvenile participants, respectively, received at least one inpatient treatment session. On average, juvenile and adult participants receiving inpatient services received 43 and 32 days, respectively. Around 80% of both adult and juvenile participants received out-patient services (slightly higher for juveniles). On average, juvenile and adult participants receiving out-patient services received 91 and 68 sessions, respectively. About 72% and 87% of juvenile and adult drug court participants, respectively, received at least one referral for ancillary services. On average, juvenile and adult participants receiving referrals for ancillary services received six and five referrals, respectively. Though not a unit of service per se, the number of status hearings attended is an important measure of judicial supervision. The average number of status hearings attended by juvenile and adult participants was 32 and 27, respectively.
Based on these and other results, NCSC generated a number of recommendations for Wyoming including the following programmatic recommendations: Employment at admission for adult participants is associated with graduation as opposed to termination, a smaller percentage of positive drug tests, and increased time in program. This suggests that drug courts should strive to address participant needs in this area. Attainment of a high school degree or GED is an important predictor of graduation as opposed to termination for adult participants. This suggests that drug courts should strive to address participant needs in this area. The number of out-patient treatment sessions is associated with increased odds of graduation for juveniles, increased time in-program for adults and juveniles, and a smaller percentage of positive drug tests for juveniles. These findings reinforce the importance of outpatient treatment for participant adjustment. Increasing number of in-patient days are generally associated with negative outcomes (decreased odds of graduation and shortened time of continuous sobriety for adults) but this likely reflects the nature of the participants referred to inpatient care, who probably suffer from the most severe substance abuse problems. Whites are more likely than nonwhites to have a smaller percentage of positive drug tests for juveniles and reduced odds of in-program recidivism for juveniles. These racial differences suggest the need for additional resources for non-white participants.
Adult and juvenile participants that had more arrests during the year prior to their participation had higher percentages of positive drug tests than offenders with lower numbers of arrests. Such offenders should be identified early and supervised accordingly.
Adult participants that abuse methamphetamine, cocaine, crack, prescription drugs, or heroin, or that were referred for a DUI offense are significant risks for being rearrested in program. Such offenders should be identified early and supervised accordingly.
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
17
Performance Measurement of Drug Courts: The Way Forward To advance the state-of-the-art of performance measurement of drug courts, we propose a three-pronged strategy: 1. To develop a core set of performance measures for drug courts that will be implemented nationwide (i.e., a National Performance Measurement System or NPMS). 2. To develop an NPMS that is: Uniform and standardized Comprehensive, including measures from a variety of domains that measure critical aspects of drug court performance Provides a balanced view of drug court performance 3. Establish appropriate performance goals, targets, and standards for this national performance measurement system. First, only about half of the states have adopted an SPMS, and this percentage should be 100%. The previous Statewide TA Bulletin on the subject of SPMSs described the advantages that an SPMS provides to policy-makers and drug courts. Briefly, these include: Removing guesswork regarding what type of data drug courts should be collecting and how it should be measured Providing drug courts with a set of critical indicators (PMs) and standards for those indicators that can be used to improve the performance of the drug court. PMs function as a "dashboard," detecting performance problems in a timely manner and providing drug courts with information that can be used to formulate a response to those performance problems. Uniformity and standardization of data permit comparisons of performance across drug courts A good SPMS provides useful information to policy-makers about drug court performance An NPMS would accrue these same advantages but on a national scale, permitting comparisons of performance across states. NCSC's performance measurement work with drug courts revealed that states with widely varying characteristics were selecting their PMs from the same general set of measurement domains, and from these domains selecting similar sets of indicators. This led NCSC to the conclusion that the strategy of developing SPMSs on a state-by-state basis should be abandoned in favor of the development of a uniform, national set of drug court performance measures,
built on the foundation of the NRAC measures but also designed to assess other critical dimensions of drug court performance. Such an effort should follow the CourTools precedent, emphasizing manner of presentation and graphic displays, being well documented and providing examples. Secondly, an NPMS for drug courts should possess several important characteristics. By design, an NPMS will provide uniform and standardized measurement of key aspects of drug court performance. This will permit comparisons of drug court performance across jurisdictions and over time. An NPMS should be comprehensive and include measures from a variety of relevant domains. NCSC's work assisting states to develop SPMSs revealed that states have chosen measures from the following measurement domains: NRAC Core and Recommended Measures Accountability Social Functioning Processing Interaction with Other Agencies Cost and Cost Avoidance Compliance with Quality Standards These performance measurement domains were identified inductively and reflect what drug court stakeholders feel are important to measure. Consequently, they can provide useful guidance for the identification of performance measures that could be included in an NPMS. In addition to inductively identifying PMs, recent developments in the measurement of court performance make possible a deductive approach to the identification of PMs that should be included in an NPMS. The deductive approach can identify useful measures that are not currently included in any SPMS, as well as lending balance to the measures selected to be part of the NPMS. The "Court Performance Framework" developed by NCSC (Clarke , Schauffler, Ostrom, Ostrom, and Hanson, 2008) to provide a theoretical framework for the development of performance measures for all types of courts can provide the basis for this deductive strategy. The "Court Performance Framework" or (CPF) organizes court performance along two dimensions. The first dimension differentiates courts' need for flexibility, discretion, and responsiveness from their need for stability, order, and control under the appropriate circumstances. Well-functioning
18
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
courts seek to assess their potential to manage flexibly and adjust judge and staff resources to ensure appropriate individual attention to cases and to better respond to evolving customer needs. Concurrently, however, courts also want to ensure that appropriate controls are in place to achieve stable, predictable, and timely case processing. The continuum ranges from organizational nimbleness at one extreme to organizational steadiness at the opposite extreme. A second dimension differentiates courts' attention to their internal environment from their attention to the external environment. A court should develop performance measures that monitor both process and results. The continuum ranges from information produced for external audiences to describe results (e.g., the accountability measures shown in Table 1) to information that can be used to monitor internal processes, much like the dashboard of an automobile, regardless of the results (e.g., retention or the processing measures such as measures of timeliness). Together, these two dimensions jointly form four quadrants, each representing a distinct set of performance measures. Figure 3 shows an application of the CPF to drug courts in particular. Note that the performance measurement domains identified inductively are mapped into their appropriate quadrant.
Procedural Satisfaction Quadrant This quadrant focuses on how services are provided. Here the values are respectful, courteous and empathetic treatment of drug court participants. There is a close relationship between participants' perceptions of how they were treated in drug court and their overall evaluation of service quality. Procedural Satisfaction includes the participants' perception of "procedural fairness" (Tyler, 2003). No states currently possess such a PM in their SPMS, though some drug courts do administer surveys to participants exiting from their drug court programs. Effectiveness Quadrant This quadrant focuses on the extent to which stated goals are achieved. The measures assess the degree of compliance of an outcome with a predetermined set of criteria. As such, they emphasize the ability of a court to control its processes within established limits and reduce variation around the target outcome (e.g., recidivism). This quadrant includes almost all drug court PMs, including measures of recidivism, sobriety, social functioning, accountability, access and fairness, timeliness, and compliance with quality standards. All of these PMs share in common that they possess performance goals that drug courts seek to achieve.
Figure 3: Drug Court Specif ic Version of Performance Mapping
Effectiveness Specific or intended outcomes or results Recidivism Measures Sobriety Social Functioning Measures Accountability Timeliness Access and Fairness
External
Procedural Satisfaction Customer level of satisfaction with court interaction No Performance Measures in this quadrant
Control Ef f iciency Relationship between court performance and resources expended Retention Measures Time in Program Referral & Admission Process Drug Court Coordinator Activities
Internal
Flexibility Productivity Ratio of value added time to cycle time Cost and Cost Avoidance Number by Type of Exit: Units of Service Sanctions and Incentives Judicial Status Hearings Drug Tests Administered Case Manager/PO contacts
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
19
Efficiency Quadrant This quadrant includes measures that examine the relationship between court performance and resources (including manpower, time, dollars, and treatment resources) expended. These measures assist in managing in the most cost efficient manner and define "what resources are required to achieve what we do". Efficiency measures inform judgments about how well resources are used to achieve intended aims--the question of "bang for the buck"--by comparing input indicators with output indicators. These indicators are also designed to assess and minimize variability in key processes. They help ensure stability in processes. There are several examples of efficiency-related PMs for drug courts that relate some form of input (e.g., drug court admissions) to an output measure, in particular type of exit (including graduation, termination and withdrawal) currently in use by an SPMS. Retention compares the number of participants that enter a program (input) with the number that exit, by type of exit (output). Other PMs in this quadrant include time in program, processing measures that scrutinize the referral and admission processes (i.e., percentage of referrals admitted, percentage of appropriate referrals admitted, and percentage of referrals found appropriate for drug court), processing measures that examine drug court coordinator activities (i.e., number of activities planned per drug court coordinator and number of external contacts per drug court coordinator) and number of significant others served. Productivity Quadrant This quadrant assesses the degree to which the internal processes add value. PMs from this quadrant assess value-added time, the amount of drug court team work time consumed in a process, as contrasted with the total amount of time required to complete a process. This information can be very helpful when one is attempting to reduce the total amount of time required to complete a process. Several processing indicators can be classified into this category, including number of judicial status hearings, number of drug court case manager/probation officer contacts per participant, number of drug and/or alcohol tests administered, number of sanctions imposed, number of incentives granted, units of service, and cost/cost avoidance measures, in particular as they relate to the type of exit taken by participants. For example, the average number of status hearings attended, incentives granted, and sanctions administered per successful graduate are productivity measures.
In short, the high performance court framework shows that existing SPMSs are currently unbalanced with respect to the four quadrants shown in Figure 3, being particularly overloaded with Effectiveness indicators. A balanced approach to drug court performance measurement will require the development of measures in these domains. Once the major measurement domains have been identified, performance indicators from these domains must be carefully selected. Poister (2003) lists several criteria for identifying useful performance indicators: Valid and reliable Meaningful and understandable Balanced and comprehensive Clear regarding preferred direction of movement Timely and actionable Resistant to goal displacement Cost-sensitive (nonredundant) Thirdly, while progress has been made in identifying key PMs for drug courts, the next challenge is to establish appropriate performance goals, targets, and standards for these PMs. What is an acceptable post-exit recidivism rate for drug court graduates? What is an acceptable termination rate? What is the most effective amount of time in program? How many units of addiction-related services are required to ensure success? Currently our state of knowledge about such standards is very limited but should be informed by ongoing research on drug court effectiveness and efficiency.
20
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
References Bureau of Justice Assistance. (1997). Trial Court Performance Standards With Commentary. Washington, DC: U.S. Department of Justice. Bureau of Justice Assistance. (2003). Juvenile Drug Courts: Strategies in Practice. Washington, DC: U.S. Department of Justice. Casey, P. (1998). Defining optimal court performance: The Trial Court Performance Standards. Court Review, 35, 24-29. Cheesman, F., Rubio, D., and Van Duizend, R. (October 2004). Developing Statewide Performance Measures for Drug Courts. Statewide Technical Assistance Bulletin, 2. Denver, Co: National Center for State Courts. Cissner, A. and Rempel, M. (2005). The state of drug court research: moving beyond `do they work?' New York: Center for Court Innovation. Clarke, T., Schauffler, R., Ostrom, B., Ostrom, C., and Hanson, R. (2008). A unifying framework for court performance measurement. Williamsburg, VA.: National Center for State Courts. Heck, Cary. (2006). Local Drug Court Research: Navigating Performance Measures and Process Evaluations. Washington, DC: Bureau of Justice Assistance, U.S. Department of Justice. Kaplan, R. and Norton, D. (1992). The balanced scorecard: Measures that drive performance. Harvard Business Review, 70(1), 79-80. Marlowe, D., DeMatteo, D., & Festinger, D. (2003). A sober assessment of drug courts. Federal Sentencing Reporter, 16, 153-157.
National Association of Drug Court Professionals Drug Court Standards Committee. (1997). Defining drug courts: The key components. [NCJ 205621]. Washington, DC. National Institute of Justice. ( June 2006). Drug courts: The second decade. [NCJ 211081]. Washington, DC. Ostrom, B. (2005). CourTools: A Court Performance Framework. Williamsburg, VA.: National Center for State Courts. Ostrom, B. and Hall, D. (2005). CourTools. Williamsburg, VA.: National Center for State Courts. Poister, T. (2003). Measuring Performance in Public and Nonprofit Organizations. San Francisco: Jossey-Bass. Rubio, D., Cheesman, F., Maggard, S., Durkin, M., and Kauder, N. (2007). Wyoming Drug Court Performance Measures Project: Final Report. Denver, CO.: National Center for State Courts. Schauffler, R. (2007). Judicial accountability in US state courts: Measuring court performance. Utrecht Law Review, 3(1), 112-128. Tyler , T. (2003). Procedural justice, legitimacy, and the effective rule of law. In M. Tonry (ed.), Crime and Justice, 30, 431-505. U.S. government Accountability Office. (2005). Adult drug courts: Evidence indicates recidivism reductions and mixed results for other outcomes [No. GAO-05-219]. Washington, DC.
Statewide Technic al A ssis tance Bulle tin | Volume 6 | July 20 08
21
Statewide Technical Assistance Bulletin | Volume 6 | July 2008

DM Rubio

File: performance-measurement-of-drug-courts-the-state-of-the-art.pdf
Author: DM Rubio
Published: Thu Aug 28 14:51:28 2008
Pages: 24
File size: 1.13 Mb


The secret power of music, 11 pages, 0.31 Mb

, pages, 0 Mb

, pages, 0 Mb
Copyright © 2018 doc.uments.com