Measures for Justice (MFJ) is a nonprofit organization that measures the performance of the criminal justice system on a county-by-county basis. To do this, we have developed a series of performance Measures that track patterns in how cases move through the system from arrest to post-conviction.
All our Measures and analyses present data at the county level and are available for free to the public on a web-based Data Portal. The Portal is searchable and can be configured to break down performance data across multiple factors including race/ethnicity; sex; indigent status; age; offense type; offense severity; and attorney type. The Portal also allows for county-to-county comparison within and across states.
Measures for Justice receives funding from the Department of Justice, Google.org, and the Arnold, Donner, Ford, MacArthur, Open Society, and Pershing Square Foundations, among others. For more information, please see Partners on our supporters page.
Measures for Justice is not an advocacy organization. We collect and display data on local criminal justice because we support transparency and dialogue about practices and patterns that are relevant to how the justice system is functioning.
Yes. Some of the foremost experts in the field of criminal justice research and measurement have settled on three areas that are most important when thinking through how to assess system performance: Fair Process, Public Safety, Fiscal Responsibility. Fair Process Measures get at whether due process is being afforded to defendants, which is a central tenant of the Constitution. Public Safety Measures address how the public is being protected from crime, which is one of the central purposes of any justice system. Fiscal Responsibility Measures address how taxpayer money is being spent, which is especially relevant to the pursuit of an efficient justice system.
We chose states that had statewide court databases, that were geographically diverse, and that each had a large number of counties.
The Measures are a set of performance indicators designed to assess how the criminal justice system is providing basic legal services in three areas: Fair Process; Public Safety; and Fiscal Responsibility.
MFJ developed an initial set of Measures in 2011 with the help of criminal justice practitioners, researchers, and experts in the development of performance measurement systems in other fields. In 2013, MFJ piloted these Measures using Wisconsin’s administrative court data.
Since then, we have reworked the Measures with two additional advisory groups of researchers and academics, as well as with many practitioners and other stakeholders. For more on our advisory councils, please see our Team page.
For more information on the criteria used to select the final list of Measures, please see our Methodology.
Our first step is to contact state-level stakeholders to locate and evaluate centralized data resources. This systematic evaluation examines whether statewide datasets exist, what information is captured in these datasets, and whether that information can be extracted in bulk or must be compiled on a case-by-case basis. MFJ’s first and primary source for data typically comes in the form of a bulk data extraction from statewide court data repositories.
When no statewide datasets are available or to supplement statewide court data, the MFJ team collects data at the county level from each criminal justice agency (court, largest arresting agency, prosecutor, public defender, and sheriff/jail).
No. We acknowledge that administrative data have limitations, but we are not suggesting that the data we collect are inappropriate for use. Most of the datasets we acquire are generally reliable. When we find issues in the data, they usually affect only parts of cases and almost never invalidate the whole dataset. To deal with these issues, we implement correctives such as coding problematic values as missing and excluding unreliable data elements. We have developed a careful and lengthy data quality control process through internal and external audits to ensure data are accurate and reliable. For a more detailed explanation of our process, please see Section Four in our Methodology.
Yes, other organizations have similar measures in use. For instance, the Institute for State and Local Governance uses several measures as part of MacArthur’s Safety and Justice Challenge that overlap with MFJ’s. What is unique to MFJ are the scope and breadth of our measurement: We collect data on the entire criminal justice system across the United States and facilitate county-by-county comparison via our Data Portal.
Our Measures need to be populated with the appropriate data, which are not always available for every state. Some states do not collect the data we need; others collect data but record them in ways MFJ cannot use (e.g., the data are missing key elements). We will continue to work with states to encourage more widespread and comprehensive data collection in the hopes of eventually populating all of our Measures.
MFJ researchers’ first step is to define a “case” to mean all the charges processed within a given jurisdiction that were associated with the same defendant and filed on the same date. This involves standardizing our data files across states by using a common codebook that identifies the driving characteristics of a case (e.g., case disposition, most serious charge, most severe sentence, etc.). That done, we remove personal identifiers from the dataset, create a unique case number that is fully de-identified to protect the privacy of defendants, and turn over the datasets and calculations to the Technology Team. The Technology Team, in turn, develops algorithms to render the calculations and datasets into the results that appear on the Portal.
For a more detailed explanation of our process, please see Section Three in our Methodology.
Addressing data quality and uniformity has been a critical MFJ goal from the outset. We have developed a careful and lengthy process to prepare our data in house and then run them through internal and external audits to make sure they are correct.
This process involves:
Per these reviews, MFJ does not use any data elements identified as problematic. For a more detailed explanation of our process, please see Section Four in our Methodology.
Yes. We provide contextual Measures for each county including crime rates, demographics, public health indicators, the availability of certain resources, and voting records. We also provide legal context for each state as it relates to criminal justice statutes such as eligibility criteria for indigent defense, pretrial release assessments, use of sentencing guidelines, among other things.
MFJ understands that every county is different, and that counties across the country have different cultures and practices that stem from those cultures. This is why we do not compare county characteristics or ideologies. Instead, we compare how similar cases are handled in different counties.
We have also made it a priority to provide information that encourages responsible comparison. To do this, we standardize all our data by looking for shared characteristics among cases, which ensures we are, in fact, comparing apples to apples. In addition, we continually seek input, local context, and information at multiple stages of our work that we pass on to users. Every Measure is accompanied by:
We generally gather information about each state’s case management system to familiarize ourselves with local practices. When possible, we meet with local and state officials to discuss their data systems and specific guidelines for data collection and storage. We examine how each county names its data elements and groups its data, and use text-mining techniques to identify the multiple ways in which the same concept may be recorded using different terms across jurisdictions. As needed, we walk our data sources through how we got from local data sets to our finalized data sets to ensure accuracy and to fix any errors that might have occurred along the way.
Yes. We meet state and local criminal justice stakeholders in person or we conduct webinars to explain our work and solicit input.
We understand that not everyone who uses the Portal will be a criminal justice practitioner or criminologist and may need a little more context for why a Measure is important and how it plays into the criminal justice process. To help explain the bigger picture, each Measure includes a short and a long definition. The long definitions provide some additional background on the practices being discussed, and can be accessed by clicking the “information” button next to the Measure title. We want to get a conversation started about what the data show and why; that conversation begins with you.
We started with court data because they are the most widely available. But we are actively collecting law enforcement data and will have them available in the Portal when they are ready.
We are missing data for some Measures for several reasons:
The Portal presents users with warnings when missing or unavailable data could be problematic, and we always report the percentage of missing values.
Yes. We will populate measures for which we don’t currently have data, add new states, and more years of data in the Portal once the data have been collected, processed, and audited.
We pool five years of data whenever possible because this allows us to include more counties in the Portal. If we used annual data, some small counties would not have enough data to present. When we are able to collect more years of data in the future, we will be better able to present trends over time.
Our ambition was to design a way of displaying data that is comprehensive and detailed, but user friendly. We want to give users the opportunity to sift through an enormous amount of data and walk away with information that is actionable.
Everyone. We believe that transparency is critical for positive and productive dialogue among criminal justice stakeholders, citizens, and individuals who encounter the system every day. To this end, we think it’s important to make our data available without restriction.
MFJ hopes data on the Portal will:
Our data sharing agreements prohibit the release of raw data and the identifying information they contain. But you can download and print out the results of any county data explorations you pursue on our platform.
At the moment, you can produce individualized reports quickly and with ease at the county level. We are currently not focused on customized reporting, but please contact us with your interest.
Please contact us with any issues, questions, or comments via our Contact page.
Please see the wizard by clicking the '?' at the top of the Exploration page. Still not sure? Please contact us via our Contact page.
Yes. We follow industry standards’ best practices to make sure the data is secure.