FAQs

MFJ Overview

1

What does Measures for Justice do?

Measures for Justice (MFJ) is a nonpartisan nonprofit that measures the performance of the criminal justice system on a county-by-county basis. To do this, we have developed a series of performance Measures that track patterns in how cases move through the system from arrest to post-conviction.

All of our Measures and analyses present data at the county level and are available for free to the public on a web-based Data Portal. The Portal is searchable and can be configured to break down performance data across multiple factors including race/ethnicity, sex, indigent status, age, offense type, offense severity, court type, and attorney type. The Portal also allows for county-to-county comparison within and across states.

2

Who are your funders?

Measures for Justice receives funding from Google.org, MacArthur's Safety and Justice Challenge, the Laura and John Arnold Foundation, Pershing Square, Draper Richards Kaplan, and Open Society Foundations, among others. For more information, please see Partners on our supporters page.

3

Do you advocate specific criminal justice reforms?

Measures for Justice is not an advocacy organization. We collect and display local criminal justice data because we support transparency and dialogue about local practices that are relevant to how the justice system is functioning.

4

Do the Measures address any topics in particular?

Yes. Some of the foremost experts in the field of criminal justice research and measurement have settled on three areas that are most important when assessing system performance: Fair Process, Public Safety, and Fiscal Responsibility. Fair Process Measures assess whether due process is being afforded to defendants, which is a central tenet of the Constitution. Public Safety Measures address how the public is being protected from crime, which is one of the central purposes of any justice system. Fiscal Responsibility Measures address how taxpayer money is being spent, which is especially relevant to the pursuit of an efficient justice system.

5

How did you choose your initial six states?

We chose states that had statewide court databases, that were geographically diverse, and that had a large number of counties.

The Measures

1

What are the Measures?

The Measures are a set of performance indicators designed to assess how the criminal justice system is providing basic legal services in three areas: Fair Process; Public Safety; and Fiscal Responsibility.

2

Who came up with your Measures and how?

MFJ developed an initial set of Measures in 2011 with the help of criminal justice researchers and practitioners, and experts in developing performance measurement for other fields. In 2013, MFJ piloted these Measures using Wisconsin’s administrative court data.

Since then, we have reworked the Measures with two additional advisory groups of researchers and academics, as well as with many practitioners and other stakeholders. For more on our advisory councils, please see our Team page.

For more information on the criteria used to select the final list of Measures, please see our Methodology.

3

How do you get your data?

Our first step is to contact state-level stakeholders to locate and evaluate centralized data resources. This systematic evaluation examines whether statewide datasets exist, what information is captured in these datasets, and whether that information can be extracted in bulk or must be compiled on a case-by-case basis. MFJ’s primary source for data typically comes in the form of a bulk data extraction from statewide court data repositories.

When no statewide datasets are available, or to supplement statewide court data, the MFJ team collects data at the county level from each criminal justice agency (court, largest arresting agency, prosecutor, public defender, sheriff/jail, and state departments of correction).

4

Have any of these Measures been used previously?

Other organizations have similar measures in use. For instance, the Institute for State and Local Governance uses several measures as part of MacArthur’s Safety and Justice Challenge that overlap with MFJ’s. What is unique to MFJ are the scope and breadth of our measurement: collecting data on the entire criminal justice system across the United States and facilitating county-by-county comparison via our Data Portal.

5

Why do some states have more Measures than others?

Our Measures need to be populated with the appropriate data, which are not always available for every state. Some states do not collect the data we need; others collect data but record them in ways MFJ cannot use (e.g., the data are missing key elements). We will continue to work with states to encourage more widespread and comprehensive data collection in the hopes of eventually populating all of our Measures.

6

What is your data management process?

MFJ researchers’ first step is to define a “case” to mean all the charges processed within a given jurisdiction that were associated with the same defendant and filed on the same date. This involves standardizing our data files across states by using a common codebook that identifies the driving characteristics of a case (e.g., case disposition, most serious charge, most severe sentence, etc.). That done, we create a unique case number that is fully de-identified to protect the privacy of defendants, remove personal identifiers from the dataset, and turn over the datasets and calculations to the Technology Team. The Technology Team, in turn, develops algorithms to render the calculations and datasets into the results that appear on the Portal.

For a more detailed explanation of our process, please see Section Three in our Methodology.

7

How do you ensure your data are accurate?

Addressing data quality and uniformity has been a critical MFJ goal from the outset. We have developed a careful and lengthy process to prepare our data, and then run them through internal and external audits to make sure they are valid.

This process involves:

  • Scrutinizing the data to identify elements that may be unreliable.
  • Exploring possible variations between legal and technical language used across and within states.
  • Using text-mining techniques to identify the multiple ways in which the same concept may be recorded using different terms across jurisdictions.
  • Addressing any anomalies, which often reveal that some jurisdictions may not be recording certain events and that, as a result, such events can’t be accurately measured.
  • Approaching relevant stakeholders to inquire about anomalies and to confirm facts.

Per these reviews, MFJ does not use any data elements identified as problematic. For a more detailed explanation of our process, please see Section Four in our Methodology.

8

Do you provide population/demographic information? Is there contextual data?

Yes. We provide contextual Measures for each county including crime rates, demographics, public health indicators, the availability of criminal justice resources, and voting records. We also provide legal context for each state as it relates to criminal justice statutes such as eligibility for indigent defense, pretrial release assessment, use of sentencing guidelines, among other things.

9

How do you compare counties that are so different?

MFJ understands that counties across the country have different cultures and practices that stem from those cultures. Instead of comparing ideologies, we compare how similar cases are handled in different counties.

We have also made it a priority to provide information that encourages responsible comparison. To do this, we standardize all of our data by looking for shared characteristics among cases. In addition, we continually seek input, local context, and information at multiple stages of our work that we pass on to users. Every Measure is accompanied by:

  • Footnotes. The footnotes provide technical information about individual state or county practices that may be germane to the Measure.
  • Legal context. We include relevant statutes (and links) to explain specific differences in state policies that affect criminal justice (e.g., state A uses sentencing guidelines while state B doesn’t, etc.).
  • County Context. County context includes information on demographics, population, crime, criminal justice resources, poverty, commute times, voting records, drug hospitalizations, and citizenship. This allows users to compare counties that are similar in size, demographics, and so on.

10

How do you handle different collection and recording practices across and within states?

We generally gather information about each state’s case management system to familiarize ourselves with the data. When possible, we meet with local and state officials to discuss their data systems and specific guidelines for data collection and storage. We examine how each county names its data elements and groups its data, and use text-mining techniques to identify the multiple ways in which the same concept may be recorded using different terms across jurisdictions. As needed, we walk our source agencies through how we got from local data sets to our finalized data sets to ensure accuracy and to fix any errors that might have occurred along the way.

11

Are counties given the chance to offer their feedback on your Measures?

Yes. We meet state and local criminal justice stakeholders in person or we conduct webinars to explain our work and solicit input. We list some of our contributors on this website.

12

I am not sure what some of the Measures are trying to address. How can I tell?

We understand that not everyone who uses the Portal will be a criminal justice practitioner or criminologist and may need a little more context for why a Measure is important and how it plays into the criminal justice process. To help explain the bigger picture, each Measure includes a short and a long definition. The long definitions provide some additional background on the practices being discussed, and can be accessed by clicking the “information” button next to the Measure title. We want to get a conversation started about what the data show and why.

13

Do you have any law enforcement data?

We are actively collecting law enforcement data and will have them available in the Portal when they are ready. We started with court data because they are the most widely available.

14

Why are some data missing?

Measures are missing data for several reasons:

  1. We have yet to collect all of the data we need.
  2. We are still in the process of working with some of the data we have.
  3. We cannot use the data as provided, so we have excluded the Measure until better data are available.

The Portal presents users with warnings when missing or unavailable data could be problematic, and we always report the percentage of missing values.

15

Will you update the data in the Portal?

Yes, we will populate measures for which we currently do not have data. We will also add new states, and more years of data in the Portal once the data have been collected, processed, and audited.

16

Why do you use five years’ worth of data?

We pool five years of data whenever possible because this allows us to include more counties in the Portal. If we used annual data, some small counties would not have enough data to present. When we are able to collect more years of data in the future, we will be better able to present trends over time.

The Data Portal

1

Why did you decide to build a Data Portal?

Our ambition was to design a way of displaying data that is comprehensive and detailed, but user friendly. We want to give users the opportunity to sift through an enormous amount of data and walk away with information that is actionable.

2

Who has access to the Data Portal?

Everyone. We believe that transparency is critical for positive and productive dialogue among criminal justice stakeholders, citizens, and individuals who encounter the system every day. To this end, we think it is important to make the Portal available without restriction.

3

What are people supposed to do with the data?

MFJ hopes data on the Portal will:

  • give policymakers a new tool to inform their policy decisions, such as where to focus their resources and attention;
  • stimulate dialogue about practices and patterns that are relevant to how the justice system is functioning;
  • help the public understand and engage with the criminal justice system;
  • give the media and academics reliable data to support stories, reports, and research on criminal justice systems across the U.S.

4

Can I download the raw data?

Our data sharing agreements prohibit the release of raw data and the identifying information they contain. But you can download the county level datasets for each state on our resources page.

5

Can you provide a customized report?

At the moment, you can produce individualized reports quickly and with ease at the county level. We are not currently focused on customized reporting, but please contact us with your interest.

6

I think I see a problem in the Data Portal. What should I do?

Please contact us with any issues, questions, or comments via our Contact page.

7

How do I use the Data Portal?

Please see the wizard by clicking the '?' at the top of the Exploration page. Still not sure? Please contact us via our Contact page.

8

How do I reference the MFJ portal and measures?

The suggested citations for the Portal, the Methodology, and the Measures are provided below.

  • Portal. Measures for Justice. (2017). Measures for Justice Data Portal (Data Release: 1.0.0, Portal: 1.0.5). Retrieved from https://measuresforjustice.org/portal/ on date.
  • Methodology. Measures for Justice. (2017). The Measures for Justice Methodology (Version 1.7). Rochester, NY: Measures for Justice Research Department.
  • Measures. Measures for Justice. (2017). The Measures for Justice (Version 1.7). Rochester, NY: Measures for Justice.

9

Do you make sure your data are secure?

Yes. We follow industry standards’ best practices to make sure the data is secure.

Glossary

ABA

American Bar Association

Absolute disparity

Shows the disparity between two groups of defendants using the exact difference between their measure values.

Acquittal

When an accused criminal defendant is found not guilty.

Adjudication

When a court makes an order or judgment in a case.

Arraignment

After an arrest is made, the accused is called in to have their charges read to them. This is also the time in which they enter their plea and bail is set.

Attorney Withdrawal

An attorney terminates her representation of her client, whether voluntarily or under circumstances that require it.

Bail

Money pledged to the court in return for the release of a defendant from custody, with the understanding that the individual will return to court for trial.

Bond

A portion of bail paid by a defendant, or a bail bondsman, or other contractor on his/her behalf. Typically, the defendant must offer a certain percentage of bail in cash and the rest in some form of collateral. The defendant or representative is responsible to pay the remainder of the bail if the defendant fails to appear for his/her court dates.

Bond Motion

A motion to allow a defendant out on bond or reduce the existing amount.

Case

In order to have a standard definition across states, we use a proxy definition of "case" to mean all charges associated with the same defendant and filed (or referred for prosecution, in the case of declinations) on the same date as a single case. We assume that when a prosecutor files multiple charges together, even for different incidents, they intend to resolve these charges at the same time. Since the focus of our Measures is case processing, we believe this approach, while not perfect, is currently the best way to standardize case definition across jurisdictions.

Case Dismissal

The case proceedings are terminated without acquittal or conviction.

Case Disposition

The final settlement or decision determined by the court.

Case Filing

Criminal proceedings are initiated if and when the prosecutor decides to file a case with the court.

Clearance Rate

Number of crimes "cleared" (solved) divided by the total number of crimes recorded.

Conviction

The defendant is declared guilty of a criminal offense.

County

We measure criminal justice performance at the county level because it is usually at this level that charging, disposition, and sentencing decisions are made.

Court-appointed private attorney

In lieu of public defenders, some states may contract with private attorneys to provide counsel for defendants unable to afford an attorney on their own.

Criminogenic

Factors likely to cause or predict criminal behavior.

Declination

A case that was referred for prosecution by law enforcement but was not filed in court by prosecutors.

Decriminalization

The act of reducing and/or completely removing criminal penalties for certain acts; not to be confused with legalization.

Deferred Prosecution

An agreement between the prosecutor and defendant to dismiss a case with concessions made by the defendant. If the defendant breaks the agreement, the prosecutor can refile charges with the defendant's confession.

Discovery

The process through which information and evidence in a case is shared by opposing counsel.

Discretionary Arrest

Discretionary arrest laws are a classification of warrantless arrest laws that do not mandate an arrest, but rather allow individual police officers to make that decision.

Disparity

How often defendants belonging to a group may have been affected by a criminal justice practice compared to defendants from another group.

Dispose/Disposition

An action by a court that closes a case. Examples include conviction, acquittal, dismissal, diversion/deferral.

District Attorney

Represents the government in the prosecution of criminal offenses.

Drug offense

Category of crime that involves the use, possession, distribution, or manufacturing of any illegal drug or drug paraphernalia. The full offense crosswalk is available on the NACJD website.

Due Process

The Fifth and Fourteenth amendments each contain a clause preventing the government from "depriv[ing] any person of life, liberty, or property" without fair and formal treatment under the law; including the laws under which they are prosecuted and the legal proceedings they endure. There should be no arbitrary or unreasonable treatment of a defendant.

Eighth Amendement

"Excessive bail shall not be required, nor excessive fines imposed, nor cruel and unusual punishments inflicted."

Expunction

Sealing, or making unavailable, a criminal record.

Felony

Category of crimes, both violent and nonviolent, that are more serious than misdemeanors and punishable by imprisonment (of one year or more) or death.

Felony Case

A case where the most serious filing charge (or referred charge for a declination) was a felony.

Fifth Amendment

"No person shall be held to answer for a capital, or otherwise infamous crime, unless on a presentment or indictment of a grand jury, except in cases arising in the land or naval forces, or in the militia, when in actual service in time of war or public danger; nor shall any person be subject for the same offense to be twice put in jeopardy of life or limb; nor shall be compelled in any criminal case to be a witness against himself, nor be deprived of life, liberty, or property, without due process of law; nor shall private property be taken for public use, without just compensation."

Fourteenth Amendment

"All persons born or naturalized in the United States and subject to the jurisdiction thereof, are citizens of the United States and of the State wherein they reside. No State shall make or enforce any law which shall abridge the privileges or immunities of citizens of the United States; nor shall any State deprive any person of life, liberty, or property, without due process of law; nor deny to any person within its jurisdiction the equal protection of the laws."

Gross Misdemeanor

A class of misdemeanor that is more serious than a regular misdemeanor but less serious than a felony.

Guilty Plea

A defendant admits guilt to a crime and forfeits his/her right to trial.

Incarceration

A sentence involving prison or jail confinement.

Index crime

The FBI classifies murder and non-negligent manslaughter, forcible rape, robbery, aggravated assault, burglary, larceny, arson, and auto theft as index crimes in its annual crime report because they are considered more serious.

Indigent

A person is of indigent status if he or she does not have sufficient income to afford an attorney as determined by the courts.

Insufficient Cases

There were fewer than 30 cases in the pool from which the Measure was to be derived.

Misdemeanor

Category of crimes, both violent and nonviolent, that are more serious than infractions, but less serious than felonies, and punishable by up to one year in jail, or fines.

Misdemeanor case

A case where the most serious filing charge (or referred charge if the case was declined) was a misdemeanor.

Missing or Unknown

Values that were not provided in the original source data, were deemed unreliable during the coding process, or could not be standardized to the common codebook.

Motion to Compel Discovery

Utilized to settle disputes when one party refuses to turn over evidence to opposing counsel; this motion orders them to do so.

Non-custodial referral

In the situation of an arrest, a notice to appear/citation was given instead of physical detainment.

Nonviolent

The crime did not involve any form of force, injury, or other form of violence. This can include property crimes, public order crimes, etc.

Other attorney

A type of representation that does not fall into the other categories (court-appointed private, private, pro-se, public defender).

Other offense

Category of crime that includes offenses that do not fit into the other offense categories (violent, property, drug, public order). The full offense crosswalk is available on the NACJD website.

Per Capita

Per person

Permanent attorney

Retained or assigned counsel that is expected to remain throughout the case.

Plea Bargain

The defendant agrees to plead guilty, typically to lesser or fewer charges, in return for concessions made by the prosecutor.

Presumptive (citations)

Some states require the issuance of citations (instead of arrest) when certain factors are present, for example, a misdemeanor possession of marijuana.

Presumptive vs. Mandatory

A mandatory sentence is dictated by state law and leaves no room for judicial discretion, while presumptive sentencing sets a baseline to be considered along with other aggravating or mitigating factors of the case.

Pretrial Detention

A defendant is held in custody before a case proceeds to trial or disposition.

Pretrial Diversion

A defendant is steered away from standard criminal justice proceedings, and possible conviction, and into rehabilitative programs instead.

Private attorney

A defense attorney retained at the defendant's own expense.

Pro se

Opting to represent oneself during court proceedings rather than retain an attorney.

Property offense

Category of crime relating to the theft or destruction of another's property. The full offense crosswalk is available on the NACJD website.

Prosecutor

Attorney who initiates and conducts cases against the defendant in criminal proceedings.

Proxy Measure

A variable that is not itself a direct measure of a phenomenon, but operates in place of another, immeasurable variable. It should, however, have a strong correlation, or relationship, with the outcome of interest.

Public Defender

A court-appointed attorney for defendants unable to afford legal counsel on their own.

Public Order offense

Category of crime that does not require or identify any single victim, but interferes with the social norm or is considered harmful to the community. The full offense crosswalk is available on the NACJD website.

Rated jail beds

Maximum number of beds assigned by a rating official to a facility, excluding separate temporary holding areas. (Minton, T.D. & Z. Zeng (2016). Jail Inmates in 2015. Bureau of Justice Statistics, Department of Justice, p.2. Retrieved on December 2016).

Recidivism

If a convicted criminal commits a new offense (often measured by new arrest, new arraignment, or new conviction).

Referred Charge

A charge recommended for prosecution by law enforcement.

Relative disparity

Shows the disparity between two groups of defendants using a ratio between their measure values.

ROR

Released on Recognizance (requiring no payment)

ROR Revocation

If, at any time, the accused violates the conditions of his/her pre-trial release, the judge can revoke the release and place him/her back in custody.

Rural (as per Census)

The US census defines rural areas as places with populations of fewer than 2,500 people.

Sentence credits

Defendants can earn credits, or time, against their sentence for good behavior.

Sixth Amendment

"In all criminal prosecutions, the accused shall enjoy the right to a speedy and public trial, by an impartial jury of the State and district wherein the crime shall have been committed, which district shall have been previously ascertained by law, and to be informed of the nature and cause of the accusation; to be confronted with the witnesses against him; to have compulsory process for obtaining witnesses in his favor, and to have the Assistance of Counsel for his defense."

Split sentence (aka bifurcated sentence, extended supervision)

The defendant serves part of his/her sentence in prison and part on some form of supervised release.

Summary Conviction

Some minor and petty crimes can receive judgement without a jury trial.

Surety

Person who agrees to pay the amount of money specified in a written agreement (surety undertaking) if the accused fails to appear in court. This is sometimes a requirement before a defendant is released on bail.

UCR program (Uniform Crime Reporting)

The FBI created the Uniform Crime Report in order to meet the need for consistent crime statistics across the country. The UCR is collected and published annually, showing data for city, college, county, state, tribal, and federal law enforcement agencies.

Urban (as per Census)

The US census defines urban as places with populations of 2,500 or more.

Valid data

Values that were provided by the original source data and were capable of being standardized to the common codebook.

Violent offense

Category of crime in which force, or the threatened use of force, is used against a victim or their property. The full offense crosswalk is available on the NACJD website.

With or Without Prejudice

When a case is dismissed with prejudice, it is permenantly dismissed and charges cannot be refiled. A case dismissed without prejudice can be refiled at a later date.

Zero Tolerance Policing

Strategy that focuses on the strict, non-discretionary policing of minor crimes with the hope of further reducing more major, violent crimes.