My goal in this paper is to tell you a story, a story about race, about crime, about discretion, and about hope. I want to suggest that mass incarceration in the United States is not necessary or wise. It is the product of a criminal justice system that has ballooned beyond reason or recognition from its design.

Several years ago, I interviewed Steve Cooley, the then conservative Republican district attorney in Los Angeles. He told me a story about a man named Gregory Taylor, whose case came to Cooley’s office in the 1990s, when Cooley was a junior prosecutor. Gregory Taylor was accused of prying open the door of a food pantry at a church at about 5 a.m. because he was hungry. At the time, California had a very harsh three-strikes law on the books. A third offense, even if it was non-violent, could put you in prison for life. Gregory Taylor had two previous offenses on his record: snatching a purse and attempting to steal a wallet. In both cases, no one was harmed. Now one of Cooley’s colleagues needed to decide which charges to bring against Taylor. He decided to prosecute Taylor for a charge that counted as a third strike, Taylor was convicted, and he was sentenced to life in prison.Footnote ¹

When Cooley told me the story, he said that if Taylor’s file had landed on his desk, he would not have made the same decision. The case convinced him that California’s three-strikes law was too harsh. Thus, as the district attorney several years later, Cooley supported a ballot initiative to reduce the scope of the three-strikes law.

I listened to Cooley and thought about how the fact that Gregory Taylor’s case landed on one prosecutor’s desk rather than another’s determined the future course of Taylor’s life. I had never thought about prosecutorial discretion in such terms before.

Once I noticed the discretion of prosecutors, I started to see it everywhere in my reporting on the criminal justice system. This observation led me to write a book about the role prosecutors play in the United States, in which I argue that the increase in prosecutorial discretion since the 1980s has been a major driver of mass incarceration.Footnote ²

Until the late 1970s, the United States had a fairly low incarceration rate, similar to that of Canada and Scandinavian countries.Footnote ³ Since then, the number of people in jail and prison in our country has skyrocketed, while it has remained constant in these other countries.

So, how did this start? In the 1980s in the United States, there was a substantial increase in crime, and that rise provoked a significant amount of fear. Many American politicians capitalized on such fear to win elections by promising to lock up the people who were committing harm. These elected representatives helped pass a wave of sentencing laws in every state, which ratcheted up sentences across the board. They also turned to a particular tool: the mandatory minimum sentence. Although mandatory minimums take the decision about how someone is punished after a conviction out of the hands of judges, they do not eliminate or reduce discretion. Instead, they hand the power from judges to prosecutors, who effectively determine the sentence when they decide which charges to bring and which plea bargain offer to make. Thus, mandatory minimum sentences bake the punishment into the charging decision.

In the initial charging determination, there is often flexibility, as in Gregory Taylor’s case, about whether to prosecute someone for a lesser offense or for the maximum penalty allowed by statute. Prosecutors also often have discretion about whether to stack up multiple charges for the same underlying conduct, which increases the sentencing exposure for a defendant, and thus the pressure to accept a guilty plea rather than go to trial. Since trials are labor-intensive for both prosecutors and judges, the system has evolved to avoid them.Footnote ⁴

John Pfaff, a professor at Fordham law school, conducted a study that provided an empirical foundation for understanding how prosecutors have contributed to increased incarceration by using the tools of mandatory minimum sentences and stacking charges. Pfaff found that in the 1990s and 2000s, when incarceration rose dramatically in the United States, prosecutors transitioned from charging felonies in approximately one-third of cases to about two-thirds of cases.Footnote ⁵

Pfaff’s finding is important because it isolates the role of the prosecutor. The charging decisions of prosecutors are generally independent of the police and arrest rates. These decisions are also largely independent of judges and the rulings that they make in the courtroom.

You see here also a shift in the classic image of American criminal justice. Typically, we imagine a judge at the apex of a triangle, and a defense lawyer and a prosecutor at the bottom two corners, on an even playing field, with the same level of bargaining power. But as a result of the dynamics I have described, in reality, the prosecutor has moved up to the place of the judge, in terms of their real-world decision-making authority, in many criminal cases.

This structure is very much at odds with how our system was designed. Judges are supposed to sit at the top of the triangle because they are neutral decision-makers. Prosecutors in the United States, meanwhile, have two potentially incompatible roles. They are supposed to be ministers of justice, which does give them responsibility for the fairness of the system, but they are also supposed to win convictions. In a world in which the assumption is that locking up people convicted of crimes is the way to keep the public safe, prosecutors have an incentive to seek harsher punishment. This can create a culture within a prosecutor’s office that prioritizes convictions and long sentences over other values, like the overall well-being of communities that are the most impacted, which are often low-income and those of color.

One illustration is a so-called “hammer award” that the district attorney’s office in Memphis, Tennessee, gave to prosecutors who won a guilty plea in a trial or a heavy sentence post-conviction or guilty plea. Such prosecutors received a picture of a hammer posted on their office doors, as a form of recognition.Footnote ⁶ There have been other examples of professional rewards like this in district attorney’s offices all over the U.S. They demonstrate the kind of work that is deemed valuable and that could lead to a promotion. This type of office culture, which became extremely common in the 1990s and 2000s, and continues in many offices, helps explain why prosecutors, working the job every day, have an incentive to increase mass incarceration.

Beginning in the mid-1990s, crime in the U.S. began to decline. Since then, while the rate of violent crime has plunged by more than 50 percent,Footnote ⁷ the same is not true for the incarceration rate. The punishment machine built in the 1980s and 90s has continued to operate at a high velocity, disconnected from the crime rate it was meant to address. If incarceration in the U.S. continues to fall at only the current rate, it will take 75 years, according to the Sentencing Project, to reduce our incarceration rate by half.Footnote ⁸ To achieve Canada’s rate of incarceration, the U.S. would need to cut its incarceration rate by more like 80 or even 85 percent.Footnote ⁹

Police, judges, legislatures, and some voters have also contributed to mass incarceration, but understanding the role of prosecutors is crucial to change. Since voters choose district and state’s attorneys in almost every state, they have the power to use these elections, as a shortcut around the difficult process of changing state sentencing laws, to further and more definitively reduce incarceration.

I also want to address prosecutors and the U.S. system of money bail. Judges set bail, but research shows that the factor that most influences their decision-making is the demands of the prosecutor. Before I conducted the reporting for my book, I thought the reason money bail was used was to decide whether people go to jail before a trial as opposed to being released, to ensure they had some skin in the game in order to return to court. Therefore, you pay money up front because it is necessary to ensure that you show up.

It turns out that is not true. First of all, the United States and the Philippines are the only two countries in the world in which for-profit cash bail is even legal. England, Canada, and many other countries have outlawed cash bail because of its potential for extortion and abuse.Footnote ¹⁰ There are valid concerns about putting private actors—bail bond companies—into a quasi-governmental role. We know that cash bail is not necessary to get people to come back to court inside the United States. The state of Kentucky has been operating without cash bail since 1976 and Washington D.C. effectively banned money bail in 1992. Almost everybody in these regions has come back to court. In general, the way it works in those jurisdictions is that there is a small percentage of defendants who are accused of violent crimes, and judges make individual determinations on whether those defendants could be a real risk to the community or to an individual if they are released. Defendants who are deemed to be a danger represent only a small percentage of those facing charges, usually about 10 percent, and are held in jail in what is termed preventative detention.Footnote ¹¹ Everyone else is released, and of that much larger percentage, almost everyone, nearly 90 percent of people, return to court even though they have put no money down.Footnote ¹²

In short, it turns out that what is actually important for getting people to come back to court is making sure that they know when to come back, by sending text messages to remind them, being clear about the dates, and following up with them. Making people pay to remain free is not necessary for ensuring their return to court.

If bail is not serving the purpose of ensuring court appearances, what is it actually doing? The answer is that bail turns jail into a kind of plea-bargaining mill.Footnote ¹³ Research shows that people are more likely to plead guilty if they are in jail. If you are in jail and worried about whether you are going to lose your apartment, how you are going to pay your rent, or what is going to happen to your job and your family, then you are more likely to sign a piece of paper that is going to get you out of jail and much less likely to think about the long-term consequences of having a criminal record, which you will carry for the rest of your life.

There is another surprising element to bail and pre-trial detention. A study by the Quattrone Center at the University of Pennsylvania studied people who were held in detention in Harris County, Texas, for misdemeanor offenses because they could not make bail of $500 or less. The researchers found that these people were more likely to commit another crime two years later than those who were released because they made bail.Footnote ¹⁴ This seems counterintuitive. The accused are being held in jail, which is supposed to increase public safety. But in fact, if you look two years out, it has the opposite effect. Jail is associated with being what is called criminogenic, meaning that people who go to jail tend to commit more crimes afterward. Again, this actually makes a lot of sense, because of those same kinds of desperation I described earlier; people who are worried about losing their homes, who have been in jail and get out and do not have a place to go, do not have a job, will have more trouble getting hired, and so on. Thus, people in these kinds of circumstances become more likely to commit crimes.

The Quattrone Center found that if the judges had simply let all the misdemeanor defendants who were only being held for $500 or less out of jail, the state could have prevented 4,000 new crimes in Harris County over the relevant period of time. This finding suggests that bail is actually counterproductive as a public safety measure. Indeed, a few D.A.s elected on progressive platforms have stopped asking for bail for most non-violent offenses. This is an important change that prosecutors can make to reduce unnecessary and harmful pre-trial detention.

I also want to discuss prosecutors and the vanishing trial. In the United States, we have the right to a jury trial in criminal cases and certain civil matters enshrined in our federal constitution. This is a point of pride and remains the main feature of any kind of television show or movie and also features prominently in many books about the criminal system. There is a trial in my book. Trials make for great drama! They are supposed to be how we arrive at the truth in the United States’ justice system.

But the fact is that because prosecutors are able to stack charges and threaten mandatory minimum sentences, we have very few trials, statistically speaking, in the United States. Upward of 95 percent of convictions in many states are obtained through plea bargains.Footnote ¹⁵ As I have noted, one problem with this shift is the decision-making authority it hands to prosecutors rather than judges. Another problem has to do with the elevation of secrecy. Plea negotiations take place in private haggling sessions between defense lawyers and prosecutors, not in public view in a courtroom, where we have the benefit of a record and transcript. This means we have lost an important check on the power of the state, and especially the power of the police. When the police do not follow every rule when they make an arrest (e.g., by conducting an illegal stop or unwarranted search), their unconstitutional actions often come to light in suppression hearings and trials in open court. If, however, a defendant is under a great deal of pressure to plead guilty for example, due to the mandatory sentence or stacked charges that he or she faces, the strength and legality of the state’s case will never be tested. The public does not find out that the police broke the law, and this enhances their impunity.

When I talk to groups of prosecutors and judges in the United States, my criticism of the vanishing trial and the domination of plea bargaining is controversial. If 98 percent of convictions are obtained in a certain way, it is hard to imagine doing without this feature. Plea bargaining is the oil that keeps the wheels of American criminal justice turning. Prosecutors and judges tend to say that it is for those reasons we cannot give it up. My response is that I’m not calling for an end to plea bargaining in the U.S. To be sure, there are jurisdictions in Europe in which plea bargaining is not prevalent, but they depart from the way we do justice in the United States in so many ways that they are hard to compare. I do think, however, that the United States could set constitutional limits on plea bargaining that would result in a higher proportion of trials, and thus rein in government power.

Currently, there is no constitutional limit on plea bargaining in the U.S. because of a Supreme Court decision from 1978 called Bordenkircher v. Hayes.Footnote ¹⁶ The facts of the case are relatively straightforward. Paul Hayes was a horse groom in Kentucky with a sick mother and younger siblings under his care. He passed a bad check at a grocery store to buy $85 worth of groceries. As a result, he was arrested, and the prosecutor said that if he pleaded guilty, the prosecutor would ask the judge to give him a five-year prison sentence. Paul Hayes turned down the offer. He said he wanted a trial. The prosecutor said that in that case, he would charge the bad check as Hayes’ third strike. Hayes had a criminal history and if he was convicted, he would go to prison for life. He took the gamble, went to trial, and lost. Like Gregory Taylor, he received a life sentence.

On appeal, Hayes’ lawyers argued to the United States Supreme Court that Hayes’ sentence was too harsh. They said there had to be some constitutional limit on the threat a prosecutor can make when they are trying to convince someone to plead guilty, because of the burden that the threat of a heavy sentence puts on a person’s right to a fair trial. The United States Supreme Court rejected this argument. The justices said Paul Hayes knew what he was doing, and that because there was a sufficiently level playing field between the prosecutor and the defense, there was no need to set a constitutional limit on what prosecutors can threaten in order to induce or coerce someone to plead guilty. This is still the law of the United States. You can face the death penalty if the prosecutor says the charges warrant it, even if the real reason for threatening execution is to produce a guilty plea.

Bordenkircher v. Hayes is a contribution by the Supreme Court to the power of prosecutors. It also helps explain why so many people in the United States have been exonerated after pleading guilty to their crimes. How does that happen? If you did not commit a crime, why would you ever say “I plead guilty” in court? The reason might be that you are facing a heavy sentence and you do not want to bet on the jury believing you. About 18 percent of people who have been found to be wrongfully convicted since the 1980s are people who pled guilty to their crimes.Footnote ¹⁷

The Supreme Court also contributed to the impunity of prosecutors in another decision from 1975, called Imbler v. Pachtman.Footnote ¹⁸ This was a case about prosecutorial misconduct in which the question was whether the wronged defendant could personally sue the prosecutor. The Supreme Court made it very difficult to do this by granting prosecutors what is called absolute immunity. Many government actors in the U.S., including the police, have qualified immunity, which is a barrier to suing them, and is itself controversial. Absolute immunity goes even further by essentially saying that prosecutors cannot be sued for anything they do in the course of their duties of employment. When the Supreme Court gave prosecutors this gift, the justices said that if a prosecutor gravely violates someone’s constitutional rights, someone else would come along and prosecute that prosecutor. But in all my research, I have found only two cases in the modern era in the United States where a prosecutor went to jail, even for a couple of days, for violating someone’s constitutional rights.Footnote ¹⁹ It turns out that prosecutors are not eager to punish other prosecutors. It does not really happen.

The Supreme Court also said in Imbler v. Pachtman that state and local bar associations would address prosecutorial accountability by bringing disciplinary actions against errant prosecutors, which could end in the suspension or removal of their law licenses. But this route to prosecutorial accountability has also proved to be nearly worthless. It is highly unusual for prosecutors to be disciplined by their state bar associations. There is a chapter in my book involving an ethics trial of one of the prosecutors in the case of Noura Jackson, whose story I tell. If you read that chapter, you will understand why lawyers shrink from holding their peers accountable.

Finally, the Supreme Court augmented the power of prosecutors in a case that is often considered a triumph for protecting the rights of defendants, Brady v. Maryland, decided in 1963.Footnote ²⁰ Brady gives defendants the right to see evidence that could help them prove their innocence. This is crucial because the state has the power to investigate, rope off the crime scene, collect evidence, and interview witnesses. If they have evidence that points away from you as a suspect—perhaps there is evidence that incriminates another suspect—you are unlikely to know about it unless they turn it over to you. This is why Brady is important. The problem is that according to the Supreme Court, these disclosure obligations only manifest at trial. Therefore, if you are one of the many people who plead guilty before your case goes to trial, the disclosure obligations set out in Brady are not necessarily helpful.

Several federal appeals courts have extended Brady to the plea-bargaining phase, but not all courts have done this, and those that have, differ on how early the accused can see the evidence. For example, in many states, it is really up to the prosecutor’s office to decide whether to share evidence with the accused early enough to have an effect on their decision about whether to go to trial. The other limitation of Brady is that the prosecutor gets to decide whether the evidence is material, and thus, whether it must be turned over. He or she makes this decision while the defense and the judge do not know what is in the file.

There is an alternative to this process: open file discovery, in which prosecutors open their files to the defense (while retaining the authority to redact information that might be used to threaten or tamper with a witness or victim). Texas and North Carolina have modeled open-file laws and shown they can be successful. An important test for a progressive D.A. is whether they lobby for an open-file law in their state, as well as establishing an open-file policy for their office. In Canada, the government has a broadly similar duty to disclose all relevant information to the defense, subject to the prosecutor’s discretion about timing and protecting the identities of informing witnesses.Footnote ²¹

My research indicates that the United States is the only country in the world that elects prosecutors. In 2015, a movement began to run D.A. candidates who promised to reduce incarceration and increase fairness. The idea was that it could make a big difference to take all the power and discretion of the prosecutor and put it into the hands of someone committed to slowing and even dismantling the criminal punishment machine. Another aspect of the effort to elect a new type of D.A. is to change the demographic composition of elected prosecutors. There are about 2,400 elected prosecutors in the U.S. In 2015, 79 percent were white men.

In 2016, the first year that the national movement to elect progressive prosecutors gained momentum, candidates backed by civil rights groups and local advocates for reform ran for D.A. in about 15 large metropolitan areas, several of whom were women and people of color. A dozen of these candidates won, and their ranks have subsequently grown.

A new study by Sam Krumholz shows the promise of this movement. Using data collected from 1994 to 2014, Krumholz found that when a Republican was elected district attorney in a contested election, the incarceration rate rose significantly (not all of the candidates for D.A. who have run on progressive platforms since 2016 are Democrats, but most of them are). Meanwhile, when a person of color was elected in a contested election, the incarceration rate declined. Both results were independent of the crime rate and the arrest rate.Footnote ²² This study shows that prosecutors can send many more people to prison, tearing the social fabric of the communities they come from, without achieving a decrease in crime or responding to the actions of the police.

The movement to reduce incarceration has increased bipartisan support. A 2017 survey by the American Civil Liberties Union (ACLU) found that 71 percent of Americans believe it is important to reduce the prison population, including 52 percent of Trump voters.Footnote ²³ When voters think about the price of incarceration, they are even less likely to support the sweeping tough-on-crime mantra.

In evaluating the case for transforming American prosecution and ending mass incarceration, it is crucial to recognize that it is not true that the only way to reduce crime is to lock more people up. Why does one poor neighbourhood have a high crime rate, while another with the similar socioeconomic, racial, and ethnic characteristics has a low crime rate? Sociologists have shown that one of the biggest factors is something called social cohesion. The sociologist Robert Sampson sent vans with cameras out into the streets of Chicago to photograph street life, day after day, in a couple of Chicago neighbourhoods that were affected differently by crime. Sampson found that the neighborhood with the low crime rate had many pedestrians out and about, and that people were talking to each other. By giving them questionnaires, he found that they were also looking out for each other, for example, by watching out for each other’s homes and children. People in the lower-crime neighbourhood were more likely to know their neighbours’ names. This was not the case in higher-crime neighbourhoods. The sense of community, of social cohesion, had a significant impact on preventing crime.Footnote ²⁴ The sociologist Patrick Sharkey followed up with a related finding: the addition of community non-profit organizations can cause the homicide rate to drop by 1 percent. It does not have to be a non-profit that works directly on crime prevention. It can be something like mental health treatment or drug treatment or simply health in general. It can be something like turning a vacant lot into a playground. These types of community activities bind people together. This way of preventing crime does not rely on law enforcement and does not produce the associated costs of over-policing and over-incarceration. Yet we spend relatively little money and pay far too little attention to these community-enhancing methods in the United States, as the recent movement to defund the police has exposed.Footnote ²⁵

When I was writing my book, I also wanted to do some storytelling through audio. I made a podcast miniseries that follows a young man named Tarari through his experience when he was 20, when he was charged with gun possession in New York. In New York, if you have no prior criminal record and you are caught with a gun without a license, the maximum charge you can face (i.e., criminal possession of a weapon in the second degree) and that almost everybody faces, because it gives prosecutors greater leverage for plea bargains, carries a three-and-a-half-year mandatory minimum prison sentence. This was the situation Tarari faced when he was arrested, almost certainly illegally (that is another story) at his housing project in Brooklyn for having a gun. Tarari appeared bound for prison even though he had no adult criminal record. But in Brooklyn, alone among the boroughs or counties in New York, there is a narrow escape hatch. If you are under the age of 24, have no criminal record, and get caught with a gun, you can plead guilty and accept a sentence; however, that sentence is effectively suspended while you participate in what is essentially a social work program. It is an unusual effort at diverting people from prison by requiring them to meet with a social worker over the course of a year. The social worker is employed by the D.A.’s office and monitors young people like Tarari and helps them pursue education or employment.

Over the course of his year in the diversion program, Tarari had his ups and downs, but overall, he thrived. For me, his story is about the tremendous, irreplaceable value of a second chance. Almost everyone who goes to prison comes back out, and often they are in a worse position to be a productive citizen, and sometimes they cycle back in. Tarari started community college in Brooklyn in September. He is hoping to become an accountant. More prosecutors need to think systemically about how to produce more outcomes like Tarari’s. Not just for their sake, but for the sake of their communities, and for all of us.