In 2007, I became the attorney general of the state of New Jersey. Before that, I'd been a criminal prosecutor, first in the Manhattan district attorney's office, and then at the United States Department of Justice. But when I became the attorney general, two things happened that changed
the way I see criminal justice. The first is that I asked what I thought were really basic questions. I wanted to understand who we were arresting, who we were charging, and who we were putting in our nation's jails and prisons. I also wanted to understand if we were making decisions in a way that made us safer. And I couldn't get this information out. It turned out that most big criminal justice agencies like my own didn't track the things that matter. So after about a month of being incredibly frustrated, I walked down into a conference room that was filled with detectives and stacks and stacks of case files, and the detectives were sitting there with yellow legal pads taking notes. They were trying to get the information I was looking for by going through case by case for the past five years. And as you can imagine, when we finally got the results, they weren't good. It turned out that we were doing a lot of low-level drug cases on the streets just around the corner from our office in Trenton. The second thing that happened is that I spent the day in the Camden,
New Jersey police department. Now, at that time, Camden, New Jersey, was the most dangerous city in America. I ran the Camden Police
Department because of that. I spent the day in the police department, and I was taken into a room
with senior police officials, all of whom were working hard and trying very hard to reduce crime in Camden. And what I saw in that room, as we talked about how to reduce crime, were a series of officers with a
lot of little yellow sticky notes. And they would take a yellow sticky
and they would write something on it and they would put it up on a board. And one of them said,
"We had a robbery two weeks ago. We have no suspects." And another said, "We had a shooting in this neighborhood last week. We have no suspects." We weren't using data-driven policing. We were essentially trying to fight crime with yellow Post-it notes. Now, both of these things made me realize fundamentally that we were failing. We didn't even know who was
in our criminal justice system, we didn't have any data about
the things that mattered, and we didn't share data or use analytics or tools to help us make better decisions and to reduce crime. And for the first time, I started to think about how we made decisions. When I was an assistant D.A., and when I was a federal prosecutor, I looked at the cases in front of me, and I generally made decisions based on my instinct and my experience. When I became attorney general, I could look at the system as a whole, and what surprised me is that I found that that was exactly how we were doing it across the entire system -- in police departments, in prosecutors's offices, in courts and in jails. And what I learned very quickly is that we weren't doing a good job. So I wanted to do things differently. I wanted to introduce data and analytics and rigorous statistical analysis into our work. In short, I wanted to moneyball criminal justice. Now, moneyball, as many of you know, is what the Oakland A's did, where they used smart data and statistics to figure out how to pick players that would help them win games, and they went from a system that
was based on baseball scouts who used to go out and watch players and use their instinct and experience, the scouts' instincts and experience, to pick players, from one to use smart data and rigorous statistical analysis to figure out how to pick players
that would help them win games. It worked for the Oakland A's, and it worked in the state of New Jersey. We took Camden off the top of the list as the most dangerous city in America. We reduced murders there by 41 percent, which actually means 37 lives were saved. And we reduced all crime in the city by 26 percent. We also changed the way
we did criminal prosecutions. So we went from doing low-level drug crimes that were outside our building to doing cases of statewide importance, on things like reducing violence
with the most violent offenders, prosecuting street gangs, gun and drug trafficking, and political corruption. And all of this matters greatly, because public safety to me is the most important function of government. If we're not safe, we can't be educated, we can't be healthy, we can't do any of the other things
we want to do in our lives. And we live in a country today where we face serious criminal justice problems. We have 12 million arrests every single year. The vast majority of those arrests are for low-level crimes, like misdemeanors, 70 to 80 percent. Less than five percent of all arrests are for violent crime. Yet we spend 75 billion, that's b for billion, dollars a year on state and local corrections costs. Right now, today, we have 2.3 million people in our jails and prisons. And we face unbelievable public safety challenges because we have a situation in which two thirds of the people in our jails are there waiting for trial. They haven't yet been convicted of a crime. They're just waiting for their day in court. And 67 percent of people come back. Our recidivism rate is amongst
the highest in the world. Almost seven in 10 people who are released from prison will be rearrested in a constant cycle of crime and incarceration. So when I started my job at the Arnold Foundation, I came back to looking at a lot of these questions, and I came back to thinking about how we had used data and analytics to transform the way we did criminal justice in New Jersey. And when I look at the criminal justice system in the United States today, I feel the exact same way that I did about the state of New Jersey when I started there, which is that we absolutely have to do better, and I know that we can do better. So I decided to focus on using data and analytics to help make the most critical decision in public safety, and that decision is the determination of whether, when someone has been arrested, whether they pose a risk to public safety and should be detained, or whether they don't pose a risk to public safety and should be released. Everything that happens in criminal cases comes out of this one decision. It impacts everything. It impacts sentencing. It impacts whether someone gets drug treatment. It impacts crime and violence. And when I talk to judges around the United States, which I do all the time now, they all say the same thing, which is that we put dangerous people in jail, and we let non-dangerous, nonviolent people out. They mean it and they believe it. But when you start to look at the data, which, by the way, the judges don't have, when we start to look at the data, what we find time and time again, is that this isn't the case. We find low-risk offenders, which makes up 50 percent of our
entire criminal justice population, we find that they're in jail. Take Leslie Chew, who was a Texas man who stole four blankets on a cold winter night. He was arrested, and he was kept in jail on 3,500 dollars bail, an amount that he could not afford to pay. And he stayed in jail for eight months until his case came up for trial, at a cost to taxpayers of more than 9,000 dollars. And at the other end of the spectrum, we're doing an equally terrible job. The people who we find are the highest-risk offenders, the people who we think have the highest likelihood of committing a new crime if they're released, we see nationally that 50 percent of those people are being released. The reason for this is the way we make decisions. Judges have the best intentions when they make these decisions about risk, but they're making them subjectively. They're like the baseball scouts 20 years ago who were using their instinct and their experience to try to decide what risk someone poses. They're being subjective, and we know what happens
with subjective decision making, which is that we are often wrong. What we need in this space are strong data and analytics. What I decided to look for was a strong data and analytic risk assessment tool, something that would let judges actually understand with a scientific and objective way what the risk was that was posed by someone in front of them. I looked all over the country, and I found that between five and 10 percent of all U.S. jurisdictions actually use any type of risk assessment tool, and when I looked at these tools, I quickly realized why. They were unbelievably expensive to administer, they were time-consuming, they were limited to the local jurisdiction in which they'd been created. So basically, they couldn't be scaled or transferred to other places. So I went out and built a phenomenal team of data scientists and researchers and statisticians to build a universal risk assessment tool, so that every single judge in
the United States of America can have an objective, scientific measure of risk. In the tool that we've built, what we did was we collected 1.5 million cases from all around the United States, from cities, from counties, from every single state in the country, the federal districts. And with those 1.5 million cases, which is the largest data set on pretrial in the United States today, we were able to basically find that there were 900-plus risk factors that we could look at to try to figure out what mattered most. And we found that there were nine specific things that mattered all across the country and that were the most highly predictive of risk. And so we built a universal risk assessment tool. And it looks like this. As you'll see, we put some information in, but most of it is incredibly simple, it's easy to use, it focuses on things like the
defendant's prior convictions, whether they've been sentenced to incarceration, whether they've engaged in violence before, whether they've even failed to come back to court. And with this tool, we can predict three things. First, whether or not someone will commit a new crime if they're released. Second, for the first time, and I think this is incredibly important, we can predict whether someone will commit an act of violence if they're released. And that's the single most important thing that judges say when you talk to them. And third, we can predict whether someone will come back to court. And every single judge in the
United States of America can use it, because it's been created on a universal data set. What judges see if they run the risk assessment tool is this -- it's a dashboard. At the top, you see the New Criminal Activity Score, six of course being the highest, and then in the middle you
see, "Elevated risk of violence." What that says is that this person is someone who has an elevated risk of violence that the judge should look twice at. And then, towards the bottom, you see the Failure to Appear Score, which again is the likelihood that someone will come back to court. Now I want to say something really important. It's not that I think we should be eliminating the judge's instinct and experience from this process. I don't. I actually believe the problem that we see and the reason that we have
these incredible system errors, where we're incarcerating
low-level, nonviolent people and we're releasing high-risk, dangerous people, is that we don't have an objective measure of risk. But what I believe should happen is that we should take that
data-driven risk assessment and combine that with the
judge's instinct and experience to lead us to better decision making. The tool went statewide in Kentucky on July 1, and we're about to go up in a
number of other U.S. jurisdictions. Our goal, quite simply, is that every single judge in the United States will use a data-driven risk tool within the next five years. We're now working on risk tools for prosecutors and for police officers as well, to try to take a system that runs today in America the same way it did 50 years ago, based on instinct and experience, and make it into one that runs on data and analytics. Now, the great news about all this, and we have a ton of work left to do, and we have a lot of culture to change, but the great news about all of it is that we know it works. It's why Google is Google, and it's why all these baseball teams use moneyball to win games. The great news for us as well is that it's the way that we can transform the American criminal justice system. It's how we can make our streets safer, we can reduce our prison costs, and we can make our system much fairer and more just. Some people call it data science. I call it moneyballing criminal justice. Thank you. (Applause)
Thanks for sharing!
This would probably sound really good to people who haven't worked for big police departments. I've seen it before in different forms, her version is "We used moneyball-style criminal intelligence stats to decrease crime by 24%", other versions are that "Our new deployment model has decreased crime by X%" or "Since the introduction of our (insert expensive new technology here) we've decreased drug violence by X%".
Yeah, maybe it contributed. Or maybe because the attorney general is sitting in the police station and promising politicians the city is going to be safer before the next election: overtime is getting signed, extra police are being deployed to that area, statistical fuckery was silently endorsed, etc. etc.
To top it off, her risk assessment tool is the least granular one I've ever seen. The one my police force uses distinguishes down to the individual offence level, i.e. stealing vehicles is higher risk than stealing a purse, dealing ice is higher risk than dealing cannabis etc. Can I get a Ted Talk now?
I’m a proponent of the data driven model and her statements about Google and Amazon are accurate but there’s an inconsistency in the logic: Google can not tell me what I’ll do, it tells me what I’m likely to do; probability. When we’re taking someone’s freedom or risking others’ safety the algorithm needs to be better than that. I’ve followed these efforts from afar d they all seem to come out as the “new best thing” but in short order they fail and are disbanded OR claims of racism or class favoritism take them down. The one factor we can’t predict is good old fashion spontaneity and impulse, whether it was 100 yrs ago with the crystal ball or today with the algorithm. Here’s a thought project for you: If you really believed in this tech then you’d undoubtedly support the flip-side approach that says if the “fair and reliable algorithm we use to take civil rights” classified me as a “good guy” or “no threat risk” then you can depend on that; this as a “good guy” I should be able to do what I want right? My statements will be trusted, I can borrow as much cash as I want, I get security clearance, I can buy a silenced firearm and take it on the plane with me wherever I want, statements against me will start out with prejudice against them, etc etc. Now think: Would you allow me those privileges based on same algorithm? If no, why not; your computer said I’m a really safe guy...today. (FYI I am pro 2A)
Can we get a synopsis or any sort of description beyond "theory"?