Alaska pretrial risk assessment tool
Alaska pretrial risk assessment tool
Geri Fox and Pamela Cravez
Geri Fox, Director of the Pretrial Enforcement Division of the Alaska Department of Corrections, is interviewed by Pamela Cravez, editor of the Alaska Justice Forum, about the advantages and limitiations of Alaska’s new pretrial risk assessment tool. The tool, incorporated in Alaska’s new bail statute, calculates whether a defendant is at low, moderate, or high risk for failure to appear at trial or to commit another crime if the defendant is released pretrial, and aids in the judge's decision regarding pretrial bail conditions.
"Alaska Pretrial Risk Assessment Tool" (23:58 mins.)
This is a transcript of a video presentation, which can be found at https://youtu.be/wYEP3wDnVVQ
I’m Pam Cravez, editor of the Alaska Justice Forum at UAA’s Justice Center.
I’m here with Geri Fox, who heads up the Alaska Department of Corrections’ new Pretrial Enforcement Division.
This division, beginning January 1st, 2018, is providing courts throughout the state information about defendants who are up for their first pretrial hearing. The information comes from a new pretrial risk assessment tool. The tool calculates whether a defendant is high, medium, or low risk for failure to appear at their next court appearance or for committing a new crime if released pretrial.
Geri, you’ve spent the last year going around the state and talking with judges and lawyers and law enforcement about this new pretrial risk assessment tool. Can you tell us a little bit about the tool and the work that you are doing and how you’ve come to this work?
Yeah, I have been in correctional work now for more than 20 years, so it really is my life’s work — this is where I have developed some academic background, as well as a practitioner background.
And so, in my work in this area, I have really familiarized with evidence-based practices. I started working with evidence-based models about 15 years ago or so, and I’ve been introducing those in almost every capacity ever since.
So, Alaska chose to take on an evidence-based model for our state, so when pretrial came about, I started really digging in and trying to learn about how we got here.
So one of the ways we got here is we have an 81 percent growth in our unsentenced population that is remaining behind bars, and in most cases these people are eligible for bail. So the question becomes, how are we growing at this kind of rate? It did not make sense with what we see in terms of our conviction rate. So that was one of the things that was really a red flag. Something seemed wrong in our pretrial justice system.
And we also have two baselines now that tell us a lot about what’s happening in Alaska and how a new pretrial assessment process will affect the future.
One of those baselines is for our failure to appear measurement. What we know is that if we let a defendant out of jail approximately 14 percent of those individuals will fail to appear for court.
We also now know that if someone is released from custody, there’s about a 37 percent likelihood that they will be rearrested for a new criminal offense before they ever go to trial.
Those are those are pretty telling numbers, and it gives us a starting point as we release a new assessment tool to let us know how we’re doing in the future.
So you’ll be able to measure the success of this new pretrial assessment tool if the numbers of people who are held in jail goes down, pretrial, and also, if these other numbers go down — that you have fewer people failing to appear and fewer people committing a new crime if released pretrial on bail.
And there’s all kinds of ways to interpret numbers as well, so we need to look at down the road why numbers shift and in what ways they shift.
There are really three pillars that all have to be balanced in pretrial — a good pretrial model — so good pretrial models find the bright balance between releasing people and achieving public safety and court appearance.
So those are the three things that we need to measure: What does happen to incarceration rates? — Do we continue this 81% growth in the incarcerated population? It’s a really important number that we’re going to watch.
Another thing as well — if we if we have more people out, for example: What happens to failure to appear rates? Do they go up slightly, do they go down slightly?
And how do we balance that with public safety? So those three pillars really do tell us the overall effectiveness of a pretrial process.
So how does an assessment tool help address what’s happening to the population?
Great question. So the assessment tool will help us know who is a lower risk for pretrial failure and who is a higher risk for pretrial failure. There’s a lot of detail around what those terms even mean, but an assessment tool gives the judiciary a way of objectively evaluating whether or not we’re comfortable with certain types of releases — under what conditions we should make those releases — and particularly a risk assessment helps those who are experiencing conditions of poverty achieve release if in fact they are a lower risk anyway.
So we’ve never known much about risk. What we knew is, do you have the ability to pay? That was mostly how our justice system has made release decisions. And so we end up with disparities — unintentional disparities in our system with regard to who gets released and who doesn’t, and what we missed was risk. I mean it just seems so obvious, right? — so obvious. I mean, when I talked to the public about what risk assessment does, they’re like, we haven’t been doing risk assessment?
What? How did we miss that? So now we are in our state, and it’s a national trend that we’re seeing.
So we’re doing it in our state, and it’s a national trend. How do people develop risk assessment tools?
It’s an advanced calculation — a statistical calculation. So risk assessment tools are developed in a variety of ways. Alaska chose to develop our risk assessment tool from drawing upon Alaska data. So there are tools in Ohio, and there’s tools in New Jersey, and there’s tools that are used in Kentucky. And those they’re all great, but those are not based on the population from Alaska.
And our representatives asked for a validated assessment tool. What that means is we need to look at Alaska. We need to look at our population, and so that’s what we did. We’ve spent the last year pulling data from all kinds of different resources, and then researchers from various fields, whose expertise is in criminal justice risk assessment, helped us develop an assessment tool.
What does the tool measure?
Another good question.
So the tool measures a likelihood for a failure to appear, and that means if someone is released after they’re arrested and they’re awaiting their trial, what’s the likelihood they’re gonna come back to court? That matters in a criminal justice release decision, right? We want to know that okay, if you if you go out, will you come back to court? So the first piece is failure to appear.
The next measurement is the likelihood of a new criminal arrest. So, arrest does not equal conviction. People may be arrested, and about 30 percent of people who are arrested will have their case dismissed. So an arrest is merely an arrest — it does not imply an ultimate conviction.
What are the factors that go into this model? So I’m assuming you look at things like, well, have they committed crimes in the past, or have they failed to appear in the past — that help you determine whether they’re going to do this again. Well, how do you know that these, you know, which are the right factors and how does this tool actually figure out these factors?
Yeah. So this is part of really understanding how tools are created. Our remarkable researchers ran more than 4,000 statistical calculations to find out what of those decisions or factors actually matter. What becomes predictive in determining those two things — your likelihood to appear for court or your likelihood for a new arrest?
And so what we found was there are six factors that are most likely to predict someone’s failure to appear and six factors that are most likely to predict someone’s likelihood of a new arrest. There are things like, how many felony arrests have you had in five years? How many misdemeanor arrests have you had in three years? What was the age at your first arrest? Are you arrested on a property offense? Are you arrested on a motor vehicle offense?
So, sometimes we were surprised by what a risk factor ended up being in our state, but what we did that was right — this is what is right about these things — is we let the data tell us what’s predictive.
I also understand that in other states sometimes people use both static criteria — things like prior convictions, which never change — and criteria that do change — things like whether somebody has a substance abuse problem, or they’re employed, or maybe what their income is, or ties to the community that could also change. How are all of these factors considered in our new system of using this tool?
So, static risk factors are things, as you said, that don’t change. Those are things like your criminal background — once you have an arrest, you have an arrest. That’s not going to go away.
So we chose a static tool. A couple of reasons for that. If we choose a static tool, we don’t have to interview defendants, and that’s important for a couple of reasons: it can save money, it can speed up the process, and a lot of our defense attorneys don’t really want us talking with defendants at that point after an arrest.
There are other states that do a dynamic AND static assessment. Usually they’re a combined thing. You don’t just have one or the other, if you have a dynamic component. So most states that do have a dynamic component have static questions, just like Alaska, and then they ask a couple of other things.
One common example is, “Do you have a cell phone?” or “Do you have a telephone?” — which we think — wait! oh, yeah — well, maybe that IS important so you can get a hold of someone, right? But what we find is, that is not a predictor of how you do. It might be helpful for the court to know, and for years we’ve developed tools based on the things that we thought would matter, right. So you — if you have employment, well, THAT should matter — that you’ll show up to court if you have a job, and you’ll show up to court if you have a cell phone, and you will show up to court if you have a stable residence — and what the research tells us is those things are not necessarily predictive. They’re helpful to know, they’re helpful in getting ahold of a defendant, but it doesn’t necessarily predict if you show up.
So states do have dynamic factors that also ARE predictive. One of the things that can be predictive in a state is substance abuse history or a mental health condition. Those things MAY be predictive.
So it depends on the state, and it depends on what kind of cost and timeline the state
wants. What we know as static and dynamic tools tend to perform about equally so one
is not necessarily better or worse — they tend to perform fairly, fairly equally.
So Alaska chose a good model that is cost-effective, and it gets the job done.
What about racial and gender bias? I know with a lot of static features — engrained in those static features are inherent biases that have to do with gender and race and other things. How are these accounted for in a tool that is using static criteria?
We want to make sure that our tool doesn’t perpetuate bias. So, we can’t necessarily determine if we have disparate arrest rates, for example, for a population. A pretrial tool doesn’t resolve that. But what we don’t want it to do is PERPETUATE that.
So when we developed our tool, we specifically controlled for events like, does gender make a difference? If you’re a male or female, does the tool perform differently based on your gender? Is the tool performing differently if you are Alaska Native or Caucasian or a Spanish speaker?
And so our researchers are able to control for those kind of factors and make sure that those variables don’t affect the performance of the tool. So we specifically validated an instrument that works across gender/race/ethnicity.
And of course, you know, there may be variables that we just simply don’t have data for, as well, that might end up revealing themselves as being something that we want to pay attention to down the road. But for now, we’ve developed a tool that’s controlled for those kinds of things. And so we don’t see it perpetuating bias.
How will you evaluate this tool to see whether it’s actually doing what you want it to do?
So — evaluating an assessment tool is — it’s essential — and it’s true of any assessment tool in criminal justice — it constantly needs to be revalidated. So we will run data for a year. Along the way we’re going to be doing some spot checks, so let me assure the public that we’re doing a couple of quality assurance tests along the way. We have lots of fidelity pieces that help us know that we’re administering our tool properly, and at the end of a year we will hire a new researcher — a different researcher — so that we also eliminate some research bias, or at least we control for research bias, and we’ll bring someone in to take another look at how it’s performing.
And we might find that something’s changed. We may find that one of the factors we thought was important is not as important now, and so we might say, well, how does that happen? Well, for the same reason that we all a sudden have an opioid epidemic. So, things change in our population. Things change with employment. Things change with substance abuses or the types of crimes that are happening. And those things also affect a predictive validity of a tool.
Will all of the parties at court get the information from this tool at the hearing? I’m thinking, you know, we have defense attorneys and prosecutors and court officials. How do you envision this tool actually being employed at court?
So distribution of a report is a logistical issue. It’s a challenge for us to work through. So we’ve worked through it.
I think we have a good solution. What we’ve done is created a log-on account to the Department of Corrections database, and all of our partners can get a log-on account. So private defense attorneys can get access to a log-on account, public defenders, the prosecutors and even the courts.
Is there anything else you’d like to tell me about this tool? — have we covered —?
If I could talk a little bit about the things that judges can consider.
One of the realities of any assessment tool is, it’s not a crystal ball. There is no tool out there that will tell us a hundred percent of how an individual will behave 100 percent of the time. So we need to recognize that as a criminal justice system, and Alaska did with their statute.
So the tool’s a piece — it’s one piece that the judiciary can evaluate but there are about 12 other factors in our statute that a judge can evaluate when they make release decisions.
So they can look at things like what is the weight of evidence that is against a person at the time that they are arrested? And this matters, right? — so — when there are some cases that there’s some very clear kinds of indication that we’ve got a very serious matter on our hands, the judge can look at those things. The judge can consider the type and nature of the offense that we’re dealing with. So one assault doesn’t necessarily look like another, and the judge needs some discretion, and how they think through some of these events that they have to make decisions about. A judge can consider the likelihood of threat to potential future victims.
So, there’s a variety of things that they can and should consider.
So from what I’m gathering, one of the limitations to this is that it cannot predict 100 percent. Is there a measure of how well it has been shown to predict in other places? — and actually this is two questions — and are there some things that you’ve not been able to put in the model?
We have a starting point. We have a baseline, so we know where we’re starting. And the way that we got a baseline is, people get out of jail now, and so we looked at what happens when people get out of jail in our state, and how do they do?
And so what we find is that we have low-risk defendants that are getting out of jail and we have moderate-risk defendants that get out of jail and we have high-risk defendants that get out of jail currently in our state. And now we know, based on our research, how those populations tend to do. So we do have baselines, and then we’ll be monitoring those baselines in the future.
And it’s really important to also note that a pretrial risk assessment tool, although it has the word “risk” in the title, it’s not a measure for dangerousness. So when we say pretrial risk assessment, it doesn’t measure how dangerous a person is. Remember, it measures the likelihood of failure to appear or the likelihood of a new criminal arrest. And in our state if someone is rearrested in pretrial status, most of those are for lower level misdemeanor offenses.
So if you are a high-risk person on this assessment tool right now — and again, we haven’t even started assessing, but we can apply the scores to the population that has been released in the past — and what we know is about 58 percent of the high-risk population may be returned on a new criminal offense. What that means is about 42 percent are NOT returned on a new criminal offense. So this is what a judge has to evaluate. Are you maybe part of the 58 percent? Or are you part of the 42 percent?
And so that’s where it’s not a crystal ball. We don’t really know where that individual will fall in that. But what a judge now has is pretrial enforcement officers that can monitor an individual if indeed they secure release.
Do judges know the way this tool has been developed and the standards for high risk and what percentages that they show? I mean, how have they been trained to understand that even a high-risk person may not commit a new crime?
I would say probably not well enough. I think that these are the kinds of details and nuances that — it takes time, really, to understand all the parts and pieces. It’s my life — it’s what — that’s how I spend my last 18 months. My whole life is pretrial — which is fine, and again it’s an honor to do it — but communicating all of these kinds of things, they take time.
So the risk assessment tool seems like it’s something that mirrors what we see in the insurance industry and other industries that are trying to assess risk — often when it has to do with money. And this of course has to do with people’s lives.
So yeah, it’s called an actuarial assessment tool, and to some people that has meaning and to others they have no idea what that means. So the example I like to use is that we use these tools in all kinds of industry.
One is in the medical field. So when we go to our doctor they may collect our body weight. Perhaps they ask about your family history. Perhaps they ask if you smoke, right? And the reason that the medical professionals care about all these things that we do in our lives and the way that we eat and our body weight is because those things create risk factors that might lead to us having certain medical conditions. So an example would be, what’s the likelihood that you will develop heart disease Well, perhaps you’re overweight. Perhaps there’s a family history of heart disease. Perhaps you are a smoker, right? So those are the risk factors.
So we use those kinds of modeling statistics in criminal justice. It’s the same kind of thing. So the modeling is very similar. But does it mean that someone who smokes will have heart disease? No, it doesn’t mean that. You know, we may have somebody who’s done everything right who ends up with heart disease, and that’s true in a criminal justice actuarial tool as well. It gives us a really great way to think about the probability or the likelihood of something happening, but it is not a guarantee.
That’s a very good way of putting it as far as helping people understand that it’s just one factor among many to be considered, and not to rely upon it too strongly, but to take it into consideration.
Thank you so much for giving us the opportunity to try to help people understand.
I want to assure my colleagues and the public that it’s not a hundred percent. We know that there are weaknesses with assessment tools, and so as a criminal justice system we have to know what those are and we have to be smart about this, and we have to really watch how it performs, and my officers have to be very vigilant in the future, to do whatever we can to make sure that we get it right.
Thank you, Geri.
Geri Fox is Director of the Pretrial Enforcement Division, Alaska Department of Corrections. Pamela Cravez is editor of the Alaska Justice Forum.
Fox, Geri; & Cravez, Pamela. (2018). “Alaska Pretrial Risk Assessment Tool” (streaming video). Alaska Justice Forum 34(3), Winter 2018. (23:58 mins.). Produced by the UAA Justice Center and Eric Baldwin, UAA Academic Innovations and eLearning, Jan 2018. Anchorage, AK: Justice Center, University of Alaska Anchorage.