OASys and Actuarial Reoffending Prediction

The Offender Assessment System is the structured assessment used across prisons and probation in England and Wales to evaluate an individual's likelihood of reoffending, their criminogenic needs and their risk of serious harm. It informs sentence planning, supervision intensity, and decisions about parole. The current automated version has been in operation since 2013, managed by the Ministry of Justice, with assessments collated centrally by the OASys Data, Evaluation and Analysis Team.

A distinction this site tries to draw consistently applies here, and cuts against the assumption that everything on a police AI catalogue is AI. OASys is actuarial. Its algorithmic components — principally the Offender Group Reconviction Scale, which is also used independently for short pre-sentence reports — apply defined statistical weightings to recorded factors. They do not currently use machine learning. Published analysis notes there is some indication that machine learning methods are under consideration, and that the willingness to use them in policing risk tools suggests a possible direction of travel, but that is a prospect rather than a description of the current system.

That matters because criticisms of opaque, unexplainable models do not straightforwardly apply to a system whose weightings are documented, while criticisms about validity, disparity and over-reliance apply just as forcefully.

HOW IT WORKS AND WHAT HAS BEEN BUILT ONTO IT

Assessment combines actuarial prediction with structured professional judgement by the assessor. Alongside the general reoffending prediction sits a Risk of Serious Harm section allowing the assessor to identify factors specific to that construct, which is a different question from frequency of reoffending.

Specialised components have been added over time. A Sexual Predictor component, developed to predict contact sexual reoffending, was validated on a study of almost fifteen thousand individuals who had committed sexual offences. An offence-free time component allows estimates to be adjusted for those who have already spent periods in the community without reoffending, reflecting that risk is highest immediately after sentence.

The scale is substantial: in 2007/08 around 700,000 assessments were completed on almost 350,000 individuals.

THE EVIDENCE BASE

OASys is, by the standards of this catalogue, unusually well studied. The Ministry of Justice published a compendium of research and analysis covering 2009 to 2013, examining predictive validity, assessor experience and the performance of individual components. Few systems on this site have anything comparable.

The most consequential finding for present purposes concerns disparity. Ministry of Justice analysis found that the predictive validity of the algorithms used in OASys was greater for white offenders than for offenders of Asian, Black and Mixed ethnicity. That is a government finding about a government tool, not an advocacy claim, and it is the reason this system connects directly to the wider fairness debate documented on this site's COMPAS page.

THE CASE FOR IT

Structured assessment applied consistently is a defensible improvement on unaided judgement, which varies between assessors and is not auditable. Because OASys is actuarial and its components are documented and published, it can be scrutinised in ways that a proprietary model cannot — and it has been.

The design also retains the assessor. OASys informs a professional judgement rather than replacing it, and the Risk of Serious Harm section is explicitly a space for the assessor's own analysis.

The research programme deserves credit on its own terms. Publishing a multi-year compendium including validation of individual components, and analysis that surfaces uncomfortable findings about ethnic disparity, is more transparency than most operators of risk tools have offered.

THE CASE AGAINST

Differential predictive validity by ethnicity is the central problem. A tool that predicts less accurately for some groups produces assessments that are, on average, less reliable for those groups — and it informs decisions about liberty, supervision and parole.

There is a longer-standing critique that assessment tools of this kind were adopted and promoted by correctional services without robust evidence that they improve prediction of reoffending or the management of individuals. Some practitioners have argued the process displaces the relationship-building that effective supervision depends on, though this has itself been contested as mistaking the tool for a substitute rather than an aid.

Scale creates its own risk. When a tool is applied to hundreds of thousands of assessments a year, a score becomes the default frame through which a person is understood by every subsequent decision-maker, whatever the assessor originally intended.

And the direction of travel deserves attention. If machine learning components are introduced, the documented-weighting transparency that currently distinguishes OASys from proprietary systems would be diminished, and that change should be visible rather than incremental.

WHAT IS NOT ESTABLISHED

Whether machine learning has since been introduced into any OASys component is not established by the sources reviewed.

Whether the ethnic disparity in predictive validity has been addressed in subsequent revisions, and whether any fix has been independently evaluated, is not documented.

How often assessors depart from the actuarial score, and in which direction, is not published.

Current annual assessment volumes are not available in the sources reviewed; the figures above are from 2007/08.