Independent · Updated continuously

PoliceAI News

Artificial Intelligence in Law Enforcement
Predictive policing

PredPol / Geolitica

The best-known place-based predictive policing product, which produced daily maps of where crime was likely to occur, found by independent analysis to predict at under half a percent accuracy before ceasing operations in 2023.

Geolitica (formerly PredPol); wound down 2023, assets absorbed by SoundThinking Supplier's own site ↗

PredPol was the best-known predictive policing product in the world, and its collapse is the most thoroughly documented failure in this catalogue. It ingested historical crime incident reports and produced daily predictions of where and when specific crime types were most likely to occur, marking small boxes on a map for officers to patrol. Created in 2010, it was a leading vendor by 2012, rebranded to Geolitica in 2021, and ceased operations at the end of 2023.

Its remains were absorbed by SoundThinking, the company formerly known as ShotSpotter, which hired Geolitica's engineering team, acquired patents, and migrated its customers onto its own patrol platform. SoundThinking had already bought HunchLab, a competing predictive policing product, from Azavea in 2018.

One detail of that acquisition is unusually revealing. SoundThinking Senior Vice President Sam Klepper stated that the company "was not interested in and did not purchase the source code for the modeling of the Geolitica prediction system", and that the algorithms it did take "do not include any crime prediction modeling source code at all." The buyer of a crime prediction company explicitly declined to buy the crime prediction.

HOW IT WAS SOLD

The pitch was resource allocation on an evidential basis rather than intuition: feed the system what has already happened, and it tells patrols where to be. The underlying model drew on earthquake aftershock mathematics, on the theory that crime clusters in space and time in comparable ways. In its later years the company shifted its positioning away from prediction and towards being a broader platform for managing police department data, which is itself an indication of how the core claim was faring.

WHAT INDEPENDENT ANALYSIS FOUND

Two investigations by The Markup are the substance of what is known, and both are unusually well evidenced because they rest on the company's own outputs rather than on interviews.

The first, published with Gizmodo in December 2021, analysed predictions Geolitica had supplied to police departments in 38 jurisdictions over nearly three years. The data was obtained from an unprotected cloud storage server linked from a page on the Los Angeles Police Department's own website. It found the software disproportionately directed officers to neighbourhoods with higher proportions of low-income, Black and Latino residents relative to the cities as a whole, while predictions for white middle and upper-class areas were largely absent.

The second, in October 2023, tested accuracy directly. The Markup examined 23,631 predictions generated for the Plainfield Police Department in New Jersey between 25 February and 18 December 2018. Fewer than 100 lined up with a crime of the predicted category subsequently reported to police. The success rate was below half of one percent. The Markup's own summary was that the software predicted crime at a rate slightly better than throwing darts at a map.

The most consequential finding is the least quoted. Comparing predictions against arrest data from eleven departments willing to share it, The Markup found arrest rates in predicted areas stayed the same whether or not a prediction had been made. The software did not measurably change what policing achieved.

WHAT THE CUSTOMERS SAID

Plainfield paid $20,500 for an annual subscription and a further $15,500 to extend. Captain David Guarino's account is worth quoting at length because it is rare for a force to be this direct: asked why the department bought it, he said they wanted to be more effective at reducing crime and thought knowing where to be would help, adding "I don't know that it did that." He continued: "I don't believe we really used it that often, if at all. That's why we ended up getting rid of it." He told The Markup the money could have been better spent elsewhere.

Plainfield was the only one of 38 departments willing to provide information.

THE STRUCTURAL CRITICISM

Beyond accuracy, the persistent objection is that a model trained on recorded crime learns where police have previously been, not where crime has previously happened — and those are different things. Directing patrols to those areas generates more recorded incidents there, which feeds back into the model. The scholar Ruha Benjamin characterised PredPol as a "crime production algorithm" on exactly this reasoning: officers patrol predicted zones expecting to find crime, and the expectation is self-fulfilling.

This is not a criticism that better engineering resolves. It is a property of training a predictive system on enforcement data and then using its output to direct enforcement.

IS THERE A CASE FOR IT

In fairness, the underlying problem is real: patrol resources are finite and have to be allocated somehow, and the alternative to a model is officer intuition, which carries its own well-documented biases. Place-based prediction was also, in principle, a narrower intrusion than scoring individuals, since it directs attention to a location rather than labelling a person.

But that defence requires the predictions to work, and the strongest available evidence is that these did not. A model that performs at under half a percent, and produces no measurable change in arrest outcomes, does not improve on intuition — it launders it.

WHAT IS NOT ESTABLISHED

The Markup's accuracy analysis rests on a single department over ten months. It is the most rigorous public assessment that exists, but Plainfield was the only force of 38 that cooperated, so whether that success rate was typical is unknown.

Why 37 departments declined to share arrest data is not established, and no comprehensive list of Geolitica's customers has been published.

What happened to the prediction data already generated for those 38 jurisdictions, and whether it was deleted when the company wound down, is undocumented. The 2021 investigation established that at least some of it had been sitting on an unprotected server.

Whether SoundThinking's successor products reproduce the same weaknesses is untested. The company states it did not acquire the prediction modelling, but both its platform and Geolitica's advertised incorporating risk terrain modelling, and no independent evaluation of the replacement has been identified.

Related subject: Predictive Policing and Risk Scoring

Where this is deployed

Full tracker →

No tracker entries currently reference this technology.

Sources

  1. The Markup — Predictive policing software terrible at predicting crimes (Plainfield accuracy analysis)
  2. The Markup — How we assessed the accuracy of predictive policing software (methodology)
  3. Tech Policy Press — politicians move to limit predictive policing after years of controversial failures
  4. Route Fifty — coverage of the Markup findings and the SoundThinking transition
  5. SoundThinking — successor vendor's own site