Child protection

In child abuse investigations a single device can hold hundreds of thousands of files. Automated tools decide what a human examiner looks at first; they do not reach conclusions.

Guide

What it is

  • Hash matching compares a file's digital fingerprint with databases of known illegal material held by bodies such as the Internet Watch Foundation. It is highly reliable, with very few false positives.
  • Classification machine learning that assesses content to flag material not already in any database, which is where errors occur.
  • Triage ordering seized material so that evidence most likely to identify a child at risk is seen first.

How it is used

Forces use hash matching to identify known material without anyone viewing it again, and classifiers to surface new material for human review. Both also reduce how much harmful material investigators must see. The record tracks the tools in use and the legal response to AI-generated abuse material.

Key tensions

A probability is not a finding

A classifier can be wrong in both directions, and a false positive is devastating for the person affected. The safeguard is that a human examiner reaches every conclusion that matters.

AI-generated material

Synthetic images are new by definition, so hash matching cannot catch them, and they consume review time without the signal of a real child at risk. The law here is still developing.

Scope creep

Detection built for one kind of material can be pointed at others. Proposals for scanning on encrypted messaging divide people who agree entirely about the harm.

View all child protection records Latest coverage

In depth

The full guide, about 3 minutes to read.

The problem this technology exists to address

Child protection investigation has a volume problem that is difficult to convey. A single seized device can hold hundreds of thousands of files. A single investigation can involve dozens of devices. There is no realistic prospect of a human examiner reviewing all of it, and the material that most urgently needs finding, evidence identifying a child currently at risk, may sit anywhere in that volume.

Automated triage exists to address that directly. Its purpose is to determine what a human looks at first, not to reach conclusions. That framing matters for everything that follows, because the questions raised by a prioritisation tool are quite different from those raised by a system that decides anything on its own.

Hash matching and classification are not the same thing

These two techniques are routinely discussed as one, and conflating them systematically overstates how reliable the newer one is.

Hash matching generates a cryptographic fingerprint of a file and compares it against a database of fingerprints of known illegal material, maintained by bodies such as the Internet Watch Foundation in the UK and the National Center for Missing and Exploited Children in the US. Perceptual hashing variants tolerate minor alterations such as resizing. Critically, this technique does not analyse image content at all: it asks whether this file matches a file already identified by human reviewers. It is long-established, highly reliable, and produces very few false positives.

Classification uses machine learning to assess content in order to flag material that is not already in any database. This is what allows previously unseen material to surface, which hash matching by definition cannot do. It is also where the error rate lives. A classifier produces a probability, not a determination, and it can be wrong in both directions.

The practical consequence is that classifier output is treated as a prioritisation signal for human review, not as a finding. A false positive here is extraordinarily damaging to the person affected, and the safeguard against it is that a human examiner, not the classifier, reaches every conclusion that matters.

The wellbeing dimension

There is a use case here that is rarely discussed outside the profession and is worth stating plainly: reducing how much of this material human beings have to look at.

Investigators and examiners working in this area carry a well-documented occupational burden, and forces have long-standing welfare provisions in recognition of it. Automated triage reduces exposure in two ways. It reduces the total volume a person must review, and hash matching in particular allows already-identified material to be categorised without a human viewing it again. That is a genuine benefit and one of the strongest arguments for the technology, quite separate from any efficiency case.

Generative AI has changed the picture

Generative image models have produced a category of synthetic material that did not previously exist at scale, and it complicates this area in several distinct ways.

Detection is harder, because hash matching relies on a file having been seen and catalogued before, and generated material is novel by construction. Triage is harder, because volume can be produced far faster than it can be reviewed. And investigative prioritisation is harder in a specific and consequential way: a central purpose of this work is identifying children who are currently at risk, and material depicting no real child does not carry that signal while still consuming review capacity.

The legal position varies by jurisdiction and has been actively developing. The UK has legislated in this area, and enforcement questions around generated material continue to be worked through in courts and legislatures internationally. This is one of the faster-moving areas covered on this site, and see deepfakes for the wider synthetic media picture.

Scope creep, the standing objection

The most persistent concern raised about detection infrastructure in this area is not about its use for its stated purpose, which attracts very little objection. It is about extension.

Systems built to detect one category of material can, technically, be pointed at others. Proposals to require client-side scanning on messaging platforms have been the sharpest expression of this argument: supporters view it as the only workable approach to material shared in end-to-end encrypted environments, while opponents argue it creates a general-purpose scanning capability whose future scope depends on policy rather than on technical limits. Both positions are held in good faith by people who agree entirely about the underlying harm, which is worth stating, because this is frequently reported as though one side were indifferent to it.

Records on the tracker

Open the full tracker

15 records on child protection, most recently checked first: 11 in use now, 2 withdrawn and 2 on the horizon. Each is one police force or public body using one system.

  1. Denmark2022

    Københavns Vestegns Politi (Copenhagen Vestegn Police) — Facial recognition function in a purchased IT programme, used to identify abused children

    Copenhagen Vestegn police district

    DiscontinuedChecked
  2. Denmark

    National enhed for Særlig Kriminalitet (NSK) (Danish National Special Crime Unit) — Facial recognition application for identifying child victims of sexual abuse in image material

    ProposedChecked
  3. United Kingdom

    Swansea University — DRAGON-Spotter (AI linguistic detection of online grooming in seized-device chat logs)

    Swansea

    In developmentChecked
  4. France

    Gendarmerie nationale — "Parole" speech-to-text transcription of child victim interviews

    OperationalChecked
  5. France

    Gendarmerie nationale — Automatic categorisation of seized child sexual abuse images

    OperationalChecked
  6. GermanyGriffeye

    Thüringer Landeskriminalamt — Griffeye Analyze DI Pro with 'Griffeye Brain' AI for flagging child sexual abuse material

    Thuringia

    OperationalChecked
  7. Germany2020

    Landeskriminalamt Niedersachsen — KiPo-Analyzer and Tracebook KiPo (in-house AI pre-sorting of suspected child sexual abuse material)

    Lower Saxony

    OperationalChecked
  8. Germany

    Landeskriminalamt Nordrhein-Westfalen — JAIF (Joint Artificial Intelligence Force): AI-assisted review of seized IT material

    North Rhine-Westphalia

    OperationalChecked
  9. GermanyMagnet (Griffeye)

    Polizeipräsidium Unterfranken — Griffeye pre-filtering of image and video material in child sexual abuse cases

    Bavaria (Lower Franconia)

    TrialChecked
  10. Switzerland

    Police cantonale genevoise — AI image classification in child sexual abuse investigations (model developed by an NGO)

    Geneva

    OperationalChecked
  11. United KingdomNot specified2016

    Bristol City Council — Think Family Database and child exploitation risk models

    Bristol

    DiscontinuedChecked
  12. United KingdomTrilateral Research (CESIUM)2023

    Lincolnshire Police — CESIUM child exploitation risk identification

    Lincolnshire

    OperationalChecked
  13. United KingdomTrilateral Research2024

    Nottinghamshire Police — CESIUM child exploitation risk identification

    Nottinghamshire

    TrialChecked
  14. United KingdomRoke (Vigil AI classifier); Qumodo and Cyan Forensics also credited2014

    Home Office — CSAM Detection and Classification (Vigil AI / CAID)

    UK-wide

    OperationalChecked
  15. Singapore

    Singapore Police Force — AI screening of seized devices for obscene material

    Singapore (national)

    OperationalChecked

Latest coverage