🔗 Share this article

Tech firms and child safety organizations will be granted permission to evaluate whether artificial intelligence tools can generate child exploitation material under new British laws.

Substantial Rise in AI-Generated Harmful Material

The declaration came as findings from a protection monitoring body showing that reports of AI-generated child sexual abuse material have increased dramatically in the last twelve months, growing from 199 in 2024 to 426 in 2025.

Updated Legal Structure

Under the changes, the government will permit designated AI companies and child protection organizations to inspect AI systems – the foundational systems for conversational AI and image generators – and ensure they have sufficient protective measures to stop them from creating depictions of child exploitation.

"Ultimately about preventing abuse before it occurs," declared Kanishka Narayan, adding: "Specialists, under strict conditions, can now detect the risk in AI systems promptly."

Tackling Regulatory Challenges

The changes have been introduced because it is illegal to produce and possess CSAM, meaning that AI creators and others cannot generate such content as part of a testing regime. Previously, authorities had to wait until AI-generated CSAM was published online before dealing with it.

This legislation is aimed at preventing that problem by enabling to halt the creation of those materials at source.

Legislative Structure

The changes are being added by the authorities as revisions to the criminal justice legislation, which is also implementing a ban on possessing, producing or sharing AI systems designed to create exploitative content.

Real-World Consequences

This recently, the minister visited the London base of Childline and listened to a mock-up call to counsellors involving a report of AI-based exploitation. The interaction portrayed a adolescent seeking help after being blackmailed using a sexualised deepfake of themselves, constructed using AI.

"When I learn about young people facing extortion online, it is a source of extreme anger in me and justified concern amongst parents," he stated.

Concerning Data

A leading internet monitoring organization stated that cases of AI-generated abuse material – such as online pages that may contain multiple images – had significantly increased so far this year.

Cases of category A material – the most serious form of exploitation – rose from 2,621 visual files to 3,086.

• Girls were predominantly victimized, accounting for 94% of prohibited AI images in 2025
• Portrayals of infants to toddlers rose from five in 2024 to 92 in 2025

Sector Response

The legislative amendment could "constitute a crucial step to guarantee AI products are secure before they are released," commented the chief executive of the internet monitoring foundation.

"Artificial intelligence systems have enabled so survivors can be targeted repeatedly with just a simple actions, providing offenders the ability to create possibly limitless amounts of sophisticated, photorealistic exploitative content," she added. "Content which further exploits survivors' trauma, and makes children, especially female children, less safe on and off line."

Counseling Interaction Data

The children's helpline also published information of support sessions where AI has been mentioned. AI-related harms mentioned in the sessions include:

• Using AI to rate weight, body and appearance
• AI assistants dissuading young people from talking to safe guardians about abuse
• Being bullied online with AI-generated content
• Digital extortion using AI-faked images

During April and September this year, the helpline delivered 367 counselling interactions where AI, chatbots and related terms were mentioned, significantly more as many as in the equivalent timeframe last year.

Half of the references of AI in the 2025 sessions were connected with mental health and wellbeing, including using chatbots for support and AI therapy apps.