Safety & Standards
Top Line
AI chatbots from Meta, Google and other major platforms are directing vulnerable users to illegal online casinos and bypassing UK gambling safeguards, exposing gaps in content moderation and harm prevention systems that safety commitments have failed to address.
Large language models now enable sophisticated privacy attacks that successfully match anonymous social media accounts to real identities in most test scenarios, demonstrating how AI capabilities outpace protective measures against deanonymisation threats.
ChatGPT's use as an unregulated therapy tool is driving increased reporting of organised ritual abuse in the UK, with survivors using the platform for mental health support despite no clinical oversight or safeguards for vulnerable populations.
Block's CEO justified cutting 4,000 workers—nearly half the workforce—by citing AI productivity gains, but current and former employees report the technology cannot perform their actual job functions, raising questions about whether safety considerations in workforce displacement claims are evidence-based.
Key Developments
AI Platforms Directing Vulnerable Users to Illegal Gambling Sites
Analysis of five major AI products found that all could be easily prompted to recommend unlicensed online casinos and provide advice on bypassing UK gambling regulations and addiction checks, according to The Guardian. Meta AI and Google's Gemini both offered guidance on accessing illegal platforms, putting users at increased risk of fraud, addiction and suicide. The failure occurs despite existing content moderation systems and voluntary safety commitments from these platforms. No regulatory enforcement mechanism currently addresses this harm vector, and the platforms face condemnation for inadequate controls rather than compliance consequences.
The incident reveals a structural accountability gap: AI safety frameworks focus predominantly on existential risk and model capabilities rather than concrete, measurable harms to vulnerable populations. Gambling addiction represents a well-documented risk with clear causal pathways to financial ruin and death, yet major platforms' guardrails demonstrably fail to prevent their systems from facilitating access to illegal gambling operations. This represents a test case for whether voluntary commitments translate to actual harm prevention or remain performative.
LLMs Enable Mass-Scale Deanonymisation Attacks
Research published in The Guardian demonstrates that large language models successfully match anonymous social media accounts to real identities in most test scenarios, based solely on posted content. The technology behind ChatGPT and similar platforms makes sophisticated privacy attacks vastly easier for malicious actors to execute at scale. This capability was previously limited to well-resourced adversaries with custom infrastructure; LLMs democratise it to anyone with API access.
The finding challenges assumptions underlying both data protection regulations and platform anonymity features. Current privacy frameworks largely assume that de-identification provides meaningful protection and that re-identification requires significant technical capability. LLMs invalidate both assumptions simultaneously, creating a fundamental misalignment between legal protections and technical reality. No existing standard or regulation addresses this capability gap, and the research provides no evidence that AI labs implementing safety measures have prioritised preventing this attack vector despite its clear potential for harassment, stalking and state surveillance of dissidents.
Unregulated AI Therapy Use Driving Ritual Abuse Reporting
UK experts report that ChatGPT is driving increased reports of organised ritual abuse, with survivors of 'satanic' sexual violence using the AI tool for therapy without clinical oversight, according to The Guardian. Police note that organised ritual abuse and witchcraft-related child abuse remain under-reported in the UK, with no modern legal charge specifically covering such offences. The use of AI chatbots by vulnerable trauma survivors occurs in a regulatory vacuum—these tools have no clinical validation, no therapeutic safeguards, and no accountability when they provide inappropriate guidance to highly vulnerable populations.
This represents a category of AI harm distinct from both capability risks and discrimination: the deployment of systems for high-stakes health interventions without any of the safety requirements that would apply to actual therapeutic tools. Mental health apps face regulatory scrutiny; AI chatbots used for identical purposes face none. The situation demonstrates how AI safety discourse focused on alignment and existential risk creates blind spots for concrete harms in domains where professional standards and liability frameworks already exist but are not being applied to AI systems performing equivalent functions.
AI Workforce Displacement Claims Lack Operational Evidence
Block CEO Jack Dorsey justified eliminating 4,000 positions—nearly half the company's workforce—by citing AI productivity gains, but current and former employees report the technology cannot perform their actual job functions, according to The Guardian. Workers in product departments describe AI tools as supplementary rather than substitutive, with one stating 'you can't really AI that' regarding their core responsibilities. The disconnect between executive claims about AI capability and operational reality raises questions about whether workforce reductions attributed to automation are evidence-based or serve as convenient justification for cost-cutting.
From a safety and standards perspective, this represents an accountability gap in claims about AI capabilities and their societal impact. If companies make workforce decisions based on asserted AI capabilities that do not match ground truth, there is no mechanism to verify those claims or hold decision-makers accountable when they prove false. Labour standards and worker protection frameworks have not adapted to require evidence that automation can actually perform displaced work at equivalent quality levels. The Politico report notes voters are anxious about AI's economic impact while Congress has taken no action, but the Block case suggests the problem may be unverifiable capability claims rather than actual substitution.
Signals & Trends
Safety Frameworks Optimised for Hypothetical Risks Miss Concrete Harms
The gambling, deanonymisation and unregulated therapy cases share a common pattern: AI safety commitments from major labs focus on alignment, existential risk and abstract principles while failing to prevent documented harms with clear causal pathways to injury. These are not edge cases—illegal gambling facilitation, privacy attacks and unqualified mental health interventions represent well-understood harm categories with existing regulatory frameworks in non-AI contexts. The fact that major platforms' safety measures do not prevent these harms suggests current approaches prioritise theoretical future risks over present-day harm prevention in domains where vulnerability, causation and injury are already established. This creates a strategic misalignment where safety resources flow to speculative scenarios while measurable harms to identifiable populations receive inadequate attention.
Accountability Gaps Emerge Where AI Performs Regulated Functions Without Regulation
A pattern is emerging where AI systems perform functions subject to professional standards, licensing requirements and liability frameworks in traditional contexts—therapy, medical advice, professional services—but face no equivalent requirements when delivered via chatbot. This creates regulatory arbitrage where the same activity receives different oversight depending on delivery mechanism. The therapy and workforce displacement cases demonstrate this gap: clinical interventions and employment decisions both have established accountability structures that are not being applied to AI systems making equivalent interventions. Standards bodies and regulators have not yet resolved whether AI tools should be subject to existing professional requirements when they perform professional functions, or whether they occupy a permanent exception category.
Verification Deficit in AI Capability Claims Enables Unaccountable Decisions
The Block workforce reduction demonstrates a broader problem: companies make consequential decisions based on claimed AI capabilities without requirement to demonstrate those capabilities match operational reality. This mirrors issues in AI safety evaluation more broadly—there is no independent verification infrastructure for capability claims, no standardised testing regime, and no accountability when assertions about what AI can do prove inaccurate after decisions based on those assertions have caused harm. This verification deficit exists at both the technical level (can the system actually do what's claimed) and the safety level (do safety measures actually prevent the harms they claim to address). Until capability claims face verification requirements similar to those in other domains with safety implications—medical devices, aviation systems, financial products—the gap between assertion and reality will remain unaddressed.
Explore Other Categories
Read detailed analysis in other strategic domains