Safety & Standards
Top Line
The Pentagon designated Anthropic a supply-chain risk after the company refused military control over its AI models for autonomous weapons and domestic surveillance, marking the first time such designation has been applied to a US-based AI firm rather than adversarial nation companies like Huawei.
OpenAI released an AI agent designed to identify security vulnerabilities, with Claude already demonstrating capability by finding 22 vulnerabilities in Firefox including 14 high-severity flaws, raising questions about whether AI-assisted security testing creates net safety improvements or merely accelerates the vulnerability discovery arms race.
Draft US Commerce Department regulations would require permits for global AI chip shipments anywhere without American approval, while the Pentagon published strict new guidelines mandating civilian government AI contracts make models available for 'any lawful use', effectively forcing safety-focused providers to choose between principles and federal contracts.
The Iran conflict is exposing data centre infrastructure as military targets and accelerating AI deployment in warfare without adequate democratic oversight or multilateral controls, according to UN Secretary-General António Guterres, as geopolitical turbulence collapses the distinction between theoretical AI safety debates and real-world consequences.
Key Developments
Pentagon supply-chain designation creates precedent for AI safety accountability gaps
The Department of Defense officially designated Anthropic a supply-chain risk after negotiations collapsed over the military's demand for control over AI models, including their use in autonomous weapons and mass domestic surveillance, according to Bloomberg and TechCrunch. The designation, previously reserved for adversarial nation companies like Huawei, puts Anthropic at risk of losing a wide range of US government business beyond just the $200 million contract at stake. OpenAI immediately stepped in to accept the Pentagon deal, resulting in ChatGPT uninstalls surging 295 percent. Draft regulations published simultaneously by the Commerce Department mandate that civilian government AI contracts make models available for 'any lawful use', according to Financial Times, effectively forcing AI providers to choose between safety principles and federal revenue.
Heidy Khlaaf, chief AI scientist at AI Now Institute, told AI Now that existing guardrails for generative AI in high-stake decisions or surveillance are deeply lacking and easily compromised: 'It's highly doubtful that if they cannot guard their systems against benign cases, they'd be able to do so for complex military and surveillance operations.' Khlaaf separately told AI Now that despite Anthropic's safety-first reputation, it has always fallen short on attempts to prevent human harm. Microsoft, Google, and Amazon confirmed that Anthropic's Claude remains available to non-defense customers through their platforms, according to TechCrunch, but the supply-chain risk designation creates regulatory uncertainty about how long that access will continue.
AI vulnerability discovery accelerates without corresponding safety framework
Anthropic's Claude identified 22 separate vulnerabilities in Firefox over two weeks during a security partnership with Mozilla, with 14 classified as high-severity, according to TechCrunch. OpenAI simultaneously released an AI agent designed to help security teams find and patch vulnerabilities in large databases, according to Bloomberg, potentially displacing legacy cybersecurity firms. The tools represent meaningful technical progress in automated security testing but raise unresolved questions about asymmetric capability development.
CISA added three iOS vulnerabilities to its catalog of known exploited vulnerabilities under mysterious circumstances, according to Ars Technica, highlighting that advanced exploits continue emerging through traditional means even as AI-assisted discovery accelerates. No public framework exists for evaluating whether AI-powered vulnerability discovery creates net security improvements or simply ensures both attackers and defenders find flaws faster, leaving defenders perpetually behind the exploitation curve. President Trump signed an executive order directing officials to identify tools to combat cybercrime including fraud and extortion, according to Bloomberg, but the order contains no specific AI security provisions.
Military AI deployment outpacing oversight as Iran conflict becomes live testing ground
UN Secretary-General António Guterres warned that the speed of AI development and geopolitical turbulence is collapsing the distinction between theoretical arguments and real-world consequences, according to The Guardian, with the Iran conflict demonstrating intensified AI use in warfare and highlighting the urgent need for democratic oversight and multilateral controls. Tech Policy Press consulted experts at the intersection of technology policy, security, and international affairs on the role of technology in the expanding Middle East war, according to AI Now, though no consensus emerged on appropriate constraints. Data centres are now 'inevitable targets' in conflict, according to Sam Winter-Levy, a fellow at Carnegie Endowment for International Peace, who told Bloomberg that the conflict underscores risks of building AI infrastructure in the Gulf region.
No binding international framework governs AI weapons systems or restricts autonomous targeting decisions, and the Iran conflict is proceeding without the multilateral controls Guterres described as necessary. The US-China technology competition continues to override safety considerations, with draft Commerce Department regulations requiring permits for AI chip shipments globally, according to Bloomberg. The Pentagon's simultaneous push for unrestricted access to commercial AI models while restricting chip exports creates contradictory policy that prioritises military advantage over safety frameworks.
Consumer-facing AI safety failures reveal inadequate liability and transparency mechanisms
Grammarly's 'expert review' feature offers users writing advice 'inspired by' subject matter experts without permission, including recently deceased professors and living professionals who never consented, according to The Verge. The feature demonstrates that basic consent and attribution safeguards are absent from deployed commercial AI systems, with no clear liability framework when companies use professional identities without authorisation. No regulatory body has enforcement authority over such practices, and affected individuals have limited recourse beyond potential trademark or right-of-publicity claims that were not designed for AI-generated content.
The incident follows persistent evidence that AI systems cannot reliably prevent impersonation or unauthorised use of individuals' expertise, professional reputation, or identity. No standards exist for disclosure requirements when AI systems claim to represent or be 'inspired by' specific individuals, and companies face no mandatory pre-deployment testing to verify they have obtained necessary permissions. The lack of accountability mechanisms means companies can deploy features that appropriate professional identities, discover problems only through media coverage or user complaints, and face no penalty beyond potential reputational damage.
Signals & Trends
Federal procurement is becoming the primary mechanism for AI safety enforcement, bypassing standards development
The Pentagon's supply-chain risk designation and Commerce Department's 'any lawful use' requirement for civilian contracts demonstrate that procurement terms are establishing de facto AI safety standards faster than formal standards bodies like NIST or ISO can develop, evaluate, and adopt frameworks. This approach prioritises government access over safety properties and creates compliance requirements with no public comment process, technical evaluation, or appeals mechanism. Companies must now treat federal contracts as binding safety policy, not voluntary commercial relationships, but without the procedural protections that accompany actual regulation. The trend suggests safety professionals should monitor federal acquisition regulations and contract terms as closely as they track NIST guidelines or ISO standards development, since procurement requirements may establish market-wide norms before formal standards exist.
AI safety commitments are increasingly revealed as competitive disadvantages rather than market differentiators
Anthropic's safety-first positioning resulted in loss of a $200 million Pentagon contract, supply-chain risk designation, and OpenAI immediately capturing both the revenue and market validation for less restrictive approaches, while Claude's consumer growth surge suggests users reward capability over safety commitments. Multiple data points indicate that safety-focused approaches impose real costs—foregone federal contracts, delayed deployment, restricted use cases—without corresponding market rewards, regulatory protection, or competitive advantages. The pattern suggests that absent binding requirements, safety commitments function as handicaps that competitors can exploit rather than differentiators that customers value. This creates adverse selection pressure where the least restrictive providers capture market share and the most safety-conscious companies face punitive action, the inverse of typical market dynamics where quality commands premiums. Risk professionals should evaluate whether voluntary safety commitments remain strategically viable when they trigger government penalties rather than regulatory safe harbours.
Explore Other Categories
Read detailed analysis in other strategic domains