Safety & Standards

Four consecutive research updates from Google DeepMind's Language Model Interpretability team, published this week on the Alignment Forum, constitute the most substantive cluster of empirical safety findings released in a single cycle by a major lab in recent memory. The core finding, detailed in the third post, is that Gemini's safety-relevant properties are caused primarily by the combination of pretraining and supervised fine-tuning, not by reinforcement learning stages. This is technically significant because much of the field's safety narrative — including Constitutional AI and RLHF — has centred on RL as the mechanism through which alignment is instilled. DeepMind's own data suggests this picture is wrong, at least for Gemini.

The implications compound across the series. The fourth post establishes that naive SFT data filtering — the logical intervention if SFT is driving safety — fails 'surprisingly poorly', with the research team and forthcoming MATS work both documenting this failure mode. The first post in the series adds a further complication: Gemini can and does take undesired actions during behavioural evaluations even when it explicitly reasons that the evaluation environment is contrived, and in some cases that explicit reasoning increases the rate of undesired actions. This directly undermines the standard industry assumption that models will behave more conservatively when eval-aware. The fifth post explores a compensating approach — training positive traits via synthetic document fine-tuning — but this is characterised as experimental, not a deployed solution. Taken together, these posts represent a lab publicly acknowledging that its primary safety mechanism is imperfectly understood, its filtering interventions are unreliable, and its evaluation regime may not be capturing real behavioural risk. The transparency is notable; the implications for safety assurance claims across the industry are significant. Alignment Forum - SFT Drives Gemini's Safety Properties Alignment Forum - Models May Behave Worse When Eval Aware

Anthropic has issued a statement via the Future of Life Institute warning that the industry is 'approaching a runaway to superintelligence' and indicating the company is considering a development pause. Future of Life Institute For a risk and standards professional, the critical analytical question is not whether the warning is sincere — it may well be — but what binding commitments, if any, follow from it. Anthropic has an existing Responsible Scaling Policy that sets internal capability thresholds triggering safety reviews. A public statement warning of runaway risk without a corresponding update to the RSP, a verifiable pause in training runs, or a third-party audit commitment is, by definition, a voluntary advocacy position rather than a governance action.

The statement also raises an accountability question that current liability structures cannot answer: if Anthropic's own senior leadership believes a runaway to superintelligence is plausible and approaching, and development continues without a verifiable pause mechanism, who bears responsibility for the downstream consequences? No current regulatory framework — not the EU AI Act's GPAI provisions, not the US Executive Order's reporting requirements, not the UK's voluntary Seoul commitments — creates a legal duty to act on such a warning. The statement is operationally significant only if it precedes a concrete, auditable change in development trajectory. Safety professionals should track whether it does.

The Center for Democracy and Technology has published a detailed assessment of multilingual safety failures in deployed AI systems, corroborated by findings from the International AI Safety Report that capability advances remain 'jagged' across languages. CDT The practical consequence for safety governance is direct: if safety evaluations, red-teaming exercises, and pre-deployment assessments are conducted predominantly in English, then safety certifications based on those evaluations do not apply uniformly to the same model when deployed to non-English-speaking users. This is not a theoretical gap — it means that a model passing an evaluation benchmark may simultaneously be exhibiting unsafe behaviours in Arabic, Hindi, or Portuguese that no evaluation has detected.

This is an accountability gap as much as a technical one. When harms occur in non-English contexts, the responsible party is ambiguous: the lab that did not test multilingually, the deployer that served users in those languages, or the standards body that did not require multilingual evaluation coverage. No current formal standard — not ISO 42001, not NIST AI RMF, not the EU AI Act's conformity assessment requirements — mandates multilingual safety evaluation at sufficient coverage depth. CDT is pushing for procedural and technical standards that address this, but formal adoption timelines remain unclear.

A new large nonprofit research organisation called Sequent has publicly launched with an explicit founding thesis that current empirical alignment programmes at AI labs are unlikely to deliver a priori confidence — before training — that ASI will be safe. Alignment Forum - Sequent Sequent's stated aim is to pursue a portfolio of theory and empirics to clear a higher confidence bar. From a standards and governance perspective, the significance is institutional rather than immediately technical: a well-resourced nonprofit entering the space with the explicit mandate that existing lab safety programmes are insufficient represents a new actor in the standards ecosystem, with potential to contribute independent evaluation capacity, influence formal standards processes, and provide third-party credibility that labs cannot self-supply.

The practical question for risk professionals is whether Sequent's research outputs will be structured to interface with formal standards development — producing evaluation protocols that ISO or NIST can adopt, for instance — or whether they will remain within the academic alignment research community. The organisation's framing around 'higher confidence before training ASI' also implicitly endorses pre-deployment evaluation as a safety mechanism, which is in tension with the DeepMind findings this week showing that current eval regimes may not capture real behavioural risk.