Understanding risk and improving systemic resilience.

Frontier AI systems are rapidly advancing at consequential real-world agentic tasks, and already pose significant threats to our existing infrastructure. AI capabilities alongside appropriate control measures and societal safeguards are poorly understood. We advance the frontier of safety, security, and resilience through our research efforts.

Our affiliated researchers have published on a broad range of topics in technical AI safety, cooperative AI, and technical AI governance in top global venues such as ICML, ICLR, AAAI, and NeurIPS. Previously, we have developed AI evaluations research with the UK AI Security Institute, and are currently in a research partnership with Apart Research, investigating AI's ability for harmful manipulation of human behaviour.

Our current research bets

Multi-agent

With increasingly autonomous agents being deployed, novel risks from multi-agent interactions arise. Large systems of multiple AI actors pose unique threats that need to be empirically characterised and mitigated. Single-agent safety and alignment techniques will be insufficient for addressing these complex interactions.

Multi-polar

The frontier of AI is not dominated by a single actor, as progress is costly to push, and second movers appear to catch up relatively easily. We do not endorse racing towards greater capabilities, or assume that any single actor is responsible enough to act on behalf of the rest of humanity, but support interventions that make collective governance of this technology possible and enable distributing benefits broadly.

Decentralised defence

While distributing AI capabilities involves its own risks, it also enables decentralised defence against malicious actors (a recent example). We believe that we should be developing methods to improve the resilience of our infrastructure to handle an expanding threat landscape, and that privacy-preserving AI can be a useful tool in this endeavour.

Research projects and publications.

These are some notable research outputs that our team and community members have produced. In our in-house team, we have a focus on technical AI governance methods, mainly evaluations. You can see the research outputs of the Cooperative AI Research Fellowship on their posters page.

Research	Status	Authors	Venue	Link
Predicting Long-Horizon Task Performance using Proxy Evaluations	Under Review	Jaco Du Toit, Leo Hyams, Daniil Anisimov, Brendon Chikavanga, Joshua Stein, Magda Dubois, Cozmin Ududec, Samuel F. Brown	NeurIPS 2026 Main Conference, Evaluations & Datasets track	-
Precursors, Proxies, and Predictive Models for Long-Horizon Tasks	Published	Jaco Du Toit, Leo Hyams, Daniil Anisimov, Samuel Brown	Workshop on Evaluating the Evolving LLM Lifecycle: Benchmarks, Emergent Abilities, and Scaling at NeurIPS 2025	Open
Autonomous replication and adaptation evaluations Synthetic Environment Detection Eval Situational Awareness Eval RL Manipulation Eval Efficient Test-Time Chain of Thought Eval	Accepted	Jaco Du Toit, Leo Hyams, Asa Strickland Cooper, Joshua Olive, Noah De Nicola, Benjamin Sturgeon, Jay Bailey, Alexandra Abbas, Celia Waggoner	UK AISI ARA Stream Bounty Programme	-
HumanAgencyBench: Do Language Models Support Human Agency?	Published	Benjamin Sturgeon, Leo Hyams, Daniel Samuelson, Ethan Vorster, Jacob Haimes, Jacy Reese Anthis	HEAL workshop at CHI 2026	Open
When Role-playing, Do Models Believe What They Say?	Preprint	Benjamin Sturgeon, David Africa, Sid Black	-	Open