I want to start a mail or petition campaign on AI governance in the EU. Below is a draft letter, addressed to members of the EU parliament. Comments/feedback welcome. If you believe it is a good proposal, and you live in the EU, feel free to send the letter to your representatives.
Subject: Strategic proposal: steering European AI compute policy toward safe paradigms and avoiding a frontier arms race
Dear Member of the European Parliament,
I am writing to urge the European Union to adopt a forward-looking, safety-first strategy regarding public compute investments and data center infrastructure. To prevent risks associated with AI misuse and misalignment, and to avoid fueling a destructive global frontier arms race, Europe should strictly condition infrastructure funding and data center deployment on safer-by-design architectures, rather than scaling up training for frontier models using the current dominant paradigm.
1. The growing risks of the dominant paradigm
The prevailing frontier-AI paradigm, based predominantly on large autoregressive models combined with increasingly sophisticated post-training (reinforcement learning), reasoning and agentic scaffolding (autonomous planning and tool use), presents fundamental security and alignment vulnerabilities. As these agentic models gain real-world tool access, their failure modes become increasingly dangerous:
- Misuse & exploitation. Recent incidents and controlled evaluations demonstrate that increasingly autonomous AI systems can identify vulnerabilities, execute multi-step cyber operations and, in some cases, operate with limited human supervision. A most worrying example is the OpenAI/Hugging Face hacking incident by AI agents in July 2026.
- Reward hacking & misalignment: Optimization against imperfect objectives can produce specification gaming or reward hacking, and increasingly autonomous systems may develop strategies that exploit weaknesses in their objectives or oversight mechanisms. AI-models may use deception and develop instrumental drives, such as resisting shutdown or gathering unnecessary resources. These risks have been demonstrated experimentally, and their prevalence and severity in future frontier systems is uncertain.
2. Strategic policy: focus infrastructure on inference and safe paradigms
Rather than spending billions in public subsidies to build hyperscale training facilities that replicate this inherently risky paradigm, Europe should establish a clear policy framework:
- Pause public subsidies for frontier scaling of the current paradigm: Do not allocate EU funds, tax incentives, or grid capacity to build mega-data centers intended solely to scale auto-regressive RL agents to higher parameter counts.
- Designate European compute for safe paradigms: Dedicated public training infrastructure should be exclusively reserved for models built under new, inherently safer training paradigms.
- Focus general infrastructure on inference & open deployment: European data center expansion should primarily support inference for existing models and localized, high-efficiency applications that benefit European industries directly without compounding frontier scaling risks.
3. Safer alternative paradigms
The Commission's AI Gigafactory programme and the Cloud and AI Development Act are explicitly intended to expand European capacity for training, inference and fine-tuning advanced frontier models. However, the EU should redirect its compute policy towards architectures designed to reduce agency and improve verifiability. Emerging research demonstrates that AI capabilities can be developed without the systemic risks of standard agentic LLMs. Two examples of theoretical alignment frameworks that are allowed on European training data centers are (although they currently lack a demonstrated guarantee of corrigibility and alignment):
- Yoshua Bengio’s "Scientist AI": Focuses on goal-free probabilistic inference, causal world models, and Bayesian uncertainty estimation. By separating factual prediction from agentic goal seeking, it is designed to reduce or avoid the mechanisms that can give rise to reward hacking, strategic deception and self-preservation.
- Stuart Russell’s Cooperative Inverse Reinforcement Learning (CIRL): Incorporates explicit uncertainty about human preferences directly into the mathematical objective and creates incentives for the machine to learn from humans. Under CIRL, the AI treats human behavior as evidence rather than an absolute target, giving it a natural mathematical incentive to remain deferential and allow human intervention or shutdown.
The EU should use the Gigafactory programme to create a European testbed for architectures whose safety properties differ fundamentally from conventional agentic systems. The EU should prioritize developing non-agentic AI (e.g. Scientist AI, probabilistic world models), verified AI (e.g. theorem-proving systems, proof-producing AI, formally verified controllers), corrigible AI (e.g. CIRL-inspired systems, uncertainty over human preferences, explicit shutdown mechanisms, human-in-the-loop control) and safety infrastructure that goes beyond the minimum AI Act requirements (e.g. independent evaluation models, AI-on-AI testing, red-teaming networks, AI monitoring systems, cyber-defense models, model auditing, formal verification, deception detection, interpretability, anomaly detection). Next to funding competing safety architectures, evidence should be required that the resulting systems satisfy measurable safety criteria.
Conclusion: Europe as a global leader in safe AI
It is uncertain whether increasingly autonomous frontier scaling is the safest route to AI progress, while alternative architectures may offer qualitatively different safety properties. Therefore, public policy should avoid putting all of Europe's technological capital behind one high-risk trajectory. By directing compute infrastructure toward safety-critical inference, high-value non-agentic applications and training of mathematically verifiable, safer-by-design architectures (such as Scientist AI and CIRL), the European Union can establish true technological leadership. This strategy protects European citizens from systemic AI risks, prevents participating in an unsustainable international training arms race, and fosters a sustainable, trustworthy AI ecosystem for Europe’s future.
Thank you for your time and leadership on this critical issue.
Sincerely,
