News chronological

The sequence involves a bad actor using a powerful model to map biological vulnerabilities and identify genetic tweaks to make a pathogen more transmissible, harder to detect, or immune to treatments. The model generates candidates using genomic datasets, which are then synthesized using cheap, accessible gene-editing tools. The resulting pathogen could then spread before being detected by existing biosurveillance systems.

Open-source models can be used and adapted by anyone in private settings outside of any regulatory regime. While frontier models may have more compute power to create something novel, open models fine-tuned by individuals can evade the guardrails that frontier labs might bake into their systems.

Sam Altman (OpenAI), Dario Amodei (Anthropic), Demis Hassabis (Google DeepMind), Mustafa Suleyman (Microsoft), and Alexandr Wang (Meta) cosigned an open letter warning of the risk of AI-derived bioweapons and calling for increased safeguards.

Researchers defined a "catastrophic" outcome as an event resulting in more than 1 million deaths or $100 billion in damage. Assisting in the construction of chemical or biological weapons is categorized within this top-ranked risk.

The study, which involved 272 researchers ranking 24 top AI risks, assessed a 12% chance of AI's dangerous capabilities producing a catastrophic outcome by 2030 with mitigation efforts in place. Without those mitigation efforts, the researchers stated the chances exceed 20%.

Most AI architects and leaders identify a killer pathogen that spreads too silently, widely, and quickly to stop as their greatest worry. This concern stems from the belief that advanced AI models could assist bad actors in creating deadly pathogens before detection and prevention systems are perfected.