A girl wearing a mask over nose and mouth
Generative AI is biology’s new playground. The technology powering popular chatbots can also dream up new, entirely novel versions of life’s most basic molecules, from DNA to proteins.
Once the domain of highly trained specialists, relative novices can now design synthetic molecules using open source AI software. But ease of access is a double-edged sword. While lower barriers to entry might spur creativity or even yield new medicines, the technology could also be used for nefarious purposes, such as designing novel toxins.
In 2024, two experts wrote an essay highlighting the need for biosecurity in the field. One of them, David Baker at the University of Washington, earned a Nobel Prize for RoseTTAFold, an AI that predicts protein structures from their amino acid building blocks. The other, Harvard’s George Church, has long been at the forefront of genetic engineering and synthetic biology.
They argued we should embed a barcode into each new designer protein’s genetic sequence to form an audit trail that scientists can trace back to the protein’s origins.
But a genetic tracer alone isn’t enough. A Microsoft study found AI-designed genetic sequences often escape the biosecurity screening software used by companies synthesizing designer DNA. AI-generated proteins with alien DNA sequences confuse these programs. Anything with genetic bits previously labeled “safe” flies under the radar, even if it encodes a dangerous final product.
These early studies are raising awareness. They’re not meant to stymie progress or enthusiasm—scientists welcome ideas for self-regulation. But for AI-powered designer biology to grow responsibly and be used for good, argue Church and other experts in a new preprint, the right time to build comprehensive biosecurity is before something goes wrong, not after.
The Dual Use Dilemma
From individual proteins to DNA, RNA, and even entire cells and tissues, AI is now learning the language of biology and designing new building blocks from scratch.
These powerful AI systems don’t simply recognize patterns. They eventually generalize those learnings across biology to analyze and dream up hordes of molecules at a prompt. RFdiffusion2 and PocketGen, for example, can design proteins at the atomic level with specific health-altering purposes, like sparking biological reactions or binding to drugs.
Generative AI is also beginning to read and write RNA. Like DNA, RNA is composed of four genetic letters, but RNA treatments don’t mess with the genetic blueprint. This makes them an exciting way to tackle disease. Unfortunately, they’re hard to design. RNA folds into intricate 3D shapes that are often difficult to predict using older software.
“Generative AI models are uniquely suited” for the job of capturing these intricacies, which could bolster the field of RNA therapeutics, wrote the team.
But the same AI galvanizing the field can also be used to create dangerous biological material. A person intent on jailbreaking an algorithm can, for example, repeatedly write prompts a generative AI system would normally refuse but is tricked into answering through repetition.
The dangers aren’t theoretical. A recent study compiled a dataset of toxic and disease-causing proteins and challenged multiple popular AI protein design models to create new variants. Many of the generated proteins retained their toxicity and evaded biosecurity software. In another case, scientists developed a method to test algorithmic security called SafeProtein. They managed to jailbreak advanced protein-design models 70 percent of the time.
Beyond proteins, researchers developing a framework called GeneBreaker found carefully tailored prompts can coax AI to spit out DNA or RNA sequences resembling viruses, such as HIV. Another team produced 16 viable genomes for bacteria that infect viruses, known as bacteriophages. Some of the resulting phages outcompeted their natural peers.
Even drug discovery tools can be flipped to the dark side. In one case, researchers easily reconfigured an AI model trained to find antiviral molecules. Within hours the AI suggested a known nerve toxin as a potential drug candidate.
“This demonstrates how even well-intentioned AI models can be rapidly misused to design toxins, especially when safety constraints are absent,” wrote the team.
Embedded Safety
To address these risks, the authors argue we need rigorous frameworks and regulations at every step of the process.
Scientists are leading the charge, and governments are on board. Last year, the UK released guidance for gene synthesis screening that urges providers of DNA and RNA molecules to vet their customers and increase screening for potentially dangerous sequences. The US launched similar rules and included biosecurity in its AI Action Plan.
Meanwhile, the tech giants behind AI models in biology are echoing calls for broader oversight. Some have pledged to exclude all viral sequences that are potentially dangerous to humans from their training databases. Others have committed to rigorous screening for new designs.
These safeguards, although welcome, are fragmented.
To gain a broader picture of the biosecurity landscape, the new study interviewed 130 experts across industry, government, academia, and policy. They agreed on several themes. Most think AI misuse is an urgent concern in biology and advocate for clearer regulatory standards. Roughly half were highly skeptical of current screening systems, and a majority supported upgrades.
The authors wrote that securing generative AI for biology isn’t about “finding a single solution.”
“Instead, it requires building a fortress with multiple layers of defense, each designed to anticipate, withstand, and adapt to threats.”
They designed a roadmap based on that principle. The strategy’s primary defenses target three stages in the AI life cycle. The first step is about controlling who can access training data and different AI versions. The next would add moral training that fine-tunes AI output. And finally, “live fire drills” to stress test models could reveal ways the AI could go sideways.
For example, algorithms trained on viral genomes are useful for drug or vaccine development. But they would be restricted. Users would have to apply for access and log usage. This is similar to how scientists must record the use of controlled narcotics in research. A tiered access system would allow others to use a version of the tool trained on data without dangerous content.
Meanwhile, strategies used to ensure chatbots (mostly) behave could also keep biology-focused AI in check. Moral training would guide a model’s output such that it aims to match public health and biosecurity standards. Stress testing to pinpoint a model’s vulnerabilities, known as red-teaming, would simulate misuse scenarios and inform countermeasures. Finally, biosecurity systems won’t work in a vacuum. Increasingly sophisticated AI could benefit from greater biological or general context, in turn improving its ability to detect and raise red flags.
“An effective biosafety system is not a firewall, it is a living guardian,” wrote the team.
Awareness is only the first part of the story. Action is the next. Although a unified vision of AI biosecurity doesn’t yet exist, the team calls on the field to collectively stitch one together.
Shelly Fan
Dr. Shelly Xuelai Fan is a neuroscientist-turned-science-writer. She's fascinated with research about the brain, AI, longevity, biotech, and especially their intersection. As a digital nomad, she enjoys exploring new cultures, local foods, and the great outdoors.
