AskVantage

AI and intelligence

An AI invented over 40,000 new chemical weapons in just 6 hours but it’s OK

AI can be used to do great good, but it can also be weaponised to create great harm. This is an example of the latter, and we're just getting started - are bioweapons next? Yes.

Key takeaways

  • The study was a prime demonstrated of how AI could possibly cause negative implications not just on isolated populations, but to the whole world, even when its closely governed by humans.
  • Specialists led the research from North Carolina startup Collaborations Pharmaceuticals.
  • The aim of the examination, according to Daily Mail, was to determine how an AI could switch its potential solutions from one track to another and suggest answers that are far more lethal than what people expect from it.
Cite or link to this article

Griffin, M. (2022) 'An AI invented over 40,000 new chemical weapons in just 6 hours but it’s OK', 311 Institute, 1 April. Available at: https://www.311institute.com/an-ai-invented-over-40000-new-chemical-weapons-in-just-6-hours-but-its-ok/ (Accessed: 1 October 2026).

Artificial Intelligence (AI) now has a firm foothold in the battlefield as it pilots drones and co-pilots jets, crushes the best fighter pilots in battle time and time again in the US and China, controls swarms of increasingly sophisticated swarms of drones, missiles, and robots, and runs the occasional robot dog errand. Now though, after an AI created the world's first synthetic human genome - which tomorrow could be a genetically engineered biological weapon - in another twist an AI has just created over 40,000 chemical weapons, that included various compounds, in just six hours following a set of commands from the experts. And tomorrow thanks to Quantum computers that 6 hours might just be 6 milliseconds.

However, if your first reaction is to be worried and scream then don't be because elsewhere DARPA is working on a way to genetically engineer humans to protect us from biological and chemical weapons. So we can all sleep safely again. Yay.

Now back to the story ... In the new paper experts first witnessed how the AI analyzed potential chemical compounds that would be helpful to cure global diseases, and then in a surprising turn, watched as it developed a whole host of new chemicals based on chemical weapons formulas.

The study was a prime demonstrated of how AI could possibly cause negative implications not just on isolated populations, but to the whole world, even when its closely governed by humans. Specialists led the research from North Carolina startup Collaborations Pharmaceuticals.

The aim of the examination, according to Daily Mail, was to determine how an AI could switch its potential solutions from one track to another and suggest answers that are far more lethal than what people expect from it.

The team focused on the different ways that an AI could be abused and its corresponding effects when pushed to work on negative tasks by human influence rather than positive ones.

The AI study also kind of then accidentally discovered that an extremely intelligent algorithm could develop thousands of new chemical combinations when taken advantage of via absurd commands. The Verge reported that most of the chemicals assembled by the AI were similar to the most dangerous chemical weapons still in theory today.

Some of the chemical compounds invented by the system were pretty close to the properties of venomous agent X or VX, which is known as a toxic nerve agent that initiates a human body to twitch via small doses.

According to the authors, the discovery is truly terrifying, considering how quickly and easily the AI consolidated information from chemicals available today and curated a dataset that included toxic chemicals. These chemicals are dangerous and not a laughing matter, having destructive capacities similar to biological weapon formulas we know and still theorize today.

Collaborations Pharmaceuticals expert and lead author of the study Fabio Urbina explained that their team has gone through a lot of molecule datasets to check if they are toxic or not. The VX, which is an acetylcholinesterase inhibitor, caught their attention.

Acetylcholinesterase is utilized by neurons whenever there are activities related to the muscles of the body. It is responsible for telling the brain how to move the chunks wrapped around the entire physiology of humans.

When VX is administered to a person, their lung muscle, also known as the diaphragm, will stop from moving, and this will start the organ to become paralyzed and void its functions.

The idea to slip the switch of the AI to cause negative outcomes as presented in the Convergence Conference, an international council organized by the Swiss Federal Institute for Nuclear, Biological, and Chemical Protection.

The study was published in Nature Machine Intelligence, titled "Dual use of artificial-intelligence-powered drug discovery."

FAQ

Why does this matter?

AI can be used to do great good, but it can also be weaponised to create great harm. This is an example of the latter, and we're just getting started - are bioweapons next? Yes.

Matthew Griffin

About the author

Matthew Griffin Founder, 311 Institute

Matthew Griffin is a multi-award winning Futurist and expert in Disruption and Innovation, Geopolitics, Leadership, and Technology, who NASA have described as a "walking encyclopaedia of the future" and a "futurist Polymath."

Read full bio

Matthew Griffin is a multi-award winning Futurist and expert in Disruption and Innovation, Geopolitics, Leadership, and Technology, who NASA have described as a "walking encyclopaedia of the future" and a "futurist Polymath." 15-time best selling author of the "Codex of the Future" series, Matthew is the Founder and Futurist in Chief of the 311 Institute, a global Futures and Deep Futures advisory firm working with royal households, world leaders, G7, G20, and G77 governments, NGOs, and multi-national mid and mega cap firms to help them explore, shape, and lead the next 50 years of business and society.

An award-winning YouTube creator with over a million followers, with an unrivalled global reach and impact, Matthew is a highly sought-after international keynote speaker, lecturer, and mentor who collaborates with global leaders through the United Nations Alliance of Civilizations (UNAOC) and United Nations General Assembly (UNGA) to shape pivotal initiatives such as the UN’s AI for Humanity program, the United Nations Conference of the Parties (UN COP), and the World Economic Forum in Davos.

As the former Global Head of Cloud, National Security, and Enterprise Sales for companies including Atos, Dell-EMC, and IBM, Matthew has a proven track record of building multi-billion dollar business units and turning failing divisions into market leaders. His ability to identify, analyse, and communicate the implications of hundreds of emerging technologies and trends is unparalleled, and his insights are trusted by many of the world’s most respected organisations, including ABB, Accenture, Adidas, AON, ARM, BCG, Centrica, Citi, Coca-Cola, Dentons, Deloitte, Dow Jones, EY, Google, KPMG, Lego, Legal & General, LinkedIn, Microsoft, PepsiCo, Qualcomm, RWE, Samsung, Siemens AG and Siemens Energy, T-Mobile, UBS, VISA, Walmart, Workday, Worldpay and many others.

Regularly featured in the global media including the AP, BBC, Bloomberg, CNBC, Discovery, Forbes, Khaleej Times, Telegraph, TIME, ViacomCBS, WIRED, and the WSJ, Matthews mission is to help organisations create a fair and sustainable future whose benefits are shared by everyone irrespective of their ability, background, or circumstances.

What future do you need to see?

Choose one to get started on AI and intelligence and the future of your organisation.

Where should Matthew reply?

Takes 30 seconds. No obligation. Matthew replies quickly. Privacy

Tag Cloud

Starburst opens that technology on the interactive 311 Starburst.

Sources and further reading

  1. Collaborations Pharmaceuticals collaborationspharma.com
  2. AI came thousands chemical weapons just hours task scientists dailymail.co.uk
  3. Ai new possible chemical weapons generative models vx theverge.com
  4. venomous agent X emergency.cdc.gov
  5. Nature Machine Intelligence nature.com

Source: first published by the 311 Institute on 1 April 2022. Cite as: Griffin, M. (2022). An AI invented over 40,000 new chemical weapons in just 6 hours but it’s OK. 311 Institute. https://www.311institute.com/an-ai-invented-over-40000-new-chemical-weapons-in-just-6-hours-but-its-ok/

You are welcome to quote this article with credit and a link to the original.

Book a Keynote