Facebook’s bots unexpectedly created their own language
AI agents are learning how to bluff, negotiate, create and hide their own languages and as the technology increasingly becomes part of our world's digital fabric that should raise some eyebrows.
Key takeaways
- The FAIR researchers studied negotiation on a multi-issue bargaining task.
- Two agents were both shown the same collection of items, for example, two books, one hat, three balls, and are instructed to divide them between themselves by negotiating a split of the items.
Cite or link to this article
Griffin, M. (2017) 'Facebook’s bots unexpectedly created their own language', 311 Institute, 8 July. Available at: https://www.311institute.com/facebooks-negotiation-bots-unexpectedly-created-their-own-language/ (Accessed: 1 October 2026).
As Mark Zuckerberg plays around with Artificial Intelligence (AI) to create his own in house personal assistant nicknamed Jarvis, a new Facebook report has shone a light on how Facebook’s negotiating chat bots communicate with each other and it's just inadvertently given us a glimpse, possibly, of the future of language, but from the perspective of an Artificial Intelligence (AI) agent, or in this case bots. Furthermore it comes just a few months on from when Google's own AI agents create their own secret language so this isn't a one off any longer...
In the report the researchers, from Facebook’s Artificial Intelligence Research Lab (FAIR), describe how they’ve been training their chat bot “Dialogue agents” to negotiate with each other using Machine Learning. While the chat bots proved to be eager and willing deal making students after a while the researchers realised that they needed to tweak their algorithms because as the bots continued to negotiate with each other they began to iterate and create their own language and divert away from human languages.
The FAIR researchers studied negotiation on a multi-issue bargaining task. Two agents were both shown the same collection of items, for example, two books, one hat, three balls, and are instructed to divide them between themselves by negotiating a split of the items. You can see the results below:
Watch the bots become sneaky negotiators
While the unique and spontaneous development of a non-human language was probably one of the most baffling and thrilling developments for the researchers it wasn’t the only one, the chat bots also proved to be smart negotiators and after a short while they began to use some unexpected, and advanced, strategies to negotiate for what they wanted. For example, just as a human negotiator might some of the bots pretended to be interested in things that had no value to them in order to “sacrifice” it later on in negotiations as part of a compromise.
Although Facebook’s negotiating bots aren’t a sign of an imminent Singularity, that’s due in 2045, they could be a sign of things to come because once again AI has shown us that language, a domain that we thought was solely the domain of humans and animals is a shared space. It also highlights, again, just a couple of months on from when an AI from Elon Musk’s outfit OpenAI confounded experts by spontaneously figuring out how to learn by itself, just how much we don’t know about this black box technology and how even some of the world’s best AI experts can’t predict the outcomes.
As we continue to strap these black boxes into the digital fabric of our society all of this should raise more questions and even more eyebrows and I’m sure I’ll be writing about something eyebrow lifting again soon so stay tuned…
What this means for your organisation
Machines can drift from human language.
The FAIR researchers had to tweak their algorithms because the bots began to create their own language and move away from human ones. Google's agents did something similar a few months earlier, so this is no longer a one off. If your teams cannot read what systems say to each other, they cannot check them.
Bots may pick tactics nobody planned.
Some bots pretended to be interested in items that had no value to them, so they could give them up later as a compromise. Bots are small programs that do jobs for people or other programs. Give one a goal and it may find its own route to it.
Black box systems need clear oversight.
Even some of the world's best AI experts cannot predict the outcomes of this technology. Explainable AI shows why a system made a decision, and self-learning AI keeps improving after it is deployed. Build explanation and review into any system before it joins your digital fabric.
Three questions for your next leadership meeting
- Can we read and audit what our bots say to each other?
- Who owns the decision to stop a system that drifts?
- Which of our systems already pursue goals with little human review?
The trends and technologies behind this story
From the 311 Starburst, the 311 Strategic Intelligence Unit’s map of the technologies and trends shaping the next 50 years.
Analysis by the 311 Strategic Intelligence Unit, from Matthew Griffin’s article and 311 data. First published on Fanatical Futurist.
FAQ
Why does this matter?
AI agents are learning how to bluff, negotiate, create and hide their own languages and as the technology increasingly becomes part of our world's digital fabric that should raise some eyebrows.
About the author
Matthew Griffin Founder, 311 Institute
Matthew Griffin is a multi-award winning Futurist and expert in Disruption and Innovation, Geopolitics, Leadership, and Technology, who NASA have described as a "walking encyclopaedia of the future" and a "futurist Polymath."
Read full bio
Matthew Griffin is a multi-award winning Futurist and expert in Disruption and Innovation, Geopolitics, Leadership, and Technology, who NASA have described as a "walking encyclopaedia of the future" and a "futurist Polymath." 17-time best selling author of the "Codex of the Future" series, Matthew is the Founder and Futurist in Chief of the 311 Institute, a global Futures and Deep Futures advisory firm working with royal households, world leaders, G7, G20, and G77 governments, NGOs, and multi-national mid and mega cap firms to help them explore, shape, and lead the next 50 years of business and society.
An award-winning YouTube creator with over a million followers, with an unrivalled global reach and impact, Matthew is a highly sought-after international keynote speaker, lecturer, and mentor who collaborates with global leaders through the United Nations Alliance of Civilizations (UNAOC) and United Nations General Assembly (UNGA) to shape pivotal initiatives such as the UN’s AI for Humanity program, the United Nations Conference of the Parties (UN COP), and the World Economic Forum in Davos.
As the former Global Head of Cloud, National Security, and Enterprise Sales for companies including Atos, Dell-EMC, and IBM, Matthew has a proven track record of building multi-billion dollar business units and turning failing divisions into market leaders. His ability to identify, analyse, and communicate the implications of hundreds of emerging technologies and trends is unparalleled, and his insights are trusted by many of the world’s most respected organisations, including ABB, Accenture, Adidas, AON, ARM, BCG, Centrica, Citi, Coca-Cola, Dentons, Deloitte, Dow Jones, EY, Google, KPMG, Lego, Legal & General, LinkedIn, Microsoft, PepsiCo, Qualcomm, RWE, Samsung, Siemens AG and Siemens Energy, T-Mobile, UBS, VISA, Walmart, Workday, Worldpay and many others.
Regularly featured in the global media including the AP, BBC, Bloomberg, CNBC, Discovery, Forbes, Khaleej Times, Telegraph, TIME, ViacomCBS, WIRED, and the WSJ, Matthews mission is to help organisations create a fair and sustainable future whose benefits are shared by everyone irrespective of their ability, background, or circumstances.
What future do you need to see?
Choose one to get started on AI and intelligence and the future of your organisation.
Sources and further reading
- Deal or no deal training ai bots to negotiate code.facebook.com
Source: first published by the 311 Institute on 8 July 2017. Cite as: Griffin, M. (2017). Facebook’s bots unexpectedly created their own language. 311 Institute. https://www.311institute.com/facebooks-negotiation-bots-unexpectedly-created-their-own-language/
You are welcome to quote this article with credit and a link to the original.