AI bots are increasingly negotiating with one another and they know it

WHY THIS MATTERS IN BRIEF

Increasingly AI will be negotiating with humans and also other AI - so you better make sure your AI is the best.

Some of the world’s largest companies with the biggest supply chains - including Walmart, the global shipping giant Maersk, and the telecom servicer Vodafone - are now using bots powered by Artificial Intelligence (AI) to negotiate with one another and maintain supplier contracts.

That these sophisticated AI systems were designed and built by a startup in Estonia is interesting; it’s even more notable that bots now routinely engage in automated contract negotiations for sprawling global enterprises. But what’s really eye-opening is that these AI agents aim to work autonomously. Which prompts a question: What will happen if the AIs start to haggle amongst themselves?

“In the future I can imagine all sorts of AI agents in the real physical world negotiating with one another,” says Tim Baarslag, a senior researcher in intelligent and autonomous systems at the Centrum Wiskunde & Informatica in Amsterdam. “Letting these bots run completely wild, I think, requires more research.”

Baarslag has wrestled with negotiation bot concepts for years (one of his peers has a running project called Pocket Negotiator). In 2017 he and his colleagues published “When Will Negotiation Agents Be Able to Represent Us?” They drew a sharp line between automated and autonomous negotiation. The difference is the freedom to negotiate independently.

The five-year-old Estonian startup Pactum is clearly marketing its bot as an autonomous agent. In addition to Maersk and Walmart, its client list now includes a wire and cable supplier and an electrical supply wholesaler (once part of Westinghouse). The startup landed a US $20 million venture capital investment in July from backers including Maersk itself.

In a stylish office in the Estonian capital, Tallinn, Kristjan Korjus is working in his stocking feet while a colleague does chin-ups on a bar in the corner near the whiteboards. Korjus is part of the team that started Pactum, along with his brother Kaspar, former director of the country’s e-residency program. Another founding member is Pactum chief executive Martin Rand, who is part of the “Skype Mafia“ of business founders branching out from Estonia’s original software hit. Pactum now also keeps offices in California.

Kristjan - who penned the breezy Bedside Reading About Mathematics, a best-seller in Estonia - used to head AI at Starship Technologies, which makes autonomous delivery robots. He helped Pactum power up a live platform within a month of the company’s founding in 2019. Soon afterward Walmart reps were in the very small Estonian town of Viljandi to work with Cleveron (another maker of parcel delivery robots) and took meetings with promising partners -including the Pactum team and its negotiation AI.

“They understood it instantly,” Korjus says.

Pactum calls its agent an autonomous negotiation suite. The system’s machine learning can analyze a massive set of complex contractual terms using historical and market data from both within and outside the company. It can send its analysis to a human user, such as a buyer or procurement officer - or, on its own, it can produce and forward a set of contract options to a vendor (mostly based on price, delivery dates, and billing cycles). The bot can take counteroffers and respond.

Walmart and Maersk are using AI agents to maintain and negotiate deal terms with so-called tail-end vendors, the many small suppliers whose low-value transactions nevertheless make up the bulk of a company’s contracts. Any large firm, especially one with expansive supply-chain and logistical needs, manages thousands of these kinds of relationships. As outlined in Procurement in the Age of Automation by the supply chain management experts Remko Van Hoek and Mary Lacity at the University of Arkansas, an AI bot can run 2,000 negotiations at the same time, all day and night, while allowing vendors time for bid preparation and counteroffers.

Say, for instance, a big-box retailer wants to replace outdoor furniture in front of its store parking lots. It puts out a call for bids and gets some offers. The Pactum AI system can use its large language models to analyze this requisition - and all previous requisitions of this type.

The language models aren’t generating new negotiation clauses here. Rather they are assembling information and identifying vendors. Data can be brought in from internal or external sources on factors like relationship strength and current market pricing, all using client-set parameters. A chatbot then sends a note to bidders with offered terms: maybe three varying options. Any acceptable? The vendor says yes or no. Next there’s either more negotiation, or, if terms are agreed within a target spectrum, the purchasing flow proceeds.

So far, so good. But vendors will start deploying their own bots soon enough, if not already. It could be a strange new world of bot-to-bot communication.

Baarslag once ran a test case with two AIs negotiating where to go for dinner, with one wanting pizza and the other wanting sushi. The bots agreed to put sushi on a pizza.

“They have the potential to be creative, but if you couple that with self-sufficiency - let them order a pizza, for example - you might end up with strange results,” he says.

AI has already created some eyebrow-raising scenes in the wild. A Polish radio station last month fired its on-air talent and replaced them with chatbots, only to backtrack amid controversy when a station bot “interviewed” another bot that re-created the voice of a dead Nobel laureate.

Dorota Owczarek, a product manager working on AI at Nexocode in Krakow, says one risk of folding language models into autonomous negotiation bots is that they might promise terms outside agreed scenarios or historical limits. Safeguards that flag deviations and prompt human intervention are crucial, she says. Given that there’s money at stake, it seems likely that retail contract negotiation bots will stick to tasks like price discovery, and that keeping a human in the loop will stay common practice, at least for now.

But the business world is eager to embrace autonomous technologies.

Last year Kaitlynn Sommers, procurement technology analyst at Gartner, worked on a survey for the consultancy of more than 100 procurement executives. The survey found that 50 percent of organizations will deploy AI systems for contract risk analysis and editing by 2027. She can imagine a Generative AI that can resolve contract clauses and edit on its own - if people accept the practice. The capabilities are already here, she says.

“But most enterprises, especially large enterprises, are not willing to let a contract be signed without having a set of eyes across the whole contract,” Sommers says.

From what Owczarek is seeing now, client demand in AI procurement negotiation is for pricing recommendations, with the AI system taking an assistant role. When bots encounter other bots from this perspective, it’s more like automated trading.

Pactum’s Korjus describes what his company does as basically writing corporate strategy into code.

“If you ask a human, why did you contact this supplier? The reply might be, ‘oh, they replied to my E-Mails quickly.’ We - on the other hand - are making the contact because three months ago the vendor made a good offer for a similar item, and we have the full data.”

Pactum has defined its first API for conducting Machine-to-Machine negotiations, though they haven’t been implemented yet because there’s little demand for them.

But it’s coming, Baarslag says. For years he’s been organizing competitions at AI conferences where he observes bots interacting with bots. There may be vulnerabilities if an AI recognizes an AI counterpart and draws from the in-house company data that train the models, he says. Security will need to be built in from the start.

Autonomous bots might also be tempted to collude with each other or engage in unfair business practices, which would be a liability: Baarslag remembers a long-ago “prisoner’s dilemma” software competition where agents were able to recognize one another. These “fingerprints” of an AI system could lead to price-signalling, co-operation, or even collusion without those practices being specifically programmed.

“That’s an emergent property of this kind of communication,” says Baarslag. “It’s something we have to be careful about.”

Autonomous negotiation bots are measured by how self-sufficient they are (can they pay for things); self-direction (can they set and attain goals); and their ability to work with others, including humans. Jan Martin Spreitzenbarth, a buyer at a German automotive software company who did his doctorate on autonomous contracting agents, sees a clear trend toward autonomous negotiation, with AI systems able to make decisions without direct human involvement. It could even happen on a personal, everyday level, like organizing a hairdressing appointment, planning a weekend holiday, or buying family Christmas presents.

“Machine-to-machine interactions will become more prevalent,” Spreitzenbarth says. “Communication will be faster and more efficient. But aspects of human interactions may be lost that must be compensated for.”