The Pentagon hires the world’s best poker playing AI to play War Games

Over the past year the US Pentagon has installed an autonomous Artificial Intelligence (AI) Robo-hacker called Mayhem to monitor and protect its most critical systems from cyber attacks, and announced the creation of an AI that will “monitor the whole world for threats.” And if all that sounds like the beginnings of Skynet, the AI in the Terminator films that took over the world, using nukes and Terminator robots, robots with “human skin over a metal endoskeleton,” something that the University of Oxford also recently managed to re-create, then you might be onto something.

Now though, not satisfied with what they have created, the Pentagon has now taken an AI poker bot called Liberatus, which made headlines last year for beating the world’s top four human players at no limits Texas Hold ‘Em to win a $1.8million pot and adapted it for military use. And if that sounds odd, it actually makes perfect sense, because Poker is a game you have to try to win based on bluff and imperfect knowledge, and that almost perfectly describes war.

Libratus, which is Latin for balanced, was created by researchers from Carnegie Mellon University to test ideas for automated decision making based on game theory. Early last year, the professor who led the project, Tuomas Sandholm, founded a startup called Strategy Robot to adapt his lab's game-playing technology for government use, such as in war games and simulations used to explore military strategy and planning.

Late in August, public records show, the company received a 2 year contract of up to $10 million with the US Army. It is described as “in support of” a Pentagon agency called the Defense Innovation Unit, created in 2015 to woo Silicon Valley and speed US military adoption of new technology.

Libratus’ defeat of poker pros in 2017 was seen as a milestone in AI because the card game has complex features lacking in the board games most prominently mastered by computers. In chess and Go, every piece is exposed for both players to see, making them what are called perfect information games. In poker, not all cards are visible, meaning that, as in many real-life scenarios, some information needed to calculate the true state of play is unknown.

Libratus was built on a technology called computational game theory, and it won more than $1.8 million in play money from the poker champions by calculating how they might respond to its decisions. The software devised powerful betting strategies and even showed the ability to bluff.

Sandholm says that approach can be applied to many other games, and also military simulations. War gaming exercises typically test only small numbers of strategies for imagined opponents, even when run as computer simulations.

“That opens yourself up to a lot of exploitation, because the real adversary may not play according to your assumptions,” Sandholm says.

Sandholm declines to discuss specifics of Strategy Robot’s projects, which include at least one other government contract. He says it can tackle simulations that involve making decisions in a simulated physical space, such as where to place military units. The Defense Innovation Unit also, funnily enough, declined to comment on the project, and the Army did not respond to requests for comment.

Libratus’ poker technique suggests Strategy Robot might deliver military personnel some surprising recommendations. Pro players who took on the bot found that it flipped unnervingly between tame and hyper-aggressive tactics, all the while relentlessly notching up wins as it calculated paths to victory.

“It's weird because it doesn't seem that it overwhelms you, but then you look at the score and you realize what's happened,” Sandholm says.

Greg Allen, an adjunct fellow at think tank the Center for a New American Security, says the type of technology that powered Libratus could make war-gaming and simulation exercises more useful, and that’s before the AI eventually gets plugged into the big old military decision making machine for real out on some future battlefield.

“[War games] are still far from real, but they’re a decent enough proxy for what happens in the real world,” he says. All the same, the results will likely remain just one component of strategy planning and research, he says, because the world is much more complex and messy than the scenarios even the best AI technology can master.

Strategy Robot isn’t the Pentagon’s only new foray into AI enhanced game theory. Its research agency DARPA is also starting a program to explore how the technology can be applied to military decision making. Michael Wellman, a professor at the University of Michigan, says his group is working on applying computational game theory to cyber security under that program. He says Libratus can be seen as a sign that the technology is maturing.

“It really is time to try this in some more real domains,” he says. “The breakthrough in poker was just so striking, and things are going quickly with other games.”

In addition to Strategy Robot, Sandholm has founded a second startup called Strategic Machine, which is deploying his game-solving techniques in commercial settings, such as electricity markets, sports, and making computer-controlled players in videogames wilier adversaries. Both companies are bootstrapped, have roughly six employees, and are profitable, Sandholm says.

Back in his CMU computer lab, Sandholm is also thinking about how to make his technology more portable. When Libratus took on the poker pros, it ran on the Bridges supercomputer at the federally funded Pittsburgh Supercomputing Center, Sandholm says his startups have also used supercomputers. At NeurIPS, the world’s largest AI conference last month, he and his collaborator on Libratus, Noam Brown, presented a paper on a less powerful but more compact poker bot called Modicum that can run on a single server.

“In some applications you need it to be miniaturized, if it's onboard something,” Sandholm says. “Some platforms can't carry big computers.”

The Pentagon is pushing to make broader use of AI technology. In 2017, then US defense secretary James Mattis lamented that his department lagged behind technology companies in the adoption of technologies like machine learning. That same year, the Pentagon started a program called Project Maven, intended to employ commercially available AI techniques on US missions. Its initial project used machine learning to flag objects in drone surveillance video, with help from AI-savvy startups and large companies including Google.

Other nations, too, are exploring military uses of AI. Russian president Vladimir Putin has said that whoever leads in AI “will become the ruler of the world.” Military applications also feature prominently in China’s national AI strategy, some of which are now feeding through into the creation of artificially intelligent cruise missiles and a wide range of semi-autonomous weapons platforms.

Unsurprisingly the growing weaponization of AI unsettles some technologists who are advancing the underlying technology, with some of Google’s AI researchers joining the thousands of employees who protested against the company’s work on Project Maven, but Sandholm believes concerns about US military use of AI are overblown.

“The technology is important to help the Pentagon keep the US safe and improve operational efficiency,” he says, before adding. “I think AI's going to make the world a much safer place.”