Liberatus AI whips world’s top poker players to take $1.7m pot

An artificial intelligence (AI) called Libratus from Carnegie Mellon University (CMU) has beaten four of the world’s best poker players in a gruelling 20 day tournament that culminated late on Monday. The Brains vs Artificial Intelligence competition saw four human players - Dong Kim, Jason Les, Jimmy Chou and Daniel McAulay - spend 11 hours each day stationed at computer screens in the Rivers Casino in Pittsburgh battling a piece of software at no-limit Texas Hold’em, a two-player unlimited form of poker.

Libratus out manoeuvred them all to win over $1.7m in virtual chips - fortunately for the pro's no actual money changed hands, but nonetheless it's being taken as another crushing defeat for humanity, and a major milestone for artificial intelligence.

Machines have already become smart enough to beat humans at other games such as chess and Go, but poker is more difficult because it’s a game with imperfect information. With chess and Go, each player can see the entire board, but with poker, players don’t get to see each other’s hands. Furthermore, the AI is required to correctly interpret misleading information and bluff in order to win - the latter being something that researchers are increasingly worried about as AI gets more tightly integrated with our world's digital fabric.

“This challenge is so huge and complicated that it’s been elusive to AI researchers until now,” said Carnegie Mellon University professor of computer science Tuomas Sandholm, "the international betting sites put us as 4-1 underdog and the humans expected to win.”

But they didn't, not even close.

 

“They put up the best fight they could,” said Sandholm.

The secret to Libratus's success apparently was not just it's access to huge amounts of computing power, but it's new and enhanced algorithms that found new ways to deal with imperfect or hidden information.

“We didn’t tell Libratus how to play poker. We gave it the rules of poker and said ‘learn on your own’,” said Brown.

The bot started playing randomly but over the course of playing trillions of hands was able to refine its approach and arrive at a winning strategy.

Late each day, after the poker play ended, Sandholm would connect Libratus to the Pittsburgh Supercomputer Center’s Bridges computer to run algorithms to improve its strategy overnight and in the morning he would spend two hours getting the newly enhanced bot back up and running.

At the same time, the humans are playing until 10pm, eating dinner, then spending a few hours reviewing the AI’s hands in the game and tweaking their strategy, getting to sleep at around 2am - the schedule has been gruelling for the poker pros.

“Libratus turned out to be way better than we imagined. It’s slightly demoralizing,” said Jason Les, who played against Claudico, Liberatus' predecessor two years ago.

“If you play a human and lose, you can stop, take a break. Here we have to show up to take a beating every day for 11 hours a day. It’s a real different emotional experience when you’re not used to losing that often,” said Les.

It’s not all bad for Les and his team mates though, they get to split a $200,000 prize pot depending on how well they did relative to each other against Libratus. They've also learned from Libratus, thanks to the robot’s aggressive style of play that sees it make huge bets to win small prize pots.

“It’s just not something a human would normally do, but it forces you to be on your toes for each game,” said Les, "it’s almost like we’ve been shellshocked into being much stronger players. Nothing anyone does will seem that crazy any more.”

For Sandholm, seeing Libratus win has induced a “proud parent feeling”.

“When I see the bot bluff the humans, I’m like, ‘I didn’t tell it to do that. I had no idea it was even capable of doing that.’ It’s satisfying to know I created something that can do that.”

The algorithms that power Libratus aren’t specific to poker, which means the system could have a variety of applications outside of recreational games, from negotiating business deals to setting military or cybersecurity strategy and planning medical treatment - anywhere where humans are required to do strategic reasoning with imperfect information.

“Poker is the least of our concerns here,” said Roman Yampolskiy, a professor of computer science at the University of Louisville, “you have a machine that can kick your ass in business and military applications. I’m worried about how humanity as a whole will deal with that.”

For Sandholm, Libratus challenges preconceptions about machine intelligence versus human intelligence.

“People have this idea that poker is a very human game and that bots can’t bluff, for example. That’s totally wrong. It’s not about reading your opponent and trying to tell if they are lying, it’s about the cards and probabilities,” he said, "we are seeing a re-evaluation of the types of things machines can excel at – although I can’t see a computer writing a prize-winning novel any time soon.”

... Stay tuned for that one!