Google has taught its DeepMind AI to dream

What happens when you sleep?

Sleeping is about more than just helping our bodies rest. It helps us stabilise our memories, and REM sleep, the time when we dream the most, is the time when the brain integrates new pieces of memory to old ones, helping us solidify these into long term memories and if sleep and dreams can help keep the complex human body running, Google believes that the same can be applied for their AI, and now, the newest artificial intelligence (AI) system from Google’s DeepMind division does indeed dream - metaphorically at least, about finding apples in a maze.

Researchers at DeepMind wrote in a paper published online yesterday that they had achieved a leap in the speed and performance of a machine learning system. It was accomplished by, among other things, imbuing technology with attributes that function in a way similar to how animals are thought to dream.

The paper explains how DeepMind’s new system, named Unsupervised Reinforcement and Auxiliary Learning agent, or Unreal, learned to master a 3D maze game called Labyrinth 10 times faster than the existing best AI software. It can now play the game at 87 percent the performance of expert human players, the DeepMind researchers said.

"Our agent is far quicker to train, and requires a lot less experience from the world to train, making it much more data efficient," said DeepMind researchers Max Jaderberg and Volodymyr Mnih.

They said Unreal would allow DeepMind’s researchers to experiment with new ideas much faster because of the reduced time it takes to train the system. DeepMind has already seen its AI products achieve highly respected results teaching itself to play video games, notably the retro Atari title Breakout.

Labyrinth is a game environment that DeepMind developed, loosely based on the design style used by the popular video game series Quake. It involves a machine having to navigate routes through a maze, scoring points by collecting apples.

This style of game is an important area for AI research because the chance to score points in the game, and thus reinforce "positive" behaviours, occurs less frequently than in some other games. Additionally, the software has only partial knowledge of the maze’s layout at any one time.

One way the researchers achieved their results was by having Unreal replay its own past attempts at the game, focusing especially on situations in which it had scored points before. The researchers equated this in their paper to the way "animals dream about positively or negatively rewarding events more frequently."

The researchers also helped the system learn faster by asking it to maximize several different criteria at once, not simply its overall score in the game. One of these criterion had to do with how much it could make its visual environment change by performing various actions.

"The emphasis is on learning how your actions affect what you will see," Jaderberg and Mnih said. They said this was also similar to the way newborn babies learn to control their environment to gain rewards - like increased exposure to visual stimuli, such as a shiny or colourful object, they find pleasurable or interesting.

Jaderberg and Mnih, who are among seven scientists who worked on the paper, said it was "too early to talk about real-world applications" of Unreal or similar systems.

Mastering games, from Chess to trivia contests like the US gameshow Jeopardy!, have long served as important milestones in artificial intelligence research. DeepMind achieved what is considered a major breakthrough in the field earlier this year when its AlphaGo software beat one of the world’s reigning champions in the ancient strategy game Go.

Earlier this month DeepMind announced the creation of an interface that will open Blizzard Entertainment Inc’s science fiction video game Starcraft II to machine learning software. Starcraft is considered one of the next milestones for AI researchers to conquer because many aspects of the game approximate "the messiness of the real world," according to DeepMind researcher Oriol Vinyals. Unreal is expected to help DeepMind master the mechanics of that game.

DeepMind’s Unreal system also mastered 57 vintage Atari games, such as Breakout, much faster - and achieved higher scores - than the company’s existing software. The researchers said Unreal could play these games on average 880 percent better than top human players, compared to 853 percent for DeepMind’s older AI agent.

But on the most complex Atari games, such as Montezuma’s Revenge, Jaderberg and Mnih said the new system made bigger leaps in performance. On this game, they said, the prior AI system scored zero points, while Unreal achieved 3,000 - greater than 50 percent of an expert human’s best effort.