Uber’s breakthrough problem solving AI learns by creating then solving its own problems

Imagine an Artificial Intelligence (AI) that can solve any problem. Now imagine an AI that creates an infinite number of problems for it solve so that it can learn how to solve any problem – that’s what Uber’s AI Labs team have created. Now fast forward and imagine throwing problems at that AI, such as finding new ways to hack a heavily defended security system, or how to solve climate change, or make a self-driving car safer, and a million other problems, and hopefully you can see just how interesting this type of exponential AI platform could be when it’s fully developed – especially when connected together with other AI’s and other systems and today and tomorrow's innovative and creative Creative Machines that I discuss regularly.

Last week Uber’s researchers unveiled and released their Paired Open-Ended Trailblazer (POET) algorithm “that pairs the generation of environmental challenges and the optimization of agents to solve those challenges.” In short, what I just said above.

“The POET algorithm explores many different paths through the space of possible problems and solutions and, critically, allows these stepping-stone solutions to transfer between problems,” said David Ha who led the research.

“POET’s aim is to generate new tasks and challenges, find and optimise solutions to them, and then transfer that learning between different tasks in order to create a highly sophisticated AI that can achieve otherwise unobtainable advances,” he added. And not only is creating an AI that generates and solves its own problems a world first, but so too is solving the challenge of how to transfer the skills from one challenge to another in order to improve the models effectiveness.

So far Uber’s team have applied POET to create and solve bipedal walking challenges that were adapted from the BipedalWalker environments in OpenAI Gym that were first popularised in a series of blog posts and papers last year.

In each environment Ei, that’s tasked with solving the problems, is paired with AI agent Ai that’s tasked with generating the problems to solve. Here’s a quick video short of how the two work together – one creating an increasingly challenging environment for the other while the other tries to solve it and walk across it.

The AI creates then tries to solve increasingly complex problems. 

During the experiements the team also put POET through a number of so called transfer experiments in order to explore whether an agent optimised in one environment might serve as a stepping stone to better performance in a different environment, and it’s these types of experiments that researchers hope mean that one day POET could find some radical new solutions to, well, a whole bunch of problems with one of the areas they discussed being POET’s involvement in helping create fascinating new kinds of soft robots for unique challenges it invents that only soft robots can solve - something that a new field called Evolutionary Robotics is also trying to accomplish. It’s also thought that POET could also generate simulated test courses for autonomous cars and vehicles that both expose unique edge cases and demonstrate solutions to them.

In their blog, the researchers said that they will release the source code soon and also that “more exotic applications are conceivable, like inventing new proteins or chemical processes that perform novel functions that solve problems in a variety of application areas. Given any problem space with the potential for diverse variations, POET can blaze a trail through it”.

Read more about Paired Open-Ended Trailblazer (POET) in detail in its research paper.