AI can now listen to individuals in a crowd

Devices like Amazon’s Echo and Google Home can normally deal with requests from a single person, but like us they often still struggle when there are lots of people talking at once, like, say at a party.

Now though that might be less of a problem thanks to a new Artificial Intelligence (AI) agent that can separate the voices of multiple speakers out in real time, and it promises to give automatic speech recognition a big boost, and if it was ever combined with something like Google DeepMind's AI lip reading technology, that was recently proven to be much more accurate than the best human lip readers, then you'll never have to worry about being heard by AI ever again. And I'm almost certain that noone will ever think of using this technology to eavesdrop on you all...

The technology, which was developed by researchers at the Mitsubishi Electric Research Laboratory in Cambridge, Massachusetts, was demonstrated in public for the first time at this month’s Combined Exhibition of Advanced Technologies show in Tokyo.

It uses a machine learning technique the team calls “Deep Clustering” to identify unique features in the voiceprint of multiple speakers, and then it groups the distinct features from each speaker’s voice together, letting it disentangle multiple voices and then letting it reconstruct what each person was saying.

“It was trained using one hundred English speakers, but it can separate voices even if a speaker is Japanese,” says Niels Meinke, a spokesperson for Mitsubishi Electric.

Meinke says the system can separate and reconstruct the speech of five people speaking into a single microphone with up to 90 per cent accuracy, and if there are ten speakers the accuracy dips, but is still up to 80 per cent. In both cases though, this was with speakers the system had never encountered before.

Conventional approaches to this problem, such as using two microphones to replicate the position of a listener’s ears, have only managed 51 per cent accuracy.

In overcoming the “cocktail party effect” that has dogged AI research for decades, the new technology could help smart assistants in homes and cars work better, it could also improve automatic speech transcription, and, naturally, be used to help law enforcement agencies reconstruct recordings of conversations that could otherwise be incomprehensible.

In preliminary tests the system was able to successfully separate the voices of up to five people at once.

“The system could be used to separate speech in a range of products including lifts, air-conditioning units and household products,” says Meinke, and now he and his team are looking to integrate the technology into a number of products that they expect to be released into the market soon.

Their work was published in arxiv.org/abs/1508.04306