Nvidia’s latest autonomous drone flies off grid

GPS can be a useful tool for a drone that needs to find its way around but despite almost blanket GPS coverage there are still areas of the planet where the signal is either non existent or at best patchy, so now, as a result a team of researchers from Nvidia, those same crazy cats that have created the world’s most energy efficient supercomputer and new ways to peer into an AI’s brain, are now working on a drone navigation system that relies solely on vision.

Rather than build a flying drone from scratch the team, who are based in the US, opted to use an off the shelf drone and then they swapped out its firmware and replaced it with their new Jetson TX1 machine learning module, which relies on visual data from two cameras.

 

While the “new” drone is still at the experimental stage the team have designed it to navigate its way through forest trails on rescue missions – looking out for fallen hikers, storm damage and injured bears, that sort of stuff. That said though the team also think that the low flying drone could one day broaden its scope to be used anywhere where GPS isn’t reliable such as canyons, crowded city landscapes, and even checking stock in a warehouse - the system could even be adapted to search for damaged cables underwater.

"This works when GPS doesn't," said the team's technical lead Nikolai Smolyanskiy, "all you need is a path the drone can recognise visually."

The drone's navigation system also boasts obstacle avoidance skills - something that Skydio’s drone was also showing off last year - and it’s been taught to follow train tracks too and the team has even created a wheeled robot that zips around the halls of buildings.

The main project proving ground though has been forest paths that can be more of a challenge than the relatively predictable urban setting where points of reference such as mailboxes and curb heights often remain a constant. Wooded areas, by comparison, offer little in the way of uniformity and varying light, lack of markings and trees of different shapes and sizes all help to push a camera based navigation system to its limits.

"Our whole idea is to use cameras to understand and navigate the environment," said Smolyanskiy, "Jetson gives us the computing power to do advanced AI on board the drone, which is a requirement for operating in remote environments."

The researchers believe that the system has already managed the longest and most stable GPS-free flight to date, flying itself along the center of a forest trail for a kilometer (0.6 mi) while also dodging obstacles.

The flying drone was taught the ins and outs of forest flying by watching video captured by three GoPro cameras attached to a rig as Smolyanskiy walked through eight miles of trails in the Pacific Northwest. The team's TrailNet neural network was also fed video of trails in the Swiss Alps recorded by the Istituto Dalle Molle di Studi sull'Intelligenza Artificiale in Zurich.

In the long term the goal of the project is to be able to program coordinates into a drone's camera based navigation system and then have it make the trip on its own, and in the shorter term, Nvidia are developing software that developers can download and use to create their own versions and creations.