Cornell’s AI uses vibrations to instantly turn your dumb home into a smart home

When it comes to having privacy in your own home we know that in the future there’ll be no such thing as companies use your WiFi to track your emotions, health, and movements, and as everything in your house, from the camera in your Smart TV to the Alexa smart speaker on your kitchen work surface, and even your connected pacemaker, becomes a spying device.

Advances in technology have made many household appliances more energy efficient, but figuring out the power usage of each individual device across the home is still a tall order, so researchers at Cornell University have been working on more of a one size fits all solution, developing a vibration-sensing device that can keep tabs on your appliance usage by using Artificial Intelligence (AI) and lasers.

The team points to smart homes of the future as its inspiration for developing the VibroSense device, imagining scenarios where the house itself knows when a washing machine has completed its cycle, when a microwave has finished heating food or a faucet is dripping. While replacing each appliance with smart versions or attaching specific sensors to them could be one way to tackle this, the Cornell team sees a more efficient way forward.

“In order to have a smart home at this point, you’d need each device to be smart, which is not realistic; or you’d need to install separate sensors on each device or in each area,” says Cheng Zhang, assistant professor of information science and senior author of the study. “Our system is the first that can monitor devices across different floors, in different rooms, using one single device.” And if it can do all that then it’s not too far away from being able to track anyone and everything else in the house, including you.

The VibroSense device uses a laser Doppler vibrometer to detect tiny vibrations in surfaces. But at the heart of it is a deep learning network, which was trained to recognize the paths different types of vibrations take as they move through a house, along with the distinct noises of different appliances and objects.

In time, this deep learning network was able to distinguish between the similar but different vibrations created by individual household appliances, by recognizing the paths these vibrations took as they travelled through the house. In testing, it demonstrated an ability to identify 17 different activities across five different houses, such as dripping faucets, exhaust fans, kettles, and fridges, with almost 96 percent accuracy.

According to the team, it could also differentiate between specific stages of a device’s activity with an average accuracy of more than 97 percent. These tests were carried out with the laser pointed at an interior wall in the center of single-story homes, and at the ceiling in two-story homes, with the team noting it wouldn’t be too well suited to apartment blocks because it could detect vibrations coming from the neighbours and pose a privacy risk.

“Since our system can detect both the occurrence of an indoor event, as well as the time of an event, it could be used to estimate electricity and water-usage rates, and provide energy-saving advice for homeowners,” Zhang says. “It could also prevent water and electrical waste, as well as electrical failures such as short circuits in home appliances which means it could also be used to one day help save lives.”

Source: Cornell University