Consider everything your smartphone has done for you today. Counted your steps? Transcribed notes? Navigated you somewhere new?
Smartphones make for versatile pocket assistants. That’s because they’re equipped with a suite of sensors. And some of those sensors you may never think — or even know — about. They sense light, humidity, pressure, temperature and other factors.
Smartphones have become essential companions. So those sensors probably stayed close by throughout your day. They sat in your backpack or on the dinner table or nightstand. If you’re like most smartphone users, the device was probably on the whole time, even when its screen was blank.
“Sensors are finding their ways into every corner of our lives,” says Maryam Mehrnezhad. She’s a computer scientist at Newcastle University in England. That’s a good thing when phones are using their powers to do our bidding. But the many types of personal information that phones have access to also makes them potentially powerful spies.
Online app store Google Play has already discovered apps that are abusing their access to those sensors. Google recently booted 20 apps from Android phones and its app store. Those apps could record with the microphone, monitor a phone’s location, take photos and then extract the data. And they could do all of this without a user’s knowledge!
Stolen photos and sound bites pose obvious privacy invasions. But even seemingly innocent sensor data might broadcast sensitive information. A smartphone’s motions might reveal what a user is typing. Or it might disclose a someone’s location. Even barometer readings could be misused. These readings subtly shift with increased altitude. That could give away which floor of a building you’re on, suggests Ahmed Al-Haiqi. He’s a security researcher at the National Energy University in Kajang, Malaysia.
Such sneaky intrusions may not be happening in real life — yet. However, concerned researchers are working to head off eventual invasions.
Some scientists have designed invasive apps. Afterward, they tested them on volunteers to highlight what smartphones can reveal about their users. Other researchers are building new phone security systems to help guard users from invasions of their privacy. They could thwart efforts to do everything from stalking a user to stealing the PIN codes needed to access their bank accounts.
Motion detectors are some of the tools within smartphones that are collecting data. These include their accelerometer (Ak-sell-ur-AHM-eh-tur) and the rotation-sensing gyroscope. Such bits of technology could be prime tools for sharing data without your knowing it.
One reason: They’re not permission-protected. That means a phone’s user doesn’t have to give a newly installed app permission to access those sensors. So motion detectors are fair game for any app downloaded onto a device.
In an April 2017 study, Mehrnezhad’s team at Newcastle showed that touching different regions of a screen makes the phone tilt and shift just a tiny bit. You may not notice it. But your phone’s motion sensors will. The data they collect may “look like nonsense” to the human eye, says Al-Haiqi. Yet clever computer programs can tease out patterns in that mess. They can then match segments of motion data to taps on various regions of the screen.
For the most part, these computer programs are algorithms that make up a type of machine learning, Al-Haiqi says. Researchers first train the programs to recognize keystrokes. They do this by feeding the programs lots of motion-sensor data. Those data are then labeled with the key tap that produced a particular movement.
A pair of researchers built TouchLogger. It’s an app that collects sensor data on a phone’s orientation in space. It uses these data to figure out how a user had been tapping on a smartphone’s number keyboard. In a 2011 test on phones made by a company in Taiwan, called HTC, TouchLogger figured out more than 70 percent of key taps correctly.
Since then, more studies have come out showing similar results. Scientists have written code to infer keystrokes on number and letter keyboards for different types of phones. In one 2016 study, Al-Haiqi’s team reviewed how successful these efforts were. And they concluded that only a snoop’s imagination limits the ways motion data could be translated into key taps. Those keystrokes could reveal everything from the password entered on a banking app to the contents of a text message..!!