Beware: Keyboards May Spy on Students

Computer scientists at UC Berkeley recently revealed a new security threat that relies on audio recordings of keyboard typing to decipher text, possibly revealing passwords and other sensitive data.

Researchers applied computer science theories to develop a program that can determine the text being typed by any subject. By using a simple recording device strategically placed next to a subject, a series of 10-minute recordings was used to test out the algorithm that recognizes characters with about 96 percent accuracy.

Basing her research on a problem raised by previous researchers-Dmitri Asanov and Rakesh Agrawal from IBM-Li Zhuang, a computer science graduate student, came up with a new approach. While the previous approach involved "training" a keyboard acoustic sniffer with sound recordings, and then running programs to determine text for subsequent keystrokes, the new approach does not rely on past keystrokes. The program only needs to know that the text is in English.

"Our approach is therefore close to what someone who wished to snoop on a system would actually do," UC Berkeley computer science professor Doug Tygar said.

The idea relies on modern techniques in machine learning, but more specifically, statistical learning theory. Statistical learning theory is often used to compute and understand patterns and structures in large data sets that contain complexities beyond the ability of humans to handle. With regards to the research, techniques from the theory are used to classify sounds.

Using cheap $10 microphones, typing sounds were recorded onto a computer and then processed through an algorithm to decode the text. Speech recognition techniques were used to classify and code similar classes of sound so that when a subject is typing, a single key being typed could represent a class of sound at various points in the recording.

"The keys are mounted on a plate which acts a little like a drum-depending on where one strikes it, a different sound is made," Tygar said. "Also, each key develops its own sound as a result of the wear and tear on individual spring mechanisms in the key and idiosyncrasies in the particular construction process for each key."

Even if outside noises are recorded that could potentially interfere with the accuracy of the deciphering, in most cases, the noises can be ignored. Following the recordings, a standard cryptographic technique is used to try and match classes of keystroke sounds to letters. A spelling and grammar check is then run on the initial deciphered text to fix possible errors.

Using the improved text, researchers return to the classification system to re-classify the key sounds.

"When the original typing was made, two keys that are close to each other, such as 'e' and 'r', might have similar sounds, so they could be accidentally classified into the same group the first time around," Tygar said. "Now if we have a word like 'thr' in the text, we may correct that to 'the'. This is fed back to the sound recognizer the next time, so it can pay more attention to subtle differences that distinguish the 'e' and 'r' keys."

Unlike typical spelling and grammar checks, this statistical-based program relies on information on the distribution of letters in a word rather than a dictionary-based supply of words. Comparisons to frequently appearing three-letter patterns in words, such as 'per' or 'hig', are used to fix errors.

The result of running this entire process three times is a highly recognizable text with improved accuracy.

"We only tested English...(but) because other languages with small alphabets, such as French or Russian or Arabic, use similar principles, we believe it will work with them as well," Tygar said. "Similarly, languages with large alphabets, such as Chinese or Japanese, use a phonetic method for entry."

Tygar cites such languages as susceptible to attack. Though there is potential for training the program to apply to various other languages, the research also demonstrates the difficulty in developing future ways to prevent security holes.

"We set out to demonstrate a weakness in the current security design of keyboards, not to build the world's ultimate keyboard spying technique. We limited ourselves to common characters, the 26 lowercase letters, the space bar, the comma, the period, and the enter key," Tygar said. "However, these letters form a major part of material that is interesting to snoop."

Though it seems that there is little a typist can do to prevent private information from being leaked, the research provides ground for future improvements in information transmission.

"We use passwords for everything, from accessing AirBears to accessing our online bank accounts. Other researchers have pointed out that passwords are easy to intercept, and this work shows another way to do it as well," Tygar said. "We need to move beyond passwords to more powerful techniques for protecting data."

Tags: