New Deep Learning Model Decodes Keyboard Sounds with 95% Accuracy – Source: heimdalsecurity.com

A team of researchers has introduced an innovative approach referred to as a “deep learning-based acoustic side-channel attack,” designed to accurately classify laptop keystrokes recorded using a nearby smartphone, achieving an impressive 95% accuracy rate.

In a recent study published last week, the team comprising Joshua Harrison, Ehsan Toreini, and Maryam Mehrnezhad shared that through training their system on keystrokes obtained from the Zoom video conferencing software, they managed to achieve an exceptional accuracy rate of 93%, thereby setting a new standard in this field of study.

When trained on keystrokes recorded by a nearby phone, the classifier achieved an accuracy of 95%, the highest accuracy seen without the use of a language model.

When trained on keystrokes recorded using the video-conferencing software Zoom, an accuracy of 93% was achieved, a new best for the medium.

Extract from the Researchers’ Findings (Source)

The concept of “side-channel attacks” pertains to a category of security exploits involving the extraction of insights from a system by closely observing and quantifying its physical effects during the processing of sensitive data. Some effects that can be observed include power consumption, runtime behavior, electromagnetic radiation, acoustics, and cache usage.

A New Way to Gain Access to Sensitive Information

These practical attack methods can pose substantial risks to user privacy and security, as they hold the potential to be exploited by malicious actors to gain access to sensitive information, including passwords.

The ubiquity of keyboard acoustic emanations makes them not only a readily available attack vector, but also prompts victims to underestimate (and therefore not try to hide) their output.

For example, when typing a password, people will regularly hide their screen but will do little to obfuscate their keyboard’s sound.

Researchers’ Statement (Source)

To perform this attack, the team initially conducted experiments involving the repeated pressing of 36 keys on an Apple MacBook Pro (ranging from 0 to 9 and a to z), each key pressed 25 times consecutively, with variations in pressure and finger placement.

This data was captured using a smartphone positioned in close proximity to the laptop and also through Zoom.

The subsequent phase involved isolating individual keystrokes and transforming them into mel-spectrograms. These spectrograms were then subjected to a deep learning model known as CoAtNet which effectively classified the keystroke images, explains THN.

In light of these vulnerabilities, the researchers proposed several measures to enhance security, including: altering typing patterns, adopting randomized passwords instead of recognizable words, and introducing randomly generated fake keystrokes to counteract voice call-based attacks.

The full study is available here.

If you liked this article, follow us on LinkedIn, Twitter, Facebook, and Youtube, for more cybersecurity news and topics.