A scientific approach to eavesdropping via HDMI

Thanks to scientists at the University of the Republic (Uruguay), we now have a much better understanding of how to reconstruct an image from spurious radio emissions from monitors; more specifically — from signals leaked during data transmission via HDMI connectors and cables. Using state-of-the-art machine-learning algorithms, the Uruguayan researchers demonstrated how to use such radio noise to reconstruct text displayed on an external monitor.

What, no one’s done it before?

Sure, it’s not the first attempt at a side-channel attack aimed at reconstructing an image from radio signal emissions. A method of intercepting radio noise from a display in a neighboring room — known as a certain TEMPEST attack — was described in a study published in… 1985! Back then, Dutch researcher Wim van Eck demonstrated that it’s possible to intercept a signal from a nearby monitor. In our post about the related EM Eye attack, we talked extensively about these historical studies, so we won’t repeat ourselves here.

However, van Eck’s experiment has lost much of its usefulness today. It used a monitor from 40 years ago with a cathode-ray tube and analog data transmission. Also, the captured image back then was easy to analyze, with white letters on a black background and no graphics. Today, with a digital HDMI interface, it’s much more difficult to intercept the image, and, more importantly, to restore data. But that’s precisely what the Uruguayan team has managed to do.

How does the modern-day van Eck-like interception work?

Data is transmitted digitally to the monitor via an HDMI cable. The volume of data involved is vast. The computer transmits 60 or more frames to the monitor every second, with each frame containing millions of different-colored dots. Using a software-defined radio (SDR), we can intercept signals generated by this data stream. But can we then extract useful information from this extremely weak noise?

The authors called this attack Deep-TEMPEST — a nod to the use of deep-learning AI. The diagram clearly shows how noisy the intercepted data is before processing: we see a discolored shadow of the original image, in which only the location of the main elements can be guessed (a browser window with an open Wikipedia page was used for the experiment). It’s just about possible to distinguish the navigation menu at the top and the image in the center of the screen, but absolutely impossible to read the text or make out the image.

And here’s the result after processing. The picture quality hasn’t improved, so making out the image is no easier. But the text was recognized in its entirety, and even if the machine-learning algorithm tripped up on a couple of letters, it doesn’t greatly affect the final result. Let’s look at another example:

Above is the captured image. Some letters are distinguishable, but the text is basically unreadable. Below is the original image – a screenshot fragment. In the middle is the image after processing by the machine-learning algorithm. Some adjacent letters are hard to discern, but overall the text is quite easy to read.

How did the researchers get this result?

The Uruguayan team’s main achievement is that they developed their own method of data analysis. This was partly due to enhanced neural network training, which allowed text recognition from a rough image. To do this, the team needed pairs that consisted of an original screenshot and the corresponding SDR-captured image. Building a dataset big enough for training (several thousands of pairs) is a difficult, time-consuming task. So the researchers took a slightly different path: about half of the dataset they obtained by displaying an image on the screen and intercepting the signal; the other half they simply generated using a self-written algorithm that gives a reliable picture of the captured information based on the relevant screenshot. This proved sufficient to train the machine-learning algorithm.

The team’s second stroke of genius was the use of a neural network that delivered high-quality results without much expense. The test bed was created from relatively affordable radio-data interception tools; open-source software was used. As we said, HDMI carries vast amounts of data to the connected monitor. To analyze spurious radio emissions during such transmission, it’s important to intercept a large spectrum of radio frequencies — the bigger the band, the better the result. Ideally, what’s needed is a high-end SDR receiver capable of capturing a frequency band of up to 3200 megahertz — a piece of kit that costs about US$25 000. In this case, however, the researchers got by with a USRP 200-mini receiver (US$1500) — capable of analyzing a much narrower frequency band of up to 56 megahertz. But thanks to the enhanced neural network trained to recognize such partial information, they could compensate for the lack of raw data.

Open-source software and libraries were used to process the data. Code, screenshots and other resources have been made available on GitHub, so anyone who wishes to can reproduce the results.

Limited scope of application

In the 1999 novel Cryptonomicon by Neal Stephenson, one of the characters, upon discovering that he’s being monitored by “van Eck phreaking”, starts making things difficult for those spying in him by changing the color of letters and replacing the monochrome text background with a video clip. Generally speaking, the countermeasures against TEMPEST-type attacks described by Stephenson a quarter century ago are still effective. You can add noise to an image such that the user won’t even notice — and interception is impossible.

Naturally, the question arises: is the juice worth the squeeze? Is it really necessary to defend against such highly specialized attacks? Of course, in the vast majority of practical cases, there’s nothing to fear from this attack – much better to focus on guarding against real threats posed by malware. But if you work with super-valuable data that super-professionals are after, then it might be worth considering such attacks as part of your threat model.

Also, don’t disregard this study out of hand just because it describes interception from an external monitor. Okay, you might use a laptop, but the image is sent to the built-in display using roughly the same principles — only the transmission interface may be slightly different, while the radiation level will be slightly lower. But this can be addressed by refining the algorithms and upgrading the test equipment. So hats off to the Uruguayan researchers — for showing us once again just how complex the real world is beyond “software” and “operating systems”.