Introduction

In the wallet.fail presentation at 35C3 we (Thomas Roth, Josh Datko, Dmitry Nedospasov) presented multiple vulnerabilities in modern hardware wallets, one of which was an RF side-channel attack on the Ledger Blue cryptocurrency wallet: It was found that when entering the PIN on the device, each button press creates a significant electro magnetic signal around the 169 MHz spectrum, as can be seen on the video below. The idea was born to use TensorFlow/machine learning to automatically analyze these signals and using it to retrieve the PIN entered into the device - out of thin air!

The setup for finding and recording such a signal can range from very simple up to very complex, for this case everything was done using Software Defined Radios. A cheap RTL-SDR receiver is available for roughly $30, though a more sophisticated device such as a HackRF or a bladeRF offer significantly higher sample rates (and a higher ADC resolution). For example, the setup for recording the video above was a simple HackRF with a cheap telescopic antenna:

Even with this cheap setup, the signal could be picked up from more than 2 meters (6.5 feet) away - using a directional antenna (and maybe using emissions on a different frequency band) this range can be easily increased. It was also found that connecting the USB cable to the device increases the measured strength of the emissions significantly.

The next question was: What bus in the device is causing these emissions? Graphing the recorded emissions, one can see that there are 11 bursts, a short pause, and then more bursts:

Using a logic analyzer, we determined that the signals found are actually the signal to the display - the first 11 bytes contain the adressing information (X & Y coordinates, width and height), followed by the actual image data. If you watch the video above closely, you can see that when a digit on the display is pressed, it gets redrawn with a blue background - and as different buttons have different X & Y coordinates, the first 11 bytes contain enough information to gather the entered PIN!

As it became clear that the emissions contain information required to steal the PIN to the device, the goal was to create a neural network that can simply be fed with the recorded radio signals, and then returns the PIN digit entered.

Gathering training data

To train a neural network, it is necessary to have a lot of training data. In this case, this meant labelled recordings of button presses, for example 100 button presses of digit 0, 100 button presses of digit 1 etc. As this can be a lot of work, we decided to automate it by building a USB-controlled ‘button pusher’ - built from an Arduino, a servo motor, and some random stuff that was laying around the office:

Combining this with a GNU Radio graph and some precise triggering, we had the perfect setup to automatically collect training data for our neural network. The GNU Radio graph was very straightforward, and consisted simply of an osmocom source and a file sink, similar to this graph, but with more precise triggering:

To make it possible to play along at home, we have published the training set (50 samples of the digits 7, 8, 9 & 0), recorded from close distance (to make sure it can not be abused) here. Normally you would use a significantly higher number of training samples, but as this article will show, even with this small data set we can get a nicely performing network!

Preparing the data for TensorFlow

The full Jupyter Notebook including all sources & steps is online here! Feel free to play around with it and use it as a read-along!

After downloading and extracting the data, we can graph the samples to make sure they indeed contain the data we are looking for. Here we are plotting the real component of the I/Q signal, and also the absolute data (yielding a similar graph to what we saw above). We can clearly see the 11 bytes of the display header.

data0 = np.load("try_0_1_short.npy")[:data_length]
plt.figure()
plt.plot(data0.real)
plt.plot(np.abs(data0))
plt.show()

If we graph two different digits (in this case 0 & 7) over each other, we can see that there are no obvious differences in the signal itself:

But, if we apply a simple bandpass filter, the differences become much clearer:

# Configure the bandpass for the signal
bandpass_start = 0.55
bandpass_end = 0.60
bandpass_order = 4

def apply_bandpass(s, order=bandpass_order, low=bandpass_start, high=bandpass_end):
#     butter(order, [low, high], btype='band')
    b, a = signal.butter(order, [low, high], btype='band')
    zi = signal.lfilter_zi(b, a)
    z, _ = signal.lfilter(b, a, s, zi=zi*s[0])
    return z

If we plot the data again, we can see that there are clear differences between the two digits:

The bandpass filter both provides noise filtering and also attenuates the differences between the different digits. By pre-filtering the signal we make training our neural network much easier: As the signal of the different digits is much easier to distinguish, we need less training and are also getting better results!

Loading, labeling and splitting

Now that we know how we want to pre-process our data, it’s time to actually load it. When training a neural network, it’s important to split the available training data into two sets initially:

  • Training data: Used to train & validate the network
  • Test data: After the training is complete and no further changes are made this set is used to validate the neural network

It is important to never use the test data in-between, as this will commonly lead to over-optimizing the network towards the test data, instead of having an unbiased network. In our case, we have 50 recordings per digit, so we use 40 to train the network and 10 as our test data.

Our data loading pipeline looks like this:

  • Load 400 I/Q samples from the respective sample file
  • Apply the bandpass filter
  • Take the absolute value of the bandpass
  • Reshape the data into a 20x20 matrix (to make visualization easier)
  • Use a MinMaxScaler to normalize the samples between 0.0 and 1.0
  • Add the data + the label of the data (in our case the digit name) into two arrays
  • Shuffle both arrays in unison to ensure we are not training the network in a particular order

The code for this can be found in the Jupyter Notebook!

If we plot the data now, we get a nice 2D plot, showing that the data was normalized between 0.0 and 1.0 and that the bandpass was applied correctly:

Building our network

The network we are using is very simliar to the one from the TensorFlow tutorial, so if you are interested in the details around some of the stages feel free to read up there and check out the links in the Jupyter Notebook!

Our network has three layers:

  • Flatten layer: Flattens our 2D array into a single dimension
  • Dense layer: A layer with 128 outputs, using the Rectified Linear Unit (ReLU) activation function
  • Dense layer; A layer with 10 outputs (one per digit, though we are only using 4 in this case), using the SoftMax activation function

The code for our network looks like this:

model = keras.Sequential([
    keras.layers.Flatten(input_shape=(20, 20)),
    keras.layers.Dense(128, activation=tf.nn.relu),
    keras.layers.Dense(10, activation=tf.nn.softmax)
])

And that’s it! Feel free to play around with adding more layers and changing the number of outputs in the intermediate layers!

Next, we compile our network. For that, we need to specify an optimizer. The AdamOptimizer is a great general purpose optimizer, and you generally will not have to touch this. We also need to specify a loss function and a metric on which we want to measure the performance of the network, in our case the primary metric we are interested in is the accuracy (how good our network is at identifying the correct digit).

model.compile(optimizer=tf.train.AdamOptimizer(), 
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

Training the model

Now that we have defined our network and compiled our model, it’s time to train the network! This is as simple as calling the fit function with our training data and the training labels. We also specify how many times the network should be trained on the data (using the epochs parameter), and how the training data should be split between training and validation data:

history = model.fit(train_data, train_labels, epochs=200, verbose=0, validation_split=0.2)

Depending on the amount of training data and the number of epochs, this might take a while! Lets plot the result of this training and look at how the number of epochs impact our loss (the deviation from the correct result) and accuracy:

Something kind of unexpecting happens: The more we train the network, the higher the loss becomes on our validation set! This is something very common in machine learning, and is called overfitting: The network was trained so often on the training data that instead of learning the differences between the signals, it learns to recognize the precise training set! The trick is to find the sweet spot between accuracy & loss. In our case, this is somewhere between 28 and 50.

For example, if we train our network with 30 epochs, we get a lower loss than when training with 200 epochs:

We now have a fully trained network - awesome! Let’s test it!

Testing the network

Now is the moment of truth: How well does the network perform on data it has never seen before? Remember the test data we split of initially? We will run it through our network now to see how precise it is!

test_loss, test_acc = model.evaluate(test_data, test_labels)
print('Accuracy on testdata:', test_acc * 100, "%")
26/26 [==============================] - 0s 58us/sample - loss: 0.1962 - acc: 0.9615
Accuracy on testdata: 96.15384340286255 %

Wow! 96% accuracy! Our network is performing very well. We can also visualize the performance of our network by graphing out the output layer of the model and see how sure the network is on a respective result:

On the left we see the data itself, and the legend that shows that the network is 80% sure that the digit is 8. On the right we can see the weights for the different digits: 0 on the very left, and 7, 8 & 9 on the right. 8 is clearly the winner!

We can plot this for the entire test-set, and can see that we only get a single misclassified digit!

Wrapping up

I hope this short article gave you a good insight in how machine learning can be used to automatically and relatively easily classify side-channel data. This is not just limited to RF side-channels, the same kind of analysis can be performed on for example power SCA traces, time SCA traces etc.

If you are interested in learning more about IoT & hardware security, including side-channel attacks, check out our hands-on IoT & hardware security training. We also provide consulting services all around embedded security, feel free to contact us!