Detecting salient signal in time-frequency domain with Random Forests

In the face of rapidly increasing volumes of neural data, identifying the key elements within extensive brain recordings presents a growing challenge. We have developed a method based on the premise that a machine learning model, which is highly effective in neural decoding (specifically, in predicting stimuli from neural responses), can also serve as a tool to isolate the most pertinent aspects of neural signals. Let me show you how.

In this example we are working with a dataset of sEEG recordings obtained while participants were looking at images from various categories. While the primary task here was training a reliable decoder to identify the image category from the neural signal, the secondary goal, the one we want to talk in this article, was to detect which portions of the neural signal were actually important for the task at hand.

We will showcase the idea behind the method on a particular example, but as you will see the approach is widely applicable to various experimental paradigms and neuroimaging techniques. Here is a general overview of the method:
(a) A set of stimuli or conditions are presented to participants,
(b) their neural responses are recoded,
(c) and the electrophysiological signal is converted to time ^{(the X axis)} – frequency ^{(the Y axis)} domain.
(d) A Random Forest model is trained to classify the neural response into one of the eight image categories. Features of the model are time-frequency bins as shown on the figure above. So far this is is just a classical decoding pipeline.
(e) The additional step we make is to use feature importance analysis, so understand which components of the TF decomposition are the ones that the model is actually relying on to make the classification.
That’s it. Simple.

Despite the simplicity this small additional step opens up various avenues for subsequent analysis, which in this case was filtering and clustering the data based on the activity that occurs in the regions of the TF domain that are relevant to the task of visual object categorisation:

After this step we are left with only the data that is relevant to the cognitive task. As we can see while there are multiple changes in the signal post-stimulus across the time-frequency domain, only a portion of it carries unique information. We can now take those areas and cluster neural activity and anatomical locations based on these windows of importance:

Here we can see how neural response to images of animals (top row) elicits distinctive clusters of neural activity in time-frequency domain, and propagates into the ventral and prefrontal lobes. While the images of white noise (bottom row) leaves the clusters homogenous and important activity does not appear outside the visual areas.

References

Kuzovkin, I., Vidal, J.R., Perrone-Bertolotti, M. et al. Identifying task-relevant spectral signatures of perceptual categorization in the human cortex. Sci Rep 10, 7870 (2020). https://doi.org/10.1038/s41598-020-64243-6