top of page
Writer's pictureFelipe Martín

Neural Decoding by Machine Learning and Data Visualisation: How the brain interprets visual stimuli during different behavioural states

Updated: Feb 19





In the intersection of neuroscience and artificial intelligence lies a opportunity to unveil the mysteries of the brain, and as technology advances, the application of machine learning has become an invaluable tool in exploring how our minds work. This article delves into how we can utilize machine learning to deepen our understanding of brain activity, particularly in visual processing.





Exploring the Project and Data

For those interested in exploring the code of the project in Python, you can access it through the following link: GitHub Repository. The complete dataset can be found at the following link: Dataset (Source: Schröder et al. (2020))

Additionally, themaster's project related to this research is available at the following link: Master's Project.


The Power of Machine Learning in Neuroscience

Machine learning has solidified its position as an essential tool in classification and prediction tasks across various fields. From classifying everyday objects to detecting diseases, this technology has showcased its effectiveness. However, its application in neuroscience goes beyond conventional uses.

At the heart of this development lies the ability to measure neuronal activity through electrical signals. These signals indicate whether a neuron is active or inactive and, if active, its level of activity. If we look at this neural activity, they are like shooting stars that appear and disappear:




Let us now imagine a scenario where we show visual stimuli or images as parallel lines in different directions to a subject while measuring their brain activity. Could we predict what image the subject is seeing using machine learning based on neural activity?


Machine Learning Models in Neuroscience


To achieve this, a machine learning classification model, e.g. support vector machines (SVMs), can be used. The model uses the activity of each neuron over time as features. In other words, we are working with a multidimensional model, where the number of dimensions depends on the number of neurons we are monitoring. For instance, if we are monitoring a thousand neurons, we are operating in a thousand-dimensional space. The model is then trained with the image that was shown to the subject and we can predict the image from the neuronal activity with an accuracy of over 80% depending on the number of neurons measured and the quality of the data.



But what is the purpose of predicting the image a subject is seeing from this brain activity? This is where the research becomes even more intriguing. In addition to showing visual stimuli, we also make the subject run or do certain activities at certain times and stop at others. This places the individual in two distinct states: "stress" and "no stress."


Decoding the Impact of Stress on Visual Perception


Now, we apply our machine learning model again, but this time we create two separate models, one for the stress state and one for the no-stress state. We then compare the accuracy of these two models. What we observe from the results is that the model designed for stress situations is more accurate in predicting images than the model for no-stress situations.

This phenomenon has a logical explanation: our senses sharpen in stressful or dangerous situations. Several studies have demonstrated that neuronal activity in the visual cortex is strongly influenced by an individual's behavior. In particular, it has been shown that movement can affect neuron activity in the early stages of visual processing.

It is possible to go further and identify which neurons have the most impact on accuracy and which do not, or to make correlations between the behaviour of neural activity and the accuracy obtained, in order to understand much better how the visual perception system works.


Conclusions


In summary, machine learning's integration into neuroscience enables a deeper exploration of brain complexities. By using it to predict visual stimuli under varying emotional states, we gain insights into the brain's responses to stress and relaxation. This approach essentially applies reverse engineering to the machine learning model, trained on neural activity, thereby unraveling how neural behavior influences certain variables like accuracy. 

This approach not only improves our understanding of how the environment and actions impact brain functionality, but also holds great potential for advancing treatments for neurological disorders or the evolution of brain-computer interfaces.

27 views0 comments

Comments


bottom of page