Machine-Learning Algorithm Predicts What Mice See From Brain Data

Summary: EPFL researchers have developed a novel machine-learning algorithm called CEBRA, which can predict what mice see based on decoding their neural activity.

The algorithm maps brain activity to specific frames and can predict unseen movie frames directly from brain signals alone after an initial training period.

CEBRA can also be used to predict movements of the arm in primates and to reconstruct the positions of rats as they move around an arena, suggesting potential clinical applications.

Key Facts:

  1. Researchers from École polytechnique fédérale de Lausanne (EPFL) have developed a machine-learning algorithm called CEBRA that can learn the hidden structure in neural code to reconstruct what a mouse sees when it watches a movie or the movements of the arm in primates.
  2. CEBRA is based on contrastive learning, a technique that enables researchers to consider neural data and behavioral labels like reward, measured movements, or sensory features such as colors or textures of images.
  3. CEBRA’s strengths include its ability to combine data across modalities, limit nuances, and reconstruct synthetic data. The algorithm has exciting potential applications in animal behavior, gene-expression data, and neuroscience research.

Source: EPFL

Is it possible to reconstruct what someone sees based on brain signals alone?  The answer is no, not yet. But EPFL researchers have made a step in that direction by introducing a new algorithm for building artificial neural network models that capture brain dynamics with an impressive degree of accuracy.

Rooted in mathematics, the novel machine learning algorithm is called CEBRA (pronounced zebra), and learns the hidden structure in the neural code.

What information the CEBRA learns from the raw neural data can be tested after training by decoding – a method that is used for brain-machine-interfaces (BMIs) – and they’ve shown they can decode from the model what a mouse sees while it watches a movie.

But CEBRA is not limited to visual cortex neurons, or even brain data. Their study also shows it can be used to predict the movements of the arm in primates, and to reconstruct the positions of rats as they freely run around an arena.

The study is published in Nature.

“This work is just one step towards the theoretically-backed algorithms that are needed in neurotechnology to enable high-performance BMIs,” says Mackenzie Mathis, EPFL’s Bertarelli Chair of Integrative Neuroscience and PI of the study.

For learning the latent (i.e., hidden) structure in the visual system of mice, CEBRA can predict unseen movie frames directly from brain signals alone after an initial training period mapping brain signals and movie features.

The data used for the video decoding was open-access through the Allen Institute in Seattle, WA. The brain signals are obtained either directly by measuring brain activity via electrode probes inserted into the visual cortex area of the mouse’s brain, or using optical probes which consist of using genetically modified mice, engineered so that activated neurons glow green.

During the training period, CEBRA learns to map the brain activity to specific frames. CEBRA performs well with less than 1% of neurons in the visual cortex, considering that, in mice, this brain area consists of roughly 0.5 million neurons.

“Concretely, CEBRA is based on contrastive learning, a technique that learns how high-dimensional data can be arranged, or embedded, in a lower-dimensional space called a latent space, so that similar data points are close together and more-different data points are further apart,” explains Mathis.

“This embedding can be used to infer hidden relationships and structure in the data. It enables researchers to jointly consider neural data and behavioral labels, including measured movements, abstract labels like “reward,” or sensory features such as colors or textures of images.”

“CEBRA excels compared to other algorithms at reconstructing synthetic data, which is critical to compare algorithms,” says Steffen Schneider, the co-first author of the paper. “Its strengths also lie in its ability to combine data across modalities, such as movie features and brain data, and it helps limit nuances, such as changes to the data that depend on how they were collected.”

“The goal of CEBRA is to uncover structure in complex systems. And, given the brain is the most complex structure in our universe, it’s the ultimate test space for CEBRA. It can also give us insight into how the brain processes information and could be a platform for discovering new principles in neuroscience by combining data across animals, and even species.” says Mathis.

“This algorithm is not limited to neuroscience research, as it can be applied to many datasets involving time or joint information, including animal behavior and gene-expression data. Thus, the potential clinical applications are exciting.”

About this machine learning research news

Author: Press Office
Source: EPFL
Contact: Press Office – EPFL
Image: The image is credited to Neuroscience News

Original Research: Open access.
“Learnable latent embeddings for joint behavioural and neural analysis” by Mackenzie Mathis et al. Nature


Learnable latent embeddings for joint behavioural and neural analysis

Mapping behavioural actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioural data increases, there is growing interest in modelling neural dynamics during adaptive behaviours to probe neural representations.

In particular, although neural latent embeddings can reveal underlying correlates of behaviour, we lack nonlinear techniques that can explicitly and flexibly leverage joint behaviour and neural data to uncover neural dynamics.

Here, we fill this gap with a new encoding method, CEBRA, that jointly uses behavioural and neural data in a (supervised) hypothesis- or (self-supervised) discovery-driven manner to produce both consistent and high-performance latent spaces. We show that consistency can be used as a metric for uncovering meaningful differences, and the inferred latents can be used for decoding.

We validate its accuracy and demonstrate our tool’s utility for both calcium and electrophysiology datasets, across sensory and motor tasks and in simple or complex behaviours across species. It allows leverage of single- and multi-session datasets for hypothesis testing or can be used label free.

Lastly, we show that CEBRA can be used for the mapping of space, uncovering complex kinematic features, for the production of consistent latent spaces across two-photon and Neuropixels data, and can provide rapid, high-accuracy decoding of natural videos from visual cortex.

The Rise of the Machines: Assessing The Ethical Risks of Machine Learning And AI

As the world progresses towards digitalization, more people are adopting Artificial Intelligence (AI). The pandemic has accelerated this adoption. There are predictions that computers and robots will become more capable of comprehending multiple languages and knowledge.

Machine learning is an association of artificial intelligence (AI) and computer science that operates data and algorithms to emulate how humans learn, gradually enhancing its accuracy, defined mainly as a machine’s capability to mimic intelligent human demeanour.

Machine Learning involves machines learning on their own without explicit programming. These systems use quality data to build various machine-learning models with the help of algorithms. The selection of algorithms is determined by the nature of the data and the specific task that needs to be accomplished.

Dark AI Scenarios and Malevolent AI:
It’s important to remember that everything has two sides much like a coin for everything good , will have something bad associated with it!, including machine learning.

While it has become a popular solution for many applications, hackers and crackers are finding ways to exploit these approaches. Although machine learning can bring innovation and adaptation to various sectors, it raises concerns and potential issues.

With powerful AI applications, personal secrets have the potential to be unravelled at the behest of Artificial intelligence much against our consent. Protecting personal information is crucial, and it’s important to remain vigilant.

While certain technologies were designed with good intentions, they can be misused if they end up in the wrong hands. As we explore the neverending possibilities of this innovative technology,

it’s important to remain mindful of its ramifications and negative impacts. While using Artificial Intelligence can be highly beneficial, it can pose a significant security and privacy risk.

Label Flipping:

Label flipping involves swapping the expected outcomes. A poisoning attack occurs when the attacker adds inadequate data to your model’s training dataset, leading it to learn inappropriate information. The most anticipated result of a poisoning attack is that the model’s boundary limits shift somehow.

threats with the Machine learning model:
Adversarial Examples/Evasion Attack:
One crucial security threat to machine learning systems is Adversarial Examples or Evasion Attacks, which are extensively studied. This attack involves manipulating the input or testing data to make the machine learning system predict incorrect information.

This compromises the system’s integrity, and the confidence in the system is affected. It has been noted that a system that overfits data is vulnerable to evasion attacks.

If a hacker intercepts the interaction between the model and the interface responsible for showing results, they can display manipulated information. This type of attack is named the output integrity attack. Due to our absence of understanding of the actual inner working of a machine learning system theoretically, it becomes difficult to predict the natural result. Hence, when the system has shown the output, it is taken at face value. The attacker can control this naivety by compromising the integrity of the production.

Although machine learning algorithms have existed for decades, their popularity has increased with the growth of artificial intelligence, particularly in deep learning models that power today’s most advanced AI applications. Many major vendors, including Amazon, Google, Microsoft, IBM, and others, compete to sign up customers for their machine learning platforms.

These platforms cover machine-learning activities, including data collection, preparation, classification, model building, training, and application development. There is a growing trend towards utilizing a critical technology that many businesses across various industries are steadily adopting at a rapid pace.