Researchers from the IT intelligence laboratory and artificial MIT (CSAIL) have developed a new model of artificial intelligence inspired by neuronal oscillations in the brain, in order to considerably advance the way in which automatic learning algorithms manage long data sequences.
AI often has trouble analyzing complex information that takes place over long periods, such as climatic trends, biological signals or financial data. A new type of AI model, called “state space models”, has been specifically designed to understand these sequential models more effectively. However, existing models in the state space are often faced with challenges – they can become unstable or require a significant amount of calculation resources when processing long data sequences.
To solve these problems, CSAIL T. Konstantin Rusch and Daniela Rus researchers developed what they call “linear oscillatory state space models” (linen), which exploit the principles of forced harmonic oscillators – a concept deeply rooted in physics and observed in organic neural networks. This approach provides stable, expressive and effective predictions on the computer without too restrictive conditions on model parameters.
“Our objective was to grasp the stability and efficiency observed in organic neural systems and to translate these principles into an automatic learning framework,” explains Rusch. “With Linoss, we can now reliably learn long -range interactions, even in sequences covering hundreds of thousands of or more data points.”
The linen model is unique to ensure a stable prediction by requiring much less restrictive design choices than previous methods. In addition, researchers have rigorously proven the universal approximation capacity of the model, which means that it can approximate any continuous and causal function connecting the input and output sequences.
Empirical tests have shown that Linoss has systematically surpassed existing cutting -edge models through various classification and prediction of demanding sequence. In particular, Linoss has surpassed the widely used Mamba model of almost twice in tasks involving extreme length sequences.
Recognized for its meaning, research has been selected for an oral presentation at ICLR 2025 – an honor granted to the higher 1% of submission. MIT researchers provide that the linen model could have a significant impact on all areas that would benefit from an exciting and effective forecasting and classification, in particular the analysis of health care, climate science, autonomous conduct and financial forecasts.
“This work illustrates how mathematical rigor can lead to breakthroughs of performance and wide applications,” explains Rus. “With Linoss, we provide the scientific community with a powerful tool to understand and predict complex systems, fill the gap between biological inspiration and computer innovation.”
The team imagines that the emergence of a new paradigm as linen will interest practitioners of the automatic learning on which to rely. For the future, researchers plan to apply their model to an even wider range of different data methods. In addition, they suggest that Linoss could provide precious information on neuroscience, potentially improving our understanding of the brain itself.
Their work was supported by the Swiss National Science Foundation, the Schmidt AI2050 program and the artificial intelligence accelerator of the American Air Force department.
