The LLMs are increasingly considered key to reaching the general artificial intelligence (AG), but they are confronted with major limits in the way they manage memory. Most LLM rely on fixed knowledge stored in their short -term weights and context when using, which makes it difficult to preserve or update information over time. Techniques like CLOTH Try to incorporate external knowledge but lack management of structured memory. This leads to problems such as forgetting past conversations, poor adaptability and isolated memory on all platforms. Basically, today's LLMs do not deal with memory as a manageable, persistent or shareable system, limiting their real utility.
To approach the limits of memory in current LLMs, researchers from Memtensor (Shanghai) Technology Co., Ltd., Shanghai Jiao Tong University, Renmin University of China and The Research Institute of China Telecom have developed a memo. This memory operating system makes memory a first class resource in language models. At his heart is Memcube, an abstraction of unified memory which manages parametric memory, activation and clear text. Service notes allow structured, traceable and transversal memory handling, allowing models to adapt continuously, internalize user preferences and maintain behavioral consistency. This discrepancy transforms the LLM of passive generators into evolution systems capable of long -term learning and multiplatform coordination.
As the AI systems become more complex – by maintaining several tasks, roles and data types – LUEUR models must evolve beyond the understanding of the text to also keep memory and continuous learning. Current LLM lacks structured memory management, which limits their ability to adapt and develop over time. Memos, a new system that treats memory as a basic and planning resource. It allows long -term learning thanks to structured storage, version control and unified access to memory. Unlike traditional training, Memos supports a continuous paradigm of “memory formation” which blurs the border between learning and inference. It also emphasizes governance, ensuring traceability, access control and safe use in the evolution of AI systems.
Memos is a memory -oriented operating system for language models that processes memory not only as stored data, but as an active and evolving component of the model's cognition. It organizes memory in three distinct types: parametric memory (knowledge cooked in the weight of the model via pre-training or fine adjustment), activation memory (temporary internal states, such as KV caches and attention models, used during inference) and clear text memory (modifiable external data, such as documents or invites). These types of memory interact in a unified setting called MemoryCube (Memcube), which summarizes both content and metadata, allowing dynamic planning, versioning, access control and transformation between types. This structured system allows LLM to adapt, recall relevant information and effectively develop their capacities, transform it into more than static generators.
At the heart of the memos is a three -layer architecture: the interface layer manages user inputs and analysis in tasks related to memory; The operation layer manages the planning, organization and evolution of different types of memory; And the infrastructure layer provides safe storage, access governance and collaboration between crossed agents. All interactions within the system are mediated by Memcubes, allowing traceable and politician -based memory operations. Thanks to modules like MemsCHEDULER, MEMLIFECYCLE and MEMGOVERANCE, MEMOS maintains a continuous and adaptive memory loop – from the moment when a user sends a prompt, to the injection of memory during reasoning, to storage of data useful for future use. This design not only improves the reactivity and personalization of the model, but also guarantees that memory remains structured, secure and reusable.
In conclusion, Memos is a memory operating system designed to make memory a central and manageable component in the LLM. Unlike traditional models which depend mainly on the weights of static models and short -term execution states, Memos introduces a unified framework for the management of parametric memory, activation and clear text. At his heart is Memcube, a standardized memory unit which supports structured storage, life cycle management and the increase in memory sensitive to tasks. The system allows more coherent reasoning, adaptability and collaboration between crossed agents. Future objectives include activation of memory sharing between models, self-evolving memory blocks and the construction of a decentralized memory market to support continuous learning and intelligent evolution.
Discover the Paper. All the merit of this research goes to researchers in this project. Also, don't hesitate to follow us Twitter And don't forget to join our Subseubdredit 100k + ml and subscribe to Our newsletter.
Sana Hassan, consulting trainee at Marktechpost and double -degree student at Iit Madras, is passionate about the application of technology and AI to meet the challenges of the real world. With a great interest in solving practical problems, it brings a new perspective to the intersection of AI and real life solutions.
