Microsoft AI presents the code researcher: an in -depth research agent for the code of major systems and commits history

Rise of autonomous coding agents in the debugging system software

The use of AI in the development of software has gained ground with the emergence of large language models (LLM). These models are able to perform coding tasks. This change has led to the design of autonomous coding agents that help or even automate the tasks traditionally carried out by human developers. These agents range from simple script writers to complex systems capable of navigating code bases and diagnosing errors. Recently, the objective has moved to the license to hand over these agents to meet more sophisticated challenges. In particular those associated with extended and complex software environments. This includes fundamental systems software, where precise changes require an understanding not only of the immediate code, but also its architectural context, interdependencies and historical evolution. Thus, there is an increasing interest in the construction of agents which can carry out in -depth reasoning and synthesize the fixes or the changes with a minimum human intervention.

Challenges in the debugging of the system of large -scale systems

The update of the code of large -scale systems has a multiple challenge due to its size, complexity and inherent historical depth. These systems, such as operating systems and networking batteries, consist of thousands of interdependent files. They have been refined for decades by many contributors. This leads to highly optimized and low level implementations where even minor alterations can trigger cascade effects. In addition, the traditional descriptions of bugs in these environments often take the form of raw crash reports and traces of battery, which are generally devoid of guiding advice in natural language. Consequently, the diagnosis and repair of problems in such a code require a deep and contextual understanding. This requires not only an understanding of the current logic of the code, but also an awareness of its past modifications and its global design constraints. The automation of these diagnoses and repairs has remained elusive, as it requires in -depth reasoning that most coding agents are not equipped to perform.

Limits of existing coding agents for planting at the system level

Popular coding agents, such as Swe-Agent and Openhands, take advantage of large-language models (LLM) for the automated fixation of bugs. However, they mainly focus on smaller code bases in the application. These agents are generally based on descriptions of structured problems provided by humans to reduce their research and propose solutions. Tools such as Autocodeover explore the code base using syntax -based techniques. They are often limited to specific languages ​​like Python and avoid subtleties in the system. In addition, none of these methods incorporates information on the evolution of the code from validation stories, a vital component when managing the bugs inherited in large -scale code bases. While some use a heuristic for code navigation or the edition generation, their inability to reason deeply through the code base and to consider the historical context limits their effectiveness in the resolution of complex accidents at the level of the system.

Code of Code: a deep search agent from Microsoft

Researchers from Microsoft Research have introduced Code researcherAn in -depth research agent designed specifically for the debugging of the code at the system level. Unlike previous tools, this agent is not based on predefined knowledge of Buggy files and operates in a fully assisted mode. It has been tested on a reference of the Linux nucleus crash and a multimedia software project to assess its generalization. The code researcher was designed to execute a multi-shade strategy. First of all, he analyzes the context of the crash using various exploratory actions, such as searches for defining symbols and research of reasons. Second, it synthesizes patch solutions based on accumulated evidence. Finally, it validates these fixes using automated test mechanisms. The agent uses tools to explore the semantics of the code, identify function flows and analyze validation stories. It is a critical innovation previously absent in other systems. Thanks to this structured process, the agent operates not only as a bucket fixer but also as an autonomous researcher. He collects data and hypotheses before intervening in the code base.

Three -phase architecture: analysis, synthesis and validation

The functioning of the code researcher is broken down into three defined phases: analysis, synthesis and validation. In the analysis phase, the agent begins by treating the crash report and initiated the stages of iterative reasoning. Each step includes tool invocations to search for symbols, search for code models using regular expressions and explore messages and diffs. For example, the agent can seek a term such as “memory flight” in the commitments made to understand the modifications of code which could have introduced instability. The memory it builds is structured, recording all requests and their results. When it determines that a sufficient context has been collected, it goes to the summary phase. Here, it filters unrelated data and generates fixes by identifying one or more potentially defective extracts, even if they have spread to several files. In the final validation phase, these fixes are tested in relation to the original crash scenarios to verify their efficiency. Only validated solutions are presented for use.

Reference performance on the Linux and FFMPEG kernel

In terms of performance, the code researcher has made substantial improvements compared to his predecessors. When he compared Kbenchsyz, a set of 279 Linux kernel accidents generated by the Syzkaller Fuzzer, he resolved 58% of accidents using GPT-4O with an execution budget at 5 trajectory. On the other hand, SWE-AGENT only managed a resolution rate of 37.5%. On average, the code researcher explored 10 files per trajectory, much more than 1,33 files navigated by agent SWE. In a subset of 90 cases where the two agents modified all known Buggy files, the code researcher resolved 61.1% of accidents against 37.8% per Swe-agent. In addition, when O1, a model focused on reasoning, was only used in the patching generation stage, the resolution rate remained at 58%. This reinforces the conclusion that the strong contextual reasoning considerably stimulates the debugging results. The approach was also tested on FFMPEG, an open source multimedia project. He managed to generate accident prevention corrections in 7 accidents reported out of 10, illustrating his applicability beyond the nucleus code.

The main technical dishes of the study of the code researchers

• Achieved 58% crash resolution on the benchmark of the Linux nucleus against 37.5% by Swe-Agent.
• Explored an average of 10 files per bug, against 1.33 files by basic methods.
• Efficiency demonstrated even when the agent had to discover the Buggy files without prior guidance.
• Incorporated a new use of the analysis of the history of commitments, stimulating contextual reasoning.
• Generalized to new areas like FFMPEG, resolving 7 accidents reported out of 10.
• Structured memory used to preserve and filter the context for the generation of patches.
• Has shown that deep reasoning agents surpass traditional agents even when they have given more calculation.
• Corrects validated with real crash reproduction scripts, guaranteeing practical efficiency.

Conclusion: a step towards debugging of the autonomous system

In conclusion, this research presents a convincing progression in automated debugging for large -scale system software. By dealing with the resolution of bugs as a research problem, requiring exploration, analysis and hypothesis tests, the code researcher illustrates the future of autonomous agents in the maintenance of complex software. It avoids the traps of previous tools by operating independently, by examining in depth the current code and its historical evolution, and the synthesis of validated solutions. Significant improvements in resolution rates, especially in unknown projects such as FFMPEG, demonstrate the robustness and scalability of the proposed method. This indicates that software agents can be more than reactive responders; They can operate as investigation assistants capable of making intelligent decisions in environments previously deemed too complex for automation.

---

Discover the Paper. All the merit of this research goes to researchers in this project. Also, don't hesitate to follow us Twitter And don't forget to join our Subseubdredit 100k + ml and subscribe to Our newsletter.