Let's say you read a story or play a chess game. You may not have noticed, but each stage of the path, your mind has kept the way in which the situation (or “state of the world”) changed. You can imagine it as a kind of event list, which we use to update our prediction of what will happen next.
Language models like Chatgpt also follow the changes in their own “spirit” when the code block or anticipation of what you are writing. They generally make educated assumptions using transformers – internal architectures that help models to understand sequential data – but systems are sometimes incorrect due to erroneous thinking models. Identify and refine these underlying mechanisms helps linguistic models to become more reliable prognosticators, in particular with more dynamic tasks such as forecasting of meteorological and financial markets.
But do these AI systems deal with development situations like us? A new paper Researchers from the IT intelligence and artificial intelligence laboratory (CSAIL) and the Department of Electrical Engineering and IT show that models rather use intelligent mathematical shortcuts between each progressive step in a sequence, which ends up making reasonable predictions. The team made this observation by passing under the hood of language models, evaluating how much they could keep track of the objects that change quickly. Their results show that engineers can control when language models use specific circumvention solutions to improve the predictive capacities of systems.
Shell games
The researchers analyzed the internal functioning of these models using an intelligent experience that recalls a classic concentration game. Have you ever had to guess the final location of an object after being placed under a cup and mixed with identical containers? The team used a similar test, where the model guessed the final provision of special figures (also called permutation). The models received a starting sequence, such as “42135” and instructions at the moment and where to move each figure, such as the movement of “4” towards the third position and forward, without knowing the final result.
In these experiences, the models based on transformers have gradually learned to predict the correct final arrangements. Instead of mixing the figures according to the instructions given to them, the systems have aggregated information between successive states (or individual stages in the sequence) and calculated the final permutation.
A reference model that the team observed, called “associative algorithm”, essentially organizes the steps nearby in groups, then calculates a final assumption. You can think of this process as being structured as a tree, where the initial digital arrangement is “root”. When you go up the tree, the adjacent steps are grouped in different branches and multiplied together. At the top of the tree is the final combination of numbers, calculated by multiplying each resulting sequence on the branches together.
The other way that language models have guessed that the final permutation was by a clever mechanism called “associative parity algorithm”, which essentially regains options before grouping them. It determines whether the final provision is the result of a uniform or odd number of rearrangements of individual figures. Then, the mechanism is full of adjacent sequences from different stages before multiplying them, just like the associative algorithm.
“These behaviors tell us that the transformers perform a simulation by an associative scan. Instead of following the status of state -by -step state changes, the models organize them in hierarchies, “explains Belinda Li SM '23, affiliated with MIT PHD. “How to encourage transformers to learn better state monitoring? Instead of imposing that these systems form data on data in a sequential manner of the human type, we should perhaps respond to the approaches they naturally use when monitoring state changes.”
“An avenue of research was to extend the calculation of the test time along the depth dimension, rather than the dimension of token – by increasing the number of transformer layers rather than the number of tokens of thought chain during the reasoning of the test time”, adds Li. “Our work suggests that this approach would allow transformers to build deeper trees of reasoning.”
Through the glass in search
Li and its co-authors observed how associative and associative parity algorithms worked using tools that allowed them to look inside the “mind” of language models.
They first used a method called “survey”, which shows what information crosses an AI system. Imagine that you can examine the brain of a model to see its thoughts at a specific moment – in the same way, the technique extinguishes the predictions at mid -experience of the system on the final arrangement of the figures.
A tool called “activation correction” was then used to show where the language model deals with a situation. This implies interference with some of the system “ideas”, injecting incorrect information into certain parts of the network while keeping other constant parts and seeing how the system will adjust its predictions.
These tools revealed when the algorithms made mistakes and when the systems “understood” how to properly guess the final permutations. They observed that the associative algorithm had learned more quickly than the associative parity algorithm, while operating better on longer sequences. Li attributes the difficulties of the latter with more elaborate instructions for excessive dependence on heuristics (or rules that allow us to quickly calculate a reasonable solution) to predict permutations.
“We found that when language models use a heuristic at the start of training, they will start to integrate these tips into their mechanisms,” explains Li. “However, these models tend to generalize less than those who are not based on heuristics. We have found that certain pre-training objectives can dissuade or encourage these models, so in the future, we can seek to design techniques that discourage models from picking up bad habits. ”
The researchers note that their experiences were carried out on small -scale language models refined on synthetic data, but found that the size of the model had little effect on the results. This suggests that larger refined language models, such as GPT 4.1, would probably give similar results. The team plans to examine their hypotheses more closely by testing language models of different sizes that have not been refined, evaluating their performance on dynamic tasks of the real world such as monitoring of the code and monitoring the evolution of stories.
The Harvard Postdoc Keyon Vafa University, which was not involved in the document, says that the researchers' results could create opportunities to advance linguistic models. “Many uses of large models of language are based on monitoring state: everything, from the supply of recipes to code writing following details in a conversation,” he said. “This article makes significant progress in understanding how linguistic models perform these tasks. This progress provides us with interesting information on what language models do and offers new promising strategies to improve them. ”
Li wrote the newspaper with the undergraduate student of MIT Zifan “Carl” Guo and the main author Jacob Andreas, who is an associate professor of MIT in electrical and computer engineering and principal researcher at CSAIL. Their research was supported, in part, by open philanthropy, MIT Quest for Intelligence, National Science Foundation, Clare Boothe Luce Program for Women in Stem and a Sloan research scholarship.
The researchers presented their research at the International Conference on Automatic Learning (ICML) this week.
