Introduction to generalization in mathematical reasoning
Large-scale language models with a long reasoning on COT, such as Deepseek-R1, have shown good results on mathematics at the Olympiad level. However, the models formed by the supervised fine adjustment or the reinforcement depend on limited techniques, such as the repetition of the known rules of algebra or the defect to coordinate the geometry of diagram problems. Since these models follow models of learned reasoning rather than showing real mathematical creativity, they are confronted with challenges with complex tasks that require original ideas. Current mathematical data sets are poorly suited to the analysis of mathematical skills that RL models can learn. Large -scale corpora integrate a range of mathematical questions varying in terms of and in difficulty, which makes it difficult to isolate specific reasoning skills.
Limits of current mathematical references
Current methods, such as outside distribution generalization, focus on managing test distributions that differ from training data, which is crucial for mathematical reasoning, physical modeling and financial forecasts. Composition generalization techniques aim to help models systematically combine the skills learned. Researchers have created data sets through various methods to compare mathematical capacities, which include hiring humans to write problems such as GSM8K and Minervamath, collect exam questions such as loves and Olympiadbench, and scratching and filtering examination corpus like Numanamath and Bigmath. However, these approaches lack sufficient challenge for modern LLMs or fail to provide an analysis granularity.
Omega presentation: a controlled reference for reasoning skills
Researchers from the University of California, AI2, the University of Washington and Dmodel.ai proposed Omega, a reference designed to assess three dimensions of generalization outside distribution, inspired by the typology of Boden's creativity. He creates pairs of training and paired test designed to isolate the specific reasoning skills with three dimensions: exploratory, compositional and transformative. Omega test and train problems are built using carefully modified models, allowing precise control over the specific diversity, complexity and reasoning strategies required for solutions. In addition, he uses 40 generators of model problems in six mathematical areas: arithmetic, algebra, combinatorial, numbers of numbers, geometry and logic and puzzles.
Evaluation on the LLMS front configuration and strengthening
Researchers assess four border models, including Deepseek-R1, Claude-3.7-Sonnet, Openai-O3-Mini and Openai-O4-minini, at different levels of complexity. For RL generalization experiences, the frame applies GRPO algorithm on 1,000 training problems using QWEN2.5-7B-ISTRUCT and QWEN2.5-MATH-7B models. Exploratory generalization trains at limited levels of complexity and assesses higher complexity problems. The generalization of the composition implies the formation of models on individual skills in isolation and the test of their ability to combine and apply these skills effectively. Transformational generalization trains on conventional solution approaches and assesses performance on problems that require unconventional strategies.
Performance observations and model behavior models
LLMS reasoning tends to perform less as the complexity of the problem increases, often finding correct solutions early, but spending too many tokens in unnecessary verification. The RL applied only to low -complex problems improves generalization to medium complexity problems, with greater gains on examples in the field than examples outside distribution, indicating the efficiency of RL to strengthen familiar models. For example, in the Zebra logical field, the basic model reaches only 30% precision. However, the RL training increased the performance of 61 points on examples in the field and 53 points on examples excluding distribution without SFT.
Conclusion: towards the advancement of transformational reasoning
In conclusion, the researchers introduced Omega, a reference which isolates and assesses three areas of generalization outside distribution in mathematical reasoning: exploratory, compositional and transformers. The empirical study reveals three ideas: (a) The fine adjustment considerably improves the performance of generalization and exploratory generalization tasks, (b) the advantages of RL for composition tasks are limited and (c) rl fails to induce models of truly new reasoning. These results highlight a fundamental limitation: RL can amplify the extent and depth of problem solving, but it does not lack essential creative jumps for transformational reasoning. Future work should explore the curriculum scaffolding and metaisonning controllers.
Discover the Paper,, Project page And GitHub page. All the merit of this research goes to researchers in this project. Also, don't hesitate to follow us Twitter And don't forget to join our Subseubdredit 100k + ml and subscribe to Our newsletter.
Sajjad Ansari is a last year's first year of the Kharagpur Iit. As a technology enthusiast, he plunges into AI's practical applications by emphasizing the understanding of the impact of AI technologies and their real implications. It aims to articulate complex AI concepts in a clear and accessible way.
