Discovering faster algorithms for matrix multiplication remains a key pursuit in computer science and digital linear algebra. Since the pioneering contributions of Rhineston and Winograd in the late 1960s, which has shown that the products of the general matrix could be calculated with fewer multiplications than what was thought before, various strategies have emerged. These include methods based on gradient, heuristic techniques, group theoretical frameworks, random walks based on graphics and a deep strengthening learning. However, a significantly lower concentration has been placed on matrix products with an inherent structure, such as when the second matrix is transposition or identical to the first, or when matrices have rarity or symmetry. This surveillance is notable, given that expressions like AA ^ T appear frequently in fields such as statistics, in -depth learning and communications, representing critical constructions such as Gram and covariance matrices. In addition, xx ^ t is repetitive Great language model Training algorithms such as muon and shampoo.
Previous studies have explored the multiplication of the structured matrix using various theoretical methods and based on automatic learning. The theory of representation and the Cohn – Umans framework were used to design effective multiplication patterns for structured matrices. Reinforcement learning has also proved promising – the models have learned to discover or rediscover known algorithms like that of Stratusen. Recent work has focused on optimizing the calculation of xx ^ t on finished fields and complex domains. Among these, the most effective known method for the XX ^ T with actual value is the rhineurous algorithm, which applies the algorithm of rhinestones recursively on 2 × 2 block matrices, effectively translating the structured problem in the field of the multiplication of the general matrix.
Researchers from the Chinese University and the Shenzhen Research Institute of Big Data have developed RXTX, an algorithm to effectively calculate xx ^ t where x belongs to R ^ n * m. RXTX reduces the number of operations required – multiplications and additions – by about 5% compared to current cutting -edge methods. Unlike many algorithms that show only advantages for large matrices, RXTX provides improvements even for small sizes (for example, n = 4). The algorithm was discovered by research based on automatic learning and combinatorial optimization, by taking advantage of the specific structure of XX ^ t for the acceleration of constant factor.
The RXTX algorithm improves the multiplication of the matrix by reducing the number of operations compared to the previous methods such as recursive and rhinestone-winograd rhinestones. It uses 26 multiplications of general matrix and optimized addition patterns, which leads to fewer total operations. Theoretical analysis shows that RXTX performs fewer multiplications and combined operations, in particular for larger matrices. Practical tests on 6144 × 6144 matrices using a single wire processor show that RXTX is approximately 9% faster than standard blas routines, with accelerations observed in 99% of races. These results highlight the efficiency of RXTX for large -scale symmetrical matrix products and its advantage on traditional and advanced recursive algorithms.
The proposed methodology incorporates RL with a mixed linear linear programming pipeline at two levels (MILP) to discover effective matrix multiplication algorithms, in particular for the calculation xx ^ t. The great search for a district guided by RL generates a large set of potential bilinear products of Row-1, which are candidate expressions. MILP-A explores all the linear combinations of these products to express target outputs, while MILP-B identifies the smallest subset that can represent all targets. This configuration reflects the alphatenship approach but simplifies it by considerably reducing the action space, focusing on the products of tensors with a lower dimension and by taking advantage of MILP solvents like Gurobi for a rapid calculation.
For example, to calculate xx ^ t for a 2 × 2 x matrix, the objective is to derive expressions like x_1 ^ 2 + x_2 ^ 2 or x_1x_3 + x_2x_4. RL Rando policyMlThere samples thousands of bilinear products using coefficients of {−1, 0, +1}. MILP-A finds combinations of these products which correspond to the desired expressions, and MILP-B selects the least necessary to cover all the targets. This framework allowed the discovery of RXTX, an algorithm which performs 5% less multiplications and global operations than previous methods. RXTX is effective for large and small matrices and demonstrates a successful fusion of research and combinatorial optimization based on ML.
Discover the Paper. All the merit of this research goes to researchers in this project. Also, don't hesitate to follow us Twitter And don't forget to join our 95K + ML Subdreddit and subscribe to Our newsletter.
Sana Hassan, consulting trainee at Marktechpost and double -degree student at Iit Madras, is passionate about the application of technology and AI to meet the challenges of the real world. With a great interest in solving practical problems, it brings a new perspective to the intersection of AI and real life solutions.
