The artificial intelligence models of the neural network used in applications such as processing of medical image and speech recognition carry out operations on extremely complex data structures which require a huge amount of calculation to be treated. This is one of the reasons why deep learning models consume so much energy.
To improve the efficiency of AI models, MIT researchers have created an automated system that allows developers of deep learning algorithms to simultaneously benefit from two types of data redundancy. This reduces the quantity of calculation, bandwidth and memory storage necessary for automatic learning operations.
Existing techniques to optimize algorithms can be heavy and generally allow developers to capitalize on rarity or symmetry – two different types of redundancy that exist in deep learning data structures.
By allowing a developer to build an algorithm from zero which takes advantage of the two redundancies at the same time, the approach of MIT researchers increased the speed of calculations by almost 30 times in certain experiments.
Since the system uses a user -friendly programming language, it could optimize automatic learning algorithms for a wide range of applications. The system could also help scientists who are not in -depth learning experts but who want to improve the efficiency of AI algorithms they use to process data. In addition, the system could have applications in scientific IT.
“For a long time, the capture of these data redundancies required a lot of implementation efforts. Instead, a scientist can tell our system what he would like to calculate in a more abstract way, without saying exactly to the system how to calculate it ” paper on the systemwhich will be presented to the international symposium on the generation and optimization of the code.
She is joined by the newspaper by the main author Radha Patel '23, SM '24 and the main author Saman Amarasinghe, professor in the Department of Electric and IT engineering (CEE) and principal researcher at the IT intelligence and artificial intelligence laboratory (CSAIL).
Cut the calculation
In automatic learning, data is often represented and handled as multidimensional tables called tensors. A tensor is like a matrix, which is a rectangular table of values arranged on two axes, lines and columns. But unlike a two -dimensional matrix, a tensor can have many dimensions or axes, which makes tensors more difficult to handle.
Deep learning models carry out operations on tensors using the multiplication and addition of repeated matrix – this process is how neural networks learn complex models in data. The volume of calculations which must be carried out on these multidimensional data structures requires a huge amount of calculation and energy.
But due to the way in which data in tensors are organized, engineers can often increase the speed of a neural network by deleting redundant calculations.
For example, if a tensor represents the examination data for users of an e -commerce site, as not all users have examined all the products, most of the values in that the tensor are probably not zero. This type of data redundancy is called rarity. A model can save time and calculation by storing and operating on non -zero values.
In addition, sometimes a tensor is symmetrical, which means that the upper half and the lower half of the data structure are equal. In this case, the model should only work on half, reducing the quantity of calculation. This type of data redundancy is called symmetry.
“But when you try to capture these two optimizations, the situation becomes quite complex,” said Ahrens.
To simplify the process, she and her collaborators have built a new compiler, which is a computer program that translates the complex code into a simpler language which can be treated by a machine. Their compiler, called Systec, can optimize calculations by automatically taking advantage of scarcity and symmetry in tensors.
They started the system of system construction by identifying three key optimizations that they can perform using symmetry.
First of all, if the exit tensor of the algorithm is symmetrical, it only has to calculate half. Second, if the input tensor is symmetrical, the algorithm only needs to read half of it. Finally, if the intermediate results of the tensor operations are symmetrical, the algorithm can ignore redundant calculations.
Simultaneous optimizations
To use Systec, a developer enters its program and the system automatically optimizes their code for the three types of symmetry. Then, the second phase of Systec performs additional transformations to store only non -zero data values, optimizing the rarity program.
In the end, Systec generates ready -to -use code.
“In this way, we obtain the advantages of the two optimizations. And the interesting thing about symmetry is, because your tensor has more dimensions, you can achieve even more savings on the calculation, ”says Ahrens.
The researchers have demonstrated accelerations of almost a factor of 30 with the code generated automatically by Systec.
Because the system is automated, it could be particularly useful in situations where a scientist wants to process data using an algorithm that he writes from zero.
In the future, researchers wish to integrate the system into existing light tensor compiler systems to create a transparent interface for users. In addition, they would like to use it to optimize the code for more complicated programs.
This work is funded, in part, by Intel, the National Science Foundation, the Defense Advanced Research Projects Agency and the Department of Energy.
