If you rotate an image of a molecular structure, a human can say that the turned image is always the same molecule, but a machine learning model may think that it is a new data point. In computer language, the molecule is “symmetrical”, which means that the fundamental structure of this molecule remains the same if it undergoes certain transformations, such as rotation.
If a drug discovery model does not include symmetry, it could make inaccurate predictions on molecular properties. But despite certain empirical successes, it is not clear if there is an effective method to form a good model which is guaranteed to respect symmetry.
A new study by MIT researchers answers this question and shows the first automatic learning method with symmetry which can be proven in terms of calculation and necessary data.
These results clarify a fundamental question, and they could help researchers in the development of more powerful machine learning models designed to manage symmetry. These models would be useful in a variety of applications, from the discovery of new materials to the identification of astronomical anomalies to the detangling of complex climatic models.
“These symmetries are important because they are a kind of information that nature tells us about the data, and we must take it into account in our automatic learning models. We have now shown that it is possible to make a machine learning with symmetrical data in an effective way of this study.
He is joined on the paper by the author co-directed and student graduated from the MIT Ashkan Soleymani; Stefanie Jegelka, associate professor of electrical and computer engineering (EECS) and member of the Institute for Data, Systems and Society (IDSS) and the Laboratory of IT and Artificial Intelligence (CSAIL); And the main author Patrick Jaillet, the professor of electrical and computer engineering at Dugald C. Jackson and main computer science in the laboratory for information and decision -making systems (LIDS). Research has recently been presented at the international conference on automatic learning.
Study symmetry
Symmetrical data appear in many areas, in particular natural sciences and physics. A model that recognizes symmetries is capable of identifying an object, such as a car, it doesn't matter where this object is placed in an image, for example.
Unless a machine learning model is designed to manage symmetry, it can be less precise and subject to failure in the face of new symmetrical data in real situations. On the other hand, the models that benefit from symmetry could be faster and require less data for training.
But the formation of a model to process symmetrical data is not an easy task.
A common approach is called increased data, where researchers transform each symmetrical data point into several data points to help the model better become widespread in new data. For example, we could run a molecular structure several times to produce new training data, but if the researchers want the model to be guaranteed to respect symmetry, this can be prohibitive by calculation.
An alternative approach is to code symmetry in the architecture of the model. A well -known example of this is a network of graphic neurons (GNN), which intrinsically manages symmetrical data because of the way it is designed.
“The graphic neurons networks are fast and effective, and they take care of symmetry, but no one really knows what these models learn or why they work. Understanding the GNNS is a main motivation of our work, so we started with a theoretical evaluation of what is happening when the data is symmetrical,” said Tahmasebi.
They explored the statistical-computer compromise in automatic learning with symmetrical data. This compromise means that methods that require less data can be more expensive in calculation, so researchers must find the right balance.
Based on this theoretical evaluation, researchers have designed an effective algorithm for automatic learning with symmetrical data.
Mathematical combinations
To do this, they borrowed ideas from algebra to shrink and simplify the problem. Then they reformulated the problem using ideas of geometry that effectively capture symmetry.
Finally, they combined algebra and geometry into an optimization problem which can be effectively resolved, resulting in their new algorithm.
“Most theory and applications focused on algebra or geometry. Here we have just combined them, ”explains Tahmasebi.
Algorithm requires fewer data samples for training than conventional approaches, which would improve the accuracy and capacity of a model to adapt to new applications.
By proving that scientists can develop effective algorithms for automatic learning with symmetry, and demonstrating how it can be done, these results could lead to the development of new neural network architectures which could be more precise and less with high intensity of resources than current models.
Scientists could also use this analysis as a starting point to examine the internal functioning of NNGs, and how their operations differ from the algorithm that MIT researchers have developed.
“Once we know better, we can design more interpretable, more robust and more effective neural network architectures,” adds Soleymani.
This research is funded, in part, by the National Research Foundation of Singapore, DSO National Laboratories of Singapore, US Office of Naval Research, the US National Science Foundation and a professor Alexander von Humboldt.