An interesting scientific experience was conducted by researchers Isaac Kauvar and Chris Doyle, when they decided to determine who would excel in a top-of-one competition: the most modern IA agent or a mouse. Their Revolutionary experience, conducted at the Wu tsai neurosciences Institute in Stanfordaimed at being inspired by the natural skills of animals to improve the performance of AI systems.
The researchers have designed a simple task, motivated by their interest in animal exploration and adaptability. They placed a mouse in an empty box and an agent of ia simulated in a virtual 3D arena, both with a red bullet. The objective was to observe which subject would explore the new object more quickly.
To their surprise, the mouse quickly approached and interacted with the red ball, while the agent seemed unconscious of his presence. This unexpected result led to a deep achievement: even with the most advanced algorithm, there were still gaps in AI performance.
This revelation triggered curiosity in scholars. Could they exploit apparently simple animal behavior to strengthen AI systems? Determined to explore this potential, Kauvar, Doyle, as well as the graduate student Linqi Zhou and under the direction of deputy professor Nick Haber, launched into the design of a new training method called “Curious Replay”.
Curious Replay was aimed at encouraging AI agents to reflect on new and intriguing meetings, a bit like the mouse presented with the red ball. The addition of this method turned out to be the missing part, because it allowed the IA agent to get involved quickly with the red ball.
The importance of curiosity in our lives extends beyond intellectual activities. He plays an essential role in survival by helping us to sail in dangerous situations. Understanding the importance of curiosity, laboratories like Haber have incorporated a signal of curiosity in AI agents, in particular the learning agents in deep reinforcement based on a model. This signal encourages them to select actions that lead to more interesting results rather than reject potential opportunities.
However, Kauvar, Doyle and their team pushed curiosity a little further, by employing it to promote the understanding of its environment by the AI agent. Instead of only guiding decision -making, the researchers wanted the agent of AI to envisage and reflected on intriguing experiences, which stimulates his curiosity.
To achieve this, they adapted the common method of rereading experience used in the training of AI agents. The rereading of the experience consists in storing memories of interactions and replaying them at random to strengthen learning, as well as the hippocampus of the reactive brain certain neurons during sleep to improve memories. However, in a changing environment, the replay of all experiences may not be effective. Consequently, the researchers proposed a new approach, prioritizing the replay of the most interesting experiences, such as meeting with the red ball.
Nicknamed “Curious Replay”, this method has demonstrated immediate success, encouraging the AI agent to interact with the ball faster and more efficiently.
The success of Curious Replay promises to shape the future of AI research. By facilitating the effective exploration of agents of new or changing environments, it opens ways for more adaptive and flexible technologies, benefiting domains such as household robotics and personalized learning tools.
This research aims to fill the gap between AI and neuroscience, improving our understanding of animal behavior and underlying neural processes. You can read the full study on Curious Replay here.
