Nvidia revealed Nvidia CosmosAn innovative platform designed to accelerate the development of physical AI – artificial intelligence behind robots, autonomous vehicles (AVS) and other automated real world systems. By combining advanced world foundation models (WFMS), advanced video processing tools and a Data pipeline based on AI, Cosmos allows developers to create, train and optimize AI models more efficiently than ever.
The development of a physical AI traditionally required massive quantities of real data, making it an expensive and prolonged process. Nvidia Cosmos aims to change this by offering a generation of synthetic data based on physics, allowing developers to create photorealist 3D environments that imitate real conditions. These simulated environments help form AI models without fully based on expensive expensive costly data.
Nvidia describes the World Foundation models as fundamental for the next wave of AI, as well as the models of large languages (LLMS) have revolutionized the treatment of natural language. WFMS use a combination of text, images, video and sensors data to simulate real world interactions, which makes them essential for robotics and autonomous systems that must navigate in complex environments.
Cosmos includes a range of advanced AI tools adapted to the development of robotics and AV:
- Generation of synthetic data – Using the cosmos, developers can create high -fidelity video simulations and aware of industrial and conduct physics, reducing dependence on the collection of real world data.
- Video research and understanding – The research capacities supplied by AI allow users to quickly locate specific training scenarios, such as dangerous road conditions or crowded warehouse environments.
- Predictive intelligence and “multive” simulation – The cosmos can simulate several potential results of a real world scenario, helping AI models to predict the best action plan.
- Advanced data processing – NVIDIa's preservative Nvidia accelerates the processing of massive video data sets, making AI training more efficient.
Cosmos also introduces a visual tokenizer, which can compress and process video data 12 times faster than existing methods, which facilitates it video recordings into usable training data.
Several main robotics and car companies have already started to integrate Cosmos into their IA workflows. Among them, Xpeng, Agility Robotics, Figure AI, Wayve and Uber, each taking advantage of the cosmos to develop new generation AVs and humanoid robots. For example, Waabi, a company focused on autonomous driving directed by AI, uses the cosmos for data preservation and AV simulation, while Uber works with Nvidia to advance autonomous mobility solutions.
As the content generated by AI spreads, Nvidia has built the cosmos with strong ethical guarantees. The platform includes railings to prevent generation of harmful or deceptive content, as well as invisible watermark to identify the videos generated by AI. Cosmos alignments with global IA security initiatives, including the voluntary commitments of the White House AI.
Nvidia Cosmos is now available under an open model license on Hugging Face and the Nvidia NGC catalog. With a physical AI on the verge of transforming the manufacturing industries to transport, Nvidia Cosmos marks an important step towards the writing of robotics led by the more scalable, efficient and widely available AI.
Learn more about Cosmos World Foundation Model Platform for PhysicsIn the article available on Arxiv.
