Baai launches omnigen2: a unified diffusion and transformer model for multimodal AI

The Beijing artificial intelligence academy (BAAI) presents Omnigen2, a new generation open-generation multimodal generative model. By developing its omnigen predecessor, the new architecture unifies the text generation to the image, the image edition and the generation focused on the subject in a single framework of transformer. It innovates by decoupling the modeling of the generation of text and image, incorporating a reflective training mechanism and by implementing a specially designed reference – Omnicontext – to assess contextual consistency.

A decoupled multimodal architecture

Unlike previous models that use shared parameters through text and image methods, Omnigen2 introduces two distinct paths: an autoregressive transformer for text generation and a transformer based on diffusion for image synthesis. It also uses a new positioning strategy called Omni-Drope, which allows flexible handling of sequences, spatial coordinates and distinctions of modality, allowing the generation and editing of high fidelity images.

To preserve the capacity for generation of pre-trained text from the underlying MLLM (based on QWEN2.5-VL-3B), OMNIGEN2 nourishes the characteristics derived from VAE only to the diffusion route. This avoids compromising the understanding of the text and the generation capacities of the model while maintaining a rich visual representation for the image synthesis module.

Reflection mechanism for iterative generation

One of the remarkable characteristics of Omnigen2 is the mechanism of reflection. By integrating the feedback loops during training, the model is capable of analyzing its generated outings, identifying inconsistencies and offering refinements. This process imitates the self-corners of the test time and considerably improves the accuracy of monitoring of instructions and visual coherence, in particular for nuanced tasks such as the modification of the color, the number of objects or the positioning.

The set of reflection data has been built using multi-round comments, allowing the model to learn to revise and terminate the generation according to content assessment. This mechanism is particularly useful for filling the quality gap between open-source and commercial models.

Omnicontext benchmark: Contextual coherence assessment

To rigorously assess the generation in the context, the team introduces Omnicontext, a reference comprising three main types of tasks: simple, multiple and scene, through character, object and stage categories. Omnigen2 demonstrates advanced performance among the open source models in this area, marking 7.18 in total – via other leading models such as Bagel and UniWorld -V1.

The evaluation uses three basic measurements: Next invitation (PF), subject's consistency (SC) and overall score (geometric average), each validated by reasoning based on GPT-4.1. This framework of comparative analysis emphasizes not only on visual realism, but also semantic alignment with the invites and consistency of the cross -image.

Data pipeline and training corpus

Omnigen2 was formed on 140 m T2I samples and owner images of 10 m, supplemented by meticulously organized data sets for generation and publishing in context. These data sets were built using a video-based pipeline that extracts semantically coherent frame pairs and automatically generates instructions using QWEN2.5-VL models. The resulting annotations cover manipulations of fine -grained images, movement variations and composition changes.

For training, the MLLM parameters remain largely frozen to maintain a general understanding, while the diffusion module is trained from zero and optimized for visual attention-joint target. A special token “<| img |>»Triggers the generation of images in output sequences, rationalizing the multimodal synthetic process.

Performance between tasks

Omnigen2 provides solid results in several areas:

• Text in the image (T2i): Reached a score of 0.86 on Geneval and 83.57 on DPG-Bench.
• Image edition: Open the open source basic lines with high semantic coherence (SC = 7.16).
• Generation in context: Defines new benchmarks in Omnicontext with 7.81 (simple), 7.23 (multiple) and 6.71 (scene) tasks scores.
• Reflection: Demonstrates an effective revision of failing generations, with promising correction accuracy and termination behavior.

Conclusion

Omnigen2 is a robust and effective multimodal generative system that advances modeling unified by architectural separation, high quality data pipelines and an integrated reflection mechanism. By models, data sets and open source code, the project sets a solid base for future research in generation of controllable and coherent image text. Upcoming improvements can focus on learning strengthening for reflection of reflection and expansion of multilingual and low quality robustness.