Despite the substantial progress of the image generation of text (T2I) caused by models such as Dall -E 3, Imagen 3 and Stable Diffusion 3, reaching the quality of coherent output – both in aesthetic and alignment terms – remains a persistent challenge. Although large-scale pre-training provides general knowledge, it is insufficient to achieve high quality and alignment. The supervised fine setting (SFT) serves a critical post-training step, but its efficiency strongly depends on the quality of the fine adjustment data set.
Current public data sets used in SFT target narrow visual domains (for example, specific anime or art genres), be based on basic heuristic filters on web level data. The conservation led by humans is expensive, non -liable and frequently cannot identify the samples that give the greatest improvements. In addition, recent T2I models use internal proprietary data sets with minimum transparency, limiting the reproducibility of results and slowing down collective progress in the field.
Approach: a conservation of the data set guided by the model
To alleviate these problems, Yandex released Alchemist,, A set of SFT data for general use accessible to the public composed of 3,350 pairs of carefully selected image text. Unlike conventional data sets, Alchemist is built using a new methodology that exploits a pre-formulated dissemination model to act as an estimator of the quality of the sample. This approach allows the selection of training data with a high impact on the generative performance of the model without relying on a subjective human labeling or a simplistic aesthetic notation.
Alchemist is designed to improve the output quality of T2I models thanks to a fine targeted adjustment. The press release also includes refined versions of five stable distribution models accessible to the public. The data set and the models are accessible on Face under an open license. Learn more about methodology and experiences – in the pre -impression .
Technical design: filtering of pipelines and data
The construction of alchemist involves a several -stage filtering pipeline from web original images. The pipeline is structured as follows:
- Initial filtering: Deletion of the NSFW content and low -resolution images (threshold> 1024 × 1024 pixels).
- Coarse: Application of classifiers to exclude images with compression artifacts, a vague movement, watermark and other defects. These classifiers have been trained on standard quality image quality assessment sets such as Koniq-10k and Pipal.
- IQA deduplication and pruning: SIFT type features are used to group similar images, keeping only high quality images. The images are still noted using the Topiq model, ensuring the retention of clean samples.
- Diffusion -based selection: A key contribution is the use of the activations of the cross attention of a pre-formed diffusion model to classify the images. A rating function identifies samples that strongly activate the characteristics associated with visual complexity, aesthetic appeal and stylistic wealth. This allows the selection of the samples most likely to improve the performance of the downstream model.
- Rewriting legends: The final selected images are recovered using an adjusted language vision model to produce textual style descriptions. This step guarantees better alignment and better conviviality in SFT workflows.
Thanks to ablation studies, the authors determine that the increase in the size of the data set beyond 3,350 (for example, the 7K or 19K samples) leads to a lower quality of the refined models, reinforcing the value of the targeted and high quality data compared to the gross volume.
Results on several T2i models
The effectiveness of the alchemist was evaluated through five stable diffusion variants: SD1.5, SD2.1, SDXL, SD3.5 Medium and SD3.5 wide. Each model has been refined using three data sets: (i) the alchemist data set, (ii) a paired size subset of Laion-Elestics V2 and (iii) their respective baselines.
Human evaluation: Expert annotators have carried out evaluations side by side through four criteria – Relevance of text -image, aesthetic quality, complexity of the image and loyalty. The models set by alchemists have shown statistically significant improvements in aesthetic and complexity scores, often surpassing both the basic lines and the versions set by aesthetics by margins of 12 to 20%. Above all, the relevance of the text-image has remained stable, which suggests that rapid alignment has not been negatively affected.
Automated metrics: Through metrics such as FD-Dinov2, Score Clip, ImageWard and HPS-V2, the models set by alchemists have generally obtained a higher score than their counterparts. In particular, the improvements were more consistent with respect to models based on matches of size than the basic models.
Ablation of the size of the data set: The fine setting with larger variants of alchemist (7K and 19K samples) has led to lower performance, stressing that the stricter filtering and the higher sample quality are more impactful than the size of the data set.
Yandex used the data set to form its generative text model in the owner image, Yandexart v2.5, and plans to continue to pull it for future updates of the model.
Conclusion
Alchemist Provides a well -defined and empirically validated route to improve the quality of the text generation to the image via a supervised fine adjustment.
Although the improvements are the most remarkable in perceptual attributes such as the aesthetics and the complexity of the image, the frame also highlights the compromises that occur in fidelity, in particular for the more recent basic models already optimized thanks to the internal SFT. However, Alchemist establishes a new standard for SFT data sets for general use and offers a precious resource for researchers and developers working to advance the output quality of generative vision models.
Discover the Paper here And Alchemist Face data set. Thank you to the Yandex team for leadership / opinion resources for this article.
Asif Razzaq is the CEO of Marktechpost Media Inc .. as a visionary entrepreneur and engineer, AIF undertakes to exploit the potential of artificial intelligence for social good. His most recent company is the launch of an artificial intelligence media platform, Marktechpost, which stands out from its in-depth coverage of automatic learning and in-depth learning news which are both technically solid and easily understandable by a large audience. The platform has more than 2 million monthly views, illustrating its popularity with the public.
