Weak-to-Strong Diffusion with Reflection

🔼 Figure 2 presents a qualitative comparison of image generation results between a standard diffusion model and the proposed Weak-to-Strong Diffusion with Reflection (W2SD) method. The figure showcases examples highlighting W2SD’s improvements across multiple aspects of image generation. These aspects include accurate text rendering within images, precise object positioning, correct color representation, accurate counting of objects, and appropriate co-occurrence of objects within a scene. The results suggest that W2SD enhances the overall quality and coherence of generated images. More examples are available in Appendix C.2.

read the caption

Figure 2: The qualitative results of W2SD demonstrate the effectiveness of our method in various aspects, such as text rendering, position, color, counting, and object co-occurrence. We present more cases in Appendix C.2.

🔼 This figure illustrates the core concept of Weak-to-Strong Diffusion with Reflection (W2SD). It shows how W2SD uses the difference between a weak and a strong diffusion model to approximate the gap between the strong model and an ideal model that perfectly represents the real data distribution. The figure visually demonstrates that by iteratively applying a reflective operation that uses the weak-to-strong difference (Δ₁), the latent variable (x̃t) is guided towards the ideal latent variable (xgt). When the weak-to-strong difference is a good approximation of the strong-to-ideal difference (Δ₂ - Δ₁ is small), the refined latent variable converges to the ideal latent variable, thereby improving the quality of the generated samples.

read the caption

Figure 3: Visualizing the effectiveness of W2SD. When the weak-to-strong difference closely approximates the strong-to-ideal difference (e.g., Δ2⁢(t)−Δ1⁢(t)subscriptΔ2𝑡subscriptΔ1𝑡\Delta_{2}(t)-\Delta_{1}(t)roman_Δ start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT ( italic_t ) - roman_Δ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT ( italic_t ) is small), the refined latent variable x~tsubscript~𝑥𝑡\tilde{x}_{t}over~ start_ARG italic_x end_ARG start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT converges to the ideal latent variable xtgtsuperscriptsubscript𝑥𝑡gtx_{t}^{\mathrm{gt}}italic_x start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT start_POSTSUPERSCRIPT roman_gt end_POSTSUPERSCRIPT.

🔼 Figure 4 illustrates the denoising process of a 1D Gaussian distribution with two peaks. A weak model, trained on data lacking samples from the left peak, only generates data around the right peak (blue line). A stronger model, trained on a more complete dataset, generates data covering both peaks (red line). The proposed Weak-to-Strong Diffusion with Reflection (W2SD) method, using a reflective operator that combines the weak and strong models, effectively balances the generated distribution, producing samples from both peaks (purple line). This demonstrates W2SD’s ability to correct for deficiencies in model training data.

read the caption

Figure 4: Denoising trajectories across different settings (1-D Gauss). The weak model (blue) generates only right-peak data due to missing left-peak training samples, while the strong model (red) produces data between both peaks. W2SD balances the distribution by leveraging the reflective operator ℳinvw⁢(ℳs⁢(⋅))superscriptsubscriptℳinvwsuperscriptℳs⋅\mathcal{M}_{\mathrm{inv}}^{\mathrm{w}}(\mathcal{M}^{\mathrm{s}}(\cdot))caligraphic_M start_POSTSUBSCRIPT roman_inv end_POSTSUBSCRIPT start_POSTSUPERSCRIPT roman_w end_POSTSUPERSCRIPT ( caligraphic_M start_POSTSUPERSCRIPT roman_s end_POSTSUPERSCRIPT ( ⋅ ) ).

🔼 This figure visualizes the effectiveness of Weak-to-Strong Diffusion with Reflection (W2SD) on a 2-dimensional Gaussian mixture dataset. It shows probability contour plots and sampling trajectories for three different methods: a strong model (trained on a biased dataset), a weak model (trained on another biased dataset), and W2SD. The contour plots illustrate the learned data distributions of each method, with the ideal distribution represented by the real data distribution. The trajectories show how the latent variable evolves during the sampling process. The results demonstrate that W2SD effectively balances the learned distribution, bringing it closer to the true underlying distribution of the data, showcasing its ability to bridge the gap between weak and strong models and improve the quality of generated samples.

read the caption

Figure 5: Probability contour plot and denoising trajectories across different settings (2-D Gauss). W2SD balances the learned distribution, bringing it closer to the real data distribution

🔼 This figure displays a comparison of image generation results from a strong model, a weak model, and the proposed W2SD method, all trained on subsets of the CIFAR-10 dataset. The strong model shows a bias towards generating horse images, while the weak model struggles to generate either cars or horses effectively. The W2SD method mitigates this bias, resulting in a more balanced distribution with a higher likelihood of generating car images, showcasing the method’s ability to improve the diversity and quality of image generation by addressing imbalances in training data.

read the caption

Figure 6: Qualitative results of W2SD based on dataset differences (CIFAR-10). Our method enhances the probability of generating “cars” and promote a more balanced generation distribution.

🔼 This figure visualizes the impact of W2SD on data generation balance using t-SNE to project 32x32x3 CLIP features into a 2D space. It compares four scenarios: (a) a strong model (Ms) effectively generates cars; (b) a weak model (Mw) struggles to generate cars; (c) W2SD balances car and horse generation; and (d) applying the inverse weak model (Minv(Mw)) exacerbates the imbalance, showcasing W2SD’s ability to improve data generation balance.

read the caption

Figure 7: The CLIP feature corresponding to the generated image (32×\times×32×\times×3) is projected into a 2D space. W2SD effectively disentangles the representations of “car” and “horse” in the 2D space. (a) ℳssuperscriptℳs\mathcal{M}^{\mathrm{s}}caligraphic_M start_POSTSUPERSCRIPT roman_s end_POSTSUPERSCRIPT demonstrates the ability to generate cars; (b) ℳwsuperscriptℳw\mathcal{M}^{\mathrm{w}}caligraphic_M start_POSTSUPERSCRIPT roman_w end_POSTSUPERSCRIPT can hardly generate cars; (c) W2SD balances the generation distribution, increasing the likelihood of generating cars; (d) S2WD (i.e., ℳi⁢n⁢vs⁢(ℳw⁢(⋅))superscriptsubscriptℳ𝑖𝑛𝑣𝑠superscriptℳ𝑤⋅\mathcal{M}_{inv}^{s}(\mathcal{M}^{w}(\cdot))caligraphic_M start_POSTSUBSCRIPT italic_i italic_n italic_v end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_s end_POSTSUPERSCRIPT ( caligraphic_M start_POSTSUPERSCRIPT italic_w end_POSTSUPERSCRIPT ( ⋅ ) )) exacerbates the imbalance in data generation.

🔼 Figure 8 displays a qualitative comparison of image generation results from three different methods: a weak model, a strong model, and the proposed Weak-to-Strong Diffusion with Reflection (W2SD) method. The weak model produces lower quality results, while the strong model generates images with improved quality but still exhibits certain limitations. The W2SD method leverages the differences between these models to produce improved results across various dimensions including style, character, and clothing detail. The images demonstrate W2SD’s ability to refine the generated output, resulting in more visually appealing results. More examples are shown in Section C.2 of the paper.

read the caption

Figure 8: Qualitative comparisons with weak model (left), strong model (middle) and W2SD based on weight difference (right). Our method utilizes the differences between chosen strong and weak models (e.g., high-detail LoRA vs. standard model) to deliver improvements in various dimensions, including style, character, clothing, and beyond. We provide more qualitative results in Section C.2.

🔼 This figure displays a comparison of image generation results using DiT-MoE-S (a Mixture-of-Experts model) with and without the Weak-to-Strong Diffusion with Reflection (W2SD) method. The top row shows samples generated by the DiT-MoE-S model alone, demonstrating its limitations, especially given its relatively small size (71 million activated parameters). The bottom row presents results obtained after applying the W2SD technique to the same model. The visual difference showcases the significant improvements in image quality achieved by W2SD, highlighting its ability to enhance performance even in resource-constrained models.

read the caption

Figure 9: Quantitative Results of W2SD Based on the MoE Mechanism. The first row shows the results for DiT-MoE-S, while the second row presents W2SD. W2SD achieves significant improvements, even with small models featuring 71M activated parameters.

🔼 This figure demonstrates the effectiveness of Weak-to-Strong Diffusion with Reflection (W2SD) when using different sampling pipelines. ControlNet, a pipeline that incorporates additional network structures to allow for controllable image generation based on reference images (like edge maps), is used as the strong model (ℳ^s). The standard DDIM sampling pipeline is used as the weak model (ℳ^w). The figure shows that by employing W2SD, the generated images exhibit significantly improved alignment with the provided reference images, highlighting the method’s ability to enhance generation quality across diverse sampling methodologies.

read the caption

Figure 10: Qualitative results of W2SD based on pipeline difference. We set ControlNet as ℳssuperscriptℳs\mathcal{M}^{\mathrm{s}}caligraphic_M start_POSTSUPERSCRIPT roman_s end_POSTSUPERSCRIPT, DDIM as ℳwsuperscriptℳw\mathcal{M}^{\mathrm{w}}caligraphic_M start_POSTSUPERSCRIPT roman_w end_POSTSUPERSCRIPT. W2SD improves alignment with reference images.

🔼 This figure analyzes the relationship between the magnitude of the difference between a strong and a weak diffusion model and the improvement in image quality achieved by the Weak-to-Strong Diffusion with Reflection (W2SD) method. The x-axis represents the magnitude of this difference, while the y-axis shows the average HPS v2 score (a metric measuring human preference for generated images) on the Pick-a-Pic dataset. The graph reveals that when the strong model is significantly better than the weak model (large positive difference), W2SD leads to substantial improvements in the HPS v2 score. Conversely, if the strong model is weaker or comparable to the weak model (small or negative difference), W2SD may even yield negative gains, indicating a reduction in image quality compared to using the strong model alone.

read the caption

Figure 11: The magnitude of weak-to-strong difference is a key factor impacting the effects of improvements. The horizontal axis shows the magnitude of the weak-to-strong difference, while the vertical axis shows the average HPS v2 on the Pick-a-Pic. When ℳssuperscriptℳs\mathcal{M}^{\mathrm{s}}caligraphic_M start_POSTSUPERSCRIPT roman_s end_POSTSUPERSCRIPT is weaker than ℳwsuperscriptℳw\mathcal{M}^{\mathrm{w}}caligraphic_M start_POSTSUPERSCRIPT roman_w end_POSTSUPERSCRIPT, W2SD results in negative gains.

🔼 This figure demonstrates the impact of the magnitude of the weak-to-strong difference on the performance of the Weak-to-Strong Diffusion with Reflection (W2SD) method. The x-axis represents the difference between a strong and weak model, while the y-axis shows the improvement in the Human Preference Score v2 (HPSv2) metric. When the weak-to-strong difference is positive (strong model is significantly better than weak model), W2SD improves the results. If the difference is zero (models have similar performance), W2SD performs identically to standard sampling. If the difference is negative (weak model performs better than strong model), the results degrade, indicating that a strong model is crucial for the effectiveness of W2SD. The plots show both the results using LoRA-based methods and using different guidance scales to illustrate the versatility of this finding across different methods of establishing the weak-to-strong model pair.

read the caption

Figure 12: When the weak-to-strong difference is greater than 0, W2SD yields positive gains. When it equals 0, the process degenerates into standard sampling. When it is less than 0, negative gains occurs, resulting in poor image quality.

🔼 This figure compares the performance of W2SD against standard sampling methods in terms of generation time and image quality. The x-axis represents the average time taken to generate a single image, while the y-axis shows the HPS v2 score (a metric evaluating human preference for image quality) achieved on the Pick-a-Pic dataset. The results demonstrate that W2SD achieves a higher HPS v2 score than standard sampling while maintaining the same average generation time per image, highlighting its efficiency in improving image quality without increasing computational costs.

read the caption

Figure 13: W2SD outperforms standard sampling with identical time costs. The horizontal axis denotes the average generation time per image, the vertical axis represents the HPS v2 on Pick-a-Pic.

🔼 Table 8 presents a quantitative comparison of image generation results using a standard model and the W2SD method. The evaluation was conducted on the Drawbench dataset, which assesses various aspects of image quality relevant to human preferences. The table shows that W2SD, using a full parameter fine-tuning strategy, leads to improvements in multiple metrics, indicating better alignment with human aesthetic preferences compared to the standard model.

read the caption

Table 8: Quantitative results of W2SD based on a full parameter fine-tuning strategy. Our method generates results better aligned with human preferences. Datasets: Drawbench.

🔼 Table 9 presents a quantitative analysis of the Weak-to-Strong Diffusion with Reflection (W2SD) method. Specifically, it shows the performance improvements achieved by W2SD when leveraging differences in guidance scales between a strong and weak model. The model used is SDXL, and the results are evaluated using the DrawBench dataset. The table likely includes metrics such as FID, Inception Score, Precision, Recall, and potentially others to comprehensively assess the impact of W2SD on image generation quality under varying guidance levels.

read the caption

Table 9: Quantitative results of W2SD based on guidance difference. Model: SDXL. Datasets: DrawBench.

🔼 This table presents a quantitative analysis of the Weak-to-Strong Diffusion with Reflection (W2SD) method. The method is evaluated using a human preference LoRA model, specifically assessing its ability to generate results that are better aligned with human preferences. The results are compared against baseline models without W2SD on the Pick-a-Pic dataset. Metrics such as HPSv2, AES, PickScore, and MPS are used to evaluate different aspects of image quality and user preference, allowing for a comprehensive assessment of the method’s effectiveness.

read the caption

Table 10: Quantitative results of W2SD based on human preference LoRA model. Our method generates results better aligned with human preferences. Datasets: Pick-a-Pic.

🔼 Table 11 presents a quantitative analysis of the Weak-to-Strong Diffusion with Reflection (W2SD) method. The study focuses on how W2SD, using a human preference LoRA model, improves the alignment of generated results with human preferences. The results are evaluated using the DrawBench dataset, a comprehensive benchmark for evaluating image generation models. The table likely compares W2SD’s performance to other baseline models across various metrics relevant to image quality and human perception, showcasing the effectiveness of the W2SD technique.

read the caption

Table 11: Quantitative results of W2SD based on human preference LoRA model. Our method generates results better aligned with human preferences. Datasets: DrawBench.

🔼 This figure displays a qualitative comparison of image generation results using three different models: a weak model (SDXL), a strong model (xlMoreArtFullV1), and the proposed W2SD method. Each row shows the same prompt’s output for the three models. The images generated by the weak model alone often lack detail or deviate significantly from the prompt’s intent. The strong model generally produces more aesthetically pleasing and accurate results. W2SD, combining elements of both weak and strong models, produces images that are closer to the strong model’s quality but often contain more accurate and prompt-relevant details than the strong model alone. This demonstrates W2SD’s capability to enhance human preference by balancing the strengths of weaker and stronger models.

read the caption

Figure 14: Qualitative results of W2SD based on weight differences (human preference). Here we select xlMoreArtFullV1 as the strong model and SDXL as the weak model. W2SD can effectively enhance the performance of human preference.

🔼 Table 2 presents a quantitative comparison of the performance of the Weak-to-Strong Diffusion with Reflection (W2SD) method, using a personalized LoRA (Low-Rank Adaptation) model. The experiment focuses on measuring the impact of W2SD when the difference in model weights between a base model (SD1.5) and a LoRA-tuned version (SD1.5 + LoRA) is used to steer generation towards more personalized outputs. The table likely shows various metrics to evaluate this personalization, such as metrics measuring image quality, alignment with prompts, and possibly others. The results demonstrate how W2SD leverages this weight difference to enhance the results, showcasing the efficacy of the method in achieving personalized generation.

read the caption

Table 2: Quantitative results of W2SD based on personalized LoRA model. Here, the weight difference between ℳwsuperscriptℳ𝑤\mathcal{M}^{w}caligraphic_M start_POSTSUPERSCRIPT italic_w end_POSTSUPERSCRIPT (SD1.5) and ℳssuperscriptℳ𝑠\mathcal{M}^{s}caligraphic_M start_POSTSUPERSCRIPT italic_s end_POSTSUPERSCRIPT (SD1.5 +LoRA) biases the generated results towards a more personalized direction.

🔼 This table presents a quantitative analysis of the Weak-to-Strong Diffusion with Reflection (W2SD) method when applied to Mixture of Experts (MoE) models, using the ImageNet 50K dataset. It shows the improvements achieved by W2SD compared to a standard DiT-MoE-S model across several metrics, including Inception Score (IS), Fréchet Inception Distance (FID), Average Expert Selection (AES), and Human Preference Score version 2 (HPSv2). These metrics assess different aspects of image quality and generation, helping to demonstrate the effectiveness of W2SD in enhancing image generation performance within the MoE framework.

read the caption

Table 3: Quantitative results of W2SD based on MoE Mechanism. Datasets: ImageNet 50K.

🔼 This table presents a quantitative comparison of the performance of the Weak-to-Strong Diffusion with Reflection (W2SD) method against a standard SDXL model, using the Pick-a-Pic dataset. The comparison focuses specifically on the impact of varying the guidance scale, a parameter that influences the balance between the input prompt and the model’s own generation tendencies. The results are evaluated across four metrics: HPS v2 (Human Preference Score v2), AES (Aesthetic Quality), PickScore, and MPS (multi-perspective score). These metrics offer a comprehensive assessment of the generated images’ quality, aligning with human preferences and covering various aspects of image quality. The table helps illustrate how leveraging the differences in guidance scales (as a proxy for a weak-strong model difference) improves image generation by W2SD.

read the caption

Table 4: Quantitative results of W2SD based on guidance difference. Model: SDXL. Datasets: Pick-a-Pic.

🔼 This table presents a quantitative analysis of the Weak-to-Strong Diffusion with Reflection (W2SD) method. Specifically, it evaluates the performance improvements achieved by W2SD when leveraging semantic differences between prompts. The model used is SDXL, and the dataset is GenEval. The table likely shows various metrics (e.g., HPS v2, AES, PickScore, MPS) to assess the quality of generated images, comparing the results obtained with W2SD against a baseline method (likely the SDXL model without W2SD). The results demonstrate the effectiveness of W2SD in enhancing the image generation quality by utilizing nuanced prompt variations.

read the caption

Table 5: Quantitative results of W2SD based on semantic differences between prompts. Model: SDXL. Datasets: GenEval.

🔼 This table demonstrates that the improvements from different types of model differences (weight difference and condition difference) in W2SD can be cumulative. It shows that combining these differences leads to better results than using either one alone. The Pick-a-Pic dataset is used for evaluation, and the specific metric shown is the HPSv2 score, which reflects human preference.

read the caption

Table 6: The improvements effects from different model differences can be cumulative. Datasets: Pick-a-Pic.

🔼 This table shows different ways to create a ‘weak-to-strong’ model difference for improving diffusion models. Each row represents a different method of creating this difference, along with the specific models or techniques used, and a reference to the section in the paper where the results are discussed. The types of model differences explored include variations in model weights (full finetuning, LoRA, Mixture of Experts), differences in conditioning (guidance scale, prompt semantics), and differences in sampling pipelines.

read the caption

Table 7: Different types of model differences lead to improvements effects in different directions.

🔼 Table 12 presents a quantitative analysis of the Weak-to-Strong Diffusion with Reflection (W2SD) method on video generation tasks. The model used is AnimateDiff, and the evaluation is performed using the VBench dataset. The table shows the performance improvements achieved by W2SD across various metrics relevant to video quality. These metrics likely include aspects such as subject consistency, background consistency, motion smoothness, dynamic range, and overall aesthetic and image quality. By comparing the results obtained with W2SD to those obtained with the standard AnimateDiff model, the table demonstrates the efficacy of W2SD in enhancing the quality of generated videos.

read the caption

Table 12: Quantitative results of W2SD on the video generation task. Model: AnimateDiff. Datasets: VBench.