Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminator

🔼 This figure illustrates the Direct Discriminative Optimization (DDO) framework. It shows three key components: (1) Two models are involved: a pretrained reference model (θref) that remains frozen during training, and a target model (θ) that is initially the same as the reference model but will be updated. (2) Data sources consist of real data samples (from pdata) and synthetic samples generated by the reference model (from pθref). (3) The training objective is a GAN loss where the discriminator is implicitly defined by the likelihood ratio of the target and reference models. The goal is to train the target model (θ) to better match the true data distribution (pdata) by discriminating between real and fake samples.

read the caption

Figure 2: Illustration of DDO. (1) Models. θrefsubscript𝜃ref\theta_{\text{ref}}italic_θ start_POSTSUBSCRIPT ref end_POSTSUBSCRIPT is the (pretrained) reference model frozen during training. θ𝜃\thetaitalic_θ is the learnable model initialized as θrefsubscript𝜃ref\theta_{\text{ref}}italic_θ start_POSTSUBSCRIPT ref end_POSTSUBSCRIPT. (2) Data. Samples from pdatasubscript𝑝datap_{\text{data}}italic_p start_POSTSUBSCRIPT data end_POSTSUBSCRIPT are drawn from the training dataset. Samples from pθrefsubscript𝑝subscript𝜃refp_{\theta_{\text{ref}}}italic_p start_POSTSUBSCRIPT italic_θ start_POSTSUBSCRIPT ref end_POSTSUBSCRIPT end_POSTSUBSCRIPT are generated by the reference model, either offline or online. (3) Objective. The target model θ𝜃\thetaitalic_θ is optimized by applying the GAN discriminator loss with the implicitly parameterized discriminator dθsubscript𝑑𝜃d_{\theta}italic_d start_POSTSUBSCRIPT italic_θ end_POSTSUBSCRIPT to distinguish between real samples from pdatasubscript𝑝datap_{\text{data}}italic_p start_POSTSUBSCRIPT data end_POSTSUBSCRIPT and fake samples from pθrefsubscript𝑝subscript𝜃refp_{\theta_{\text{ref}}}italic_p start_POSTSUBSCRIPT italic_θ start_POSTSUBSCRIPT ref end_POSTSUBSCRIPT end_POSTSUBSCRIPT.

🔼 This figure compares Direct Discriminative Optimization (DDO) with Direct Preference Optimization (DPO). It illustrates the key differences in their methodologies. DPO uses paired human preference data to align a language model with human preferences by expressing the reward model as a likelihood ratio. DDO, in contrast, uses unpaired data from the data distribution and the pretrained model to improve the target model by implicitly parameterizing a discriminator using the likelihood ratio. This allows for direct and efficient finetuning of the pretrained model without the need for additional networks or alternating optimization processes. The figure visually represents these differences and highlights how DDO achieves model alignment using an implicitly defined discriminator.

read the caption

Figure 3: Comparison with DPO.

🔼 Figure 4 compares the number of parameters and the inference time required for different image generation methods. The methods compared include several guidance techniques (Classifier-Free Guidance, Discriminator Guidance, and Autoguidance) as well as the proposed Direct Discriminative Optimization (DDO) method. The baseline model is also shown. The evaluation is performed on two different datasets: CIFAR-10 and ImageNet-64, with Discriminator Guidance evaluated on class-conditional CIFAR-10 and Autoguidance on ImageNet-64. This allows for a direct comparison of the computational efficiency and model complexity associated with various approaches to improving image generation quality.

read the caption

Figure 4: Comparison of model parameter counts and inference time across different guidance methods and DDO. For DG, we measure the statistics on class-conditional CIFAR-10. For AG, we measure the statistics on ImageNet-64.

🔼 This figure shows the results of applying Direct Discriminative Optimization (DDO) to diffusion models. Specifically, it illustrates (a) the impact of iterative refinement (self-play) on model performance across multiple rounds, and training curves under different hyperparameter settings (b) α and (c) β. The plots in (b) and (c) show how the FID (Fréchet Inception Distance) score changes with the number of iterations during training for different values of α and β, demonstrating the effect of these hyperparameters on model performance.

read the caption

Figure 5: Illustrations on diffusion models. (a) Multi-round refinement and (b)(c) training curves under different α,β𝛼𝛽\alpha,\betaitalic_α , italic_β.

🔼 Figure 6 presents an analysis of autoregressive models, specifically focusing on the impact of hyperparameters on model performance. Subfigures (a) and (b) show FID (Fréchet Inception Distance) and IS (Inception Score) trade-off curves for different autoregressive models (VAR-d16 and VAR-d30). These curves illustrate the balance between image diversity (IS) and image quality (FID) at varying guidance scales. Subfigure (c) explores the effect of hyperparameter α on model performance, while keeping β constant at 0.02. This analysis reveals how adjustments to α influence the FID and IS, offering insights into optimizing the generation process.

read the caption

Figure 6: Illustrations on autoregressive models. (a)(b) FID-IS trade-off curves and (c) the impact of α𝛼\alphaitalic_α under β=0.02𝛽0.02\beta=0.02italic_β = 0.02.

🔼 This figure displays several randomly generated images from the Enhanced Diffusion Model (EDM) trained on the CIFAR-10 dataset. The generation process is unconditional, meaning that no specific class label or other guidance was provided to the model during generation. The FID score (Fréchet Inception Distance) of 1.97 indicates the model’s performance, with lower scores suggesting better image quality and similarity to real images from the CIFAR-10 dataset.

read the caption

Figure 7: Random samples of EDM (CIFAR-10, Unconditional), FID 1.97.

🔼 This figure displays 100 randomly generated images from the EDM (Energy-based Diffusion Model) model after being fine-tuned using the Direct Discriminative Optimization (DDO) method. The model was trained on the CIFAR-10 dataset in an unconditional setting (meaning no class labels were provided during training, allowing the model to generate images from all classes). The FID (Fréchet Inception Distance) score of 1.38 indicates a high level of image quality and realism, suggesting that the DDO fine-tuning significantly improved the model’s generation capabilities compared to the original EDM model.

read the caption

Figure 8: Random samples of EDM + DDO (CIFAR-10, Unconditional), FID 1.38.

🔼 This figure displays random image samples generated by the Enhanced Diffusion Model (EDM) when trained on the CIFAR-10 dataset in a class-conditional setting. The Fréchet Inception Distance (FID) score achieved by this model is 1.85, indicating its performance in generating realistic and diverse images. Each image is an example of the model’s output for a given class label.

read the caption

Figure 9: Random samples of EDM (CIFAR-10, Class-conditional), FID 1.85.

🔼 This figure displays several randomly generated images from a class-conditional generative model trained on the CIFAR-10 dataset. The model is a diffusion model enhanced with Direct Discriminative Optimization (DDO), which aims to improve its performance beyond the limitations of maximum likelihood estimation. The FID (Fréchet Inception Distance) score for these samples is 1.30, indicating high visual quality and diversity.

read the caption

Figure 10: Random samples of EDM + DDO (CIFAR-10, Class-conditional), FID 1.30.

🔼 This table presents the Fréchet Inception Distance (FID) scores achieved by various generative models on the class-conditional ImageNet-64 dataset. The models compared include several state-of-the-art GANs and diffusion models, both with and without additional techniques like classifier-free guidance (CFG) or discriminator guidance (DG). Some models also incorporate diffusion distillation with an auxiliary GAN loss. A key observation noted is the importance of strict class balance for accurate FID calculation on ImageNet-64, with a modified sampling process employed for improved results.

read the caption

Table 2: Results on class-conditional ImageNet-64. †Including diffusion distillation methods with auxiliary GAN loss. ‡We find strict class balance crucial for FID on ImageNet and slightly modify the original sampling script to ensure this.

🔼 This table presents the results of experiments conducted on the ImageNet 256x256 dataset using class-conditional models. It compares the performance of various generative models, specifically focusing on the Fréchet Inception Distance (FID) metric. The models are categorized into GANs, Diffusion models, and Autoregressive models. The table shows FID scores achieved with and without the use of classifier-free guidance (CFG) and other guidance methods, denoted as ‘G’. The number of function evaluations (NFE) required during inference is also reported; note that the NFE is doubled when guidance techniques are used.

read the caption

Table 3: Results on class-conditional ImageNet 256×\times× 256. “G” denotes guidance, including both classifier guidance and CFG. We report the NFE without guidance, which is doubled with guidance.