On Exact Inversion of DPM-Solvers

Seongmin Hong1, Kyeonghyun Lee1, Suh Yoon Jeon1, Hyewon Bae1, Se Young Chun1,2

1Dept. of Electrical and Computer Engineering,
2INMC, Interdisciplinary Program in AI
Seoul National University, Republic of Korea

CVPR 2024

[Paper] [supp] [arXiv] [github] [bibTeX]






We propose the exact inversion methods to find the initial noise of the images generated by various diffusion probabilistic models (DPMs).






  Standard
sampling
methods
Inversion of
high-order
DPM-solvers
Inversion with
classifier-free
guidance > 1
Wallace et al.
Zhang et al.
Pan et al.
Ours

We can perform exact inversion w/o model modification regardless of whether the images were generated using high-order DPM-solvers or large classifier-free guidance.






We use the backward Euler method for exact inversion. For inversion of high-order DPM-solvers, we approximate high-order terms.






Our Algorithms (rows 2, 5) significantly reduce reconstruction errors, whether it’s for images or noise, DDIM or high-order DPM-solvers, or pixel-space DPM or Stable Diffusion.






Our exact inversion improves the watermark detection of Wen et al.,






improves the background-preserving image editing of Patashnik et al.





Abstract

Diffusion probabilistic models (DPMs) are a key component in modern generative models. DPM-solvers have achieved reduced latency and enhanced quality significantly, but have posed challenges to find the exact inverse (i.e., finding the initial noise from the given image). Here we investigate the exact inversions for DPM-solvers and propose algorithms to perform them when samples are generated by the first-order as well as higher-order DPM-solvers. For each explicit denoising step in DPM-solvers, we formulated the inversions using implicit methods such as gradient descent or forward step method to ensure the robustness to large classifier-free guidance unlike the prior approach using fixed-point iteration. Experimental results demonstrated that our proposed exact inversion methods significantly reduced the error of both image and noise reconstructions, greatly enhanced the ability to distinguish invisible watermarks and well prevented unintended background changes consistently during image editing.

References



BibTeX

 
@InProceedings{Hong_2024_CVPR,
    author    = {Hong, Seongmin and Lee, Kyeonghyun and Jeon, Suh Yoon and Bae, Hyewon and Chun, Se Young},
    title     = {On Exact Inversion of DPM-Solvers},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
    pages     = {7069-7078}
}