On Exact Inversion of DPM-Solvers

Seongmin Hong¹, Kyeonghyun Lee¹, Suh Yoon Jeon¹, Hyewon Bae¹, Se Young Chun^1,2

¹Dept. of Electrical and Computer Engineering,
²INMC, Interdisciplinary Program in AI
Seoul National University, Republic of Korea

Accepted to CVPR 2024

[arXiv] [github] [bibTeX]

We propose the exact inversion methods to find the initial noise of the images generated by various diffusion probabilistic models (DPMs).

	Standard sampling methods	Inversion of high-order DPM-solvers	Inversion with classifier-free guidance > 1
Wallace et al.	✗	✗	✔
Zhang et al.	✗	✗	✔
Pan et al.	✔	✗	✗
Ours	✔	✔	✔

We can perform exact inversion w/o model modification regardless of whether the images were generated using high-order DPM-solvers or large classifier-free guidance.

We use the backward Euler method for exact inversion. For inversion of high-order DPM-solvers, we approximate high-order terms.

Our Algorithms (rows 2, 5) significantly reduce reconstruction errors, whether it’s for images or noise, DDIM or high-order DPM-solvers, or pixel-space DPM or Stable Diffusion.

Our exact inversion improves the watermark detection of Wen et al.,

improves the background-preserving image editing of Patashnik et al.

Abstract

Diffusion probabilistic models (DPMs) are a key component in modern generative models. DPM-solvers have achieved reduced latency and enhanced quality significantly, but have posed challenges to find the exact inverse (i.e., finding the initial noise from the given image). Here we investigate the exact inversions for DPM-solvers and propose algorithms to perform them when samples are generated by the first-order as well as higher-order DPM-solvers. For each explicit denoising step in DPM-solvers, we formulated the inversions using implicit methods such as gradient descent or forward step method to ensure the robustness to large classifier-free guidance unlike the prior approach using fixed-point iteration. Experimental results demonstrated that our proposed exact inversion methods significantly reduced the error of both image and noise reconstructions, greatly enhanced the ability to distinguish invisible watermarks and well prevented unintended background changes consistently during image editing.

References

• Bram Wallace, Akash Gokul, and Nikhil Naik. EDICT: Exact Diffusion Inversion via Coupled Transformations. In CVPR, pages 22532–22541, 2023.
• Guoqiang Zhang, Jonathan P Lewis, and W Bastiaan Kleijn. Exact diffusion inversion via bi-directional integration approximation. arXiv:2307.10829, 2023.
• Zhihong Pan, Riccardo Gherardi, Xiufeng Xie, and Stephen Huang. Effective real image editing with accelerated iterative diffusion inversion. In ICCV, pages 15912–15921, 2023.
• Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-Ring Watermarks: Fingerprints for diffusion images that are invisible and robust. arXiv:2305.20030, 2023.
• Or Patashnik, Daniel Garibi, Idan Azuri, Hadar Averbuch-Elor, and Daniel Cohen-Or. Localizing object-level shape variations with text-to-image diffusion models. In ICCV, 2023.

BibTeX

 
@misc{hong2023exact,
      title={On Exact Inversion of DPM-Solvers}, 
      author={Seongmin Hong and Kyeonghyun Lee and Suh Yoon Jeon and Hyewon Bae and Se Young Chun},
      year={2023},
      eprint={2311.18387},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}