Diffusion Image Analogies

Adéla Šubrtová, Michal Lukáč, Jan Čech, David Futschik, Eli Shechtman, Daniel Sýkora,

This is the official repository for the Diffusion Image Analogies paper published at the SIGGRAPH 2023 Conference Proceedings.

---

Installation

1. Clone the repo

   git clone --recurse-submodules https://github.com/subrtadel/DIA.git
   cd ./DIA

2. Create environment

   conda create -n dia_env
   conda activate dia_env
   conda install python=3.8.5 pip=20.3 cudatoolkit=11.3 pytorch=1.11.0 torchvision=0.12.0 numpy=1.19.2 -c pytorch -c nvidia -c conda-forge -c defaults
   

3. Install packages

   pip install -r requirements.txt
   cd ./stable-diffusion/
   pip install -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers
   pip install -e .

4. Download the sd-v1-4.ckpt model and put it into correct folder

   mkdir -p ./models/ldm/stable-diffusion-v1/
   
   

5. Install Image Magick.

(back to top)

---

Usage

1. Upload images into ./dataset/raw_data/ folder.

2. Run process_new_data.py. The images are assigned file_ids in a %05d format.

3. Define the triplets in a .csv file. Refer to the images by their file_id. Example file is triplets.csv. First column specifies the A input, second the A' and the third B input. Either with of without filename suffixes is fine.

4. Run the precompute_noises_and_conditionings.py script. This may take a while.

   python precompute_noises_and_conditionings.py \
       --config ./config/parameter_estimation.yaml \
       --inversion_subfolder noise \
       --token_subfolder tokens \ 
       --triplet_file triplets.csv \
       --data_path ./dataset/data/

5. Check the ./config/analogy_params.yaml.

6. Run the do_analogies.py script.

   python do_analogies.py \
       --config ./config/parameter_estimation.yaml \
       --inversion_subfolder noise \
       --token_subfolder tokens \ 
       --output_subfolder analogies \
       --triplet_file triplets.csv \
       --data_path ./dataset/data/

---

BibTeX

@inproceedings{Subrtova2023DIA,
    title = {Diffusion Image Analogies},
    author = {A. \v{S}ubrtov\'{a} and M. Luk\'{a}\v{c} and J. \v{C}ech and D. Futschik and E. Shechtman  and D. S\'{y}kora},
    booktitle = {ACM SIGGRAPH 2023 Conference Proceedings},
    year = {2023}
  }