Edit model card

Restore missing RGB channels

Restore a missing channel of a RGB image by using ControlNet to guide image generation of Stable Diffusion to infer missing channel from the other two channels.

Training

accelerate launch train_controlnet.py \
  --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" \
  --train_batch_size=4 \
  --gradient_accumulation_steps=8 \
  --proportion_empty_prompts=0.5
  --mixed_precision="fp16" \
  --learning_rate=1e-5 \
  --enable_xformers_memory_efficient_attention \
  --use_8bit_adam \
  --set_grads_to_none \
  --seed=0 \
  --num_train_epochs=2

Image dataset

  • laion2B-en aesthetics>=6.5 dataset
  • --min_image_size 512 --max_aspect_ratio 2 --resize_mode="center_crop" --image_size 512
  • Cleaned with fastdup default settings
  • Data augmented with right-left flipped images
  • Resulting in 214244 images
  • Set whole channel to 0 by alternating between R-G-B channels
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference API
Unable to determine this model's library. Check the docs .