Diffusion Templates
Diffusion Templates is a controllable generation plugin framework for Diffusion models in DiffSynth-Studio, providing additional controllable generation capabilities for base models.
Open-source code: DiffSynth-Studio
Technical report: arXiv
Project page: GitHub
Documentation reference
Introduction to Diffusion Templates: English Version、中文版
Detailed Architecture of Diffusion Templates: English Version、中文版
Template Model Inference: English Version、中文版
Template Model Training: English Version、中文版
Online demo: ModelScope
Model collection: ModelScope、ModelScope International、HuggingFace
| Model Name | ModelScope | ModelScope International | HuggingFace | Inference Code | Low VRAM Inference Code | Training Code | Training Validation Code |
|---|---|---|---|---|---|---|---|
| Structure Control | link | link | link | code | code | code | code |
| Brightness Adjustment | link | link | link | code | code | code | code |
| Color Adjustment | link | link | link | code | code | code | code |
| Image Editing | link | link | link | code | code | code | code |
| Super-Resolution | link | link | link | code | code | code | code |
| Sharpness Enhancement | link | link | link | code | code | code | code |
| Aesthetic Alignment | link | link | link | code | code | code | code |
| Local Redrawing | link | link | link | code | code | code | code |
| Content Reference | link | link | link | code | code | code | code |
| Age Control | link | link | link | code | code | code | code |
| Panda Meme (Easter Egg Model) | link | link | link | code | code | code | code |
| Dataset Name | ModelScope | ModelScope International | HuggingFace |
|---|---|---|---|
| Text-to-Image | link | link | link |
| Local Redrawing | link | link | link |
| Background Replacement | link | link | link |
| Clothing Replacement | link | link | link |
| Pose Adjustment | link | link | link |
| Foreground Modification | link | link | link |
| Local Addition/Removal | link | link | link |
| Super-Resolution | link | link | link |
| Portrait Generation | link | link | link |
| Random Cropping | link | link | link |
| Lighting Adjustment | link | link | link |
| Scene Structure | link | link | link |
| Facial Expression Editing | link | link | link |
| View Angle Adjustment | link | link | link |
| Style Transfer | link | link | link |
| Multi-Resolution | link | link | link |
| Multi-Image Merge | link | link | link |
Model Performance Overview
Super-Resolution + Sharpness Enhancement: Generate ultra-high-resolution images
| Low Resolution Input | High Resolution Output |
|---|---|
 |
 |
Structure Control + Aesthetic Alignment + Sharpness Enhancement: Fully-equipped ControlNet
| Structure Control Image | Output Image |
|---|---|
 |
 |
Structure Control + Image Editing + Color Adjustment: Artistic Style Creation at Will
| Structure Control Image | Editing Input Image | Output Image |
|---|---|---|
|
 |
 |
Brightness Control + Image Editing + Local Redrawing: Cross-dimensional Elements in Images
| Reference Image | Redrawing Area | Output Image |
|---|---|---|
|
 |
 |