Stable Diffusion 3.5 Large Controlnets released to Stylize image

stable diffusion 3.5 controlnet installation

Controlnet models for Stable Diffusion 3.5 Large has been released by StabilityAI. These models open up new ways to guide your image creations with precision and styling your art. They are out with Blur, canny and Depth trained on synthetic data and filtered data publicly availabe.

The Stability Community License says its free for Non-Commercial Usage for personal or research purposes and for Small Businesses (less that $1M Revenue annuallly). For businesses with revenue over $1M, you will need to contact Stability AI for a custom enterprise license.

Table of Contents

Overview

Basically, they have released three controlnet models listed below:

Controlnet Canny illustration

(a) Canny model guides the structure of your image using a Canny edge map. Its best fro illustrations, structural designs, and detailed sketches. This helps in maintaining the structural integrity in generated image.

Controlnet Depth illustration

(b) Depth ControlNet uses depth maps (generated by DepthFM) to inform the image generation. Its suitable for architectural renderings, texturing 3D assets, or adding depth realism. It creates images with a more realistic sense of space and layering.

Controlnet Blur illustration

(c) Blur ControlNet performs ultra-high fidelity upscaling by working with tiled input images. Its helpful for Tiling images for upscaling or merging tiles for high-resolution results. It can smoothly handles image details while boosting resolution.

Lets go through the key features, use cases, and how to do the setup process.

Installation

1. Get ComfyUI installed if not yet. Also do the basic installation for Stable Diffusion3.5 to get the overall overview.

2. To work with the controlnets, you must have installed the Stable Diffusion 3.5 Large (FP8 scaled) model. After downloading, put the model into your "ComfyUI/models/checkpoints" folder.

3. Make sure you are logged in to your Hugging face account before downloading.

Now, to download the controlnet models first and foremost is you need to accept the StabilityAI's license and agreement.

4. Download Canny model (sd3.5_large_controlnet_canny.safetensors), Depth model (sd3.5_large_controlnet_depth.safetensors) and Blur model (sd3.5_large_controlnet_blur.safetensors) from StabilityAI's Hugging face repository.

After downloading, just save the models into the "ComfyUI/models/controlnet" directory and start using them.