How to Install Control Net in Stable Diffusion 1.5 ?

 complete guide to control net

So, here the waiting is over because a new model released officially by Stability Ai which is a changing moment for the image generation community.

We have experienced and used a couple of months, including the released Control Net which gives an extra edge over others and helps much for AI lovers in the field of image generation.

Features:

  • Supported for Stable Diffusion 1.5
  • For Experimental and Production ready
  • Supports Automatic1111 and ComfyUI
  • Easy plug and play


Install ControlNet in Automatic1111:

1. First of all you need to download and install the Controlnet models from Hugging Face repository

Download ControlNet from hugging face

2.  This helps us to work on every aspect of the control net and we can enjoy the full advantage of the Controlnet. But this is optional and depends on your requirements.


controlnet files download

3. Here, the files with .pth extensions are the models and the similar name files with .yaml extensions are the respective model's configurations. 

https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main


 If you head over the control net models of Hugging Face, you will see files that end with the letter "e" which means these are used for experimental purposes and the other control net models with the letter "p" are for production-ready which can be deployed on the server for standalone applications and files with "sd15" denote that these are the Stable Diffusion 1.5 version base model.


save controlnet files

 4. After downloading these, you can move the downloaded files into the respective path folder for working.

5. Move to your stable diffusion root folder.

stable-diffusion-webui\webui\extensions\sd-webui-controlnet\models

6. Save your files inside the "models" folder.



restart automatic1111 to load controlnet

7. Now go back to Automatic1111 and click on the "Refresh" button to take effect and let all the models load on the Automatic1111 control net section.

Controlnet installed

8. Enjoy the power of ControlNet in image generation. 


Install ControlNet in ComfyUI:


ComfyUI ControlNet Manager tab

1. Open ComfyUI and move to ComfyUI Manager .


ComfyUI ControlNet  Install nodes

3. A new small pop-up with multiple options will appear. Just click on Install Custom Nodes.


ComfyUI ControlNet Install preprocessors

4. Again a new window will appear. Now on the rightmost side of the window, you will a search box. Just search for “CONTROL NET” and click the Search button which will be available in the top right corner. You will see various preprocessors related to Control Net.

5. Now click the Install button presented over to the right of the dashboard to install all the Control net-related preprocessors.

6. Now,  done. Just close by click the Close button and Restart the Comyfy UI because it doesn’t show the changes automatically.


ComfyUI ControlNet Installed preprocessors access


7. Now open it again and right click on the dashboard of the Comfy UI and move to the following option:    

Add Node > Control Net PreProcessors >

 So Inside this option, you will get access to all the preprocessors that we have installed recently.


clone hugging face controlnet lora


8. Now, move to the Hugging Face portal and install the Stable diffusion XL version model released by Stability AI. Here, select the link which is for Control – Net LORA (But actually this is not the LORA that you know in the field of stable diffusion). 

These are small and are more memory efficient.  You have to clone the repository.


comfyui installation folder


9. Place the downloaded file into the ComfyUI/models/ControlNet folder.


after putting all folder


10. Here is the result after putting all the downloaded files into the ComfyUI installation folder.


Comfy UI controlnet feature

Now let's see what these Control Net Preprocessors can do.

If you are loading the recently installed preprocessors for the first time, this will take some time in the background to download some required files which need to run the preprocessors.


Comfy UI controlnet feature

You can use some features to add some depth or sharpness to the images as well.


Conclusion: 

Control net is the most effective model that we can use as an extension on ComfyUI for making the image more depth, open pose, face swapping, consistent image, etc. It works on the concept of neural network mechanisms for pre-trained diffusion models.