Zero to Hero ControlNet Tutorial Stable Diffusion Web UI Extension Complete Feature Guide

Zero to Hero ControlNet Tutorial: Stable Diffusion Web UI Extension | Complete Feature Guide

Full tutorial link > https://www.youtube.com/watch?v=3E5fhFQUVLo

I show how to install Automatic1111 Web UI & ControlNet extension installation from scratch in this video. Moreover I show how to make amazing QR codes and inpainting and out painting of ControlNet which are very similar to Photoshop generative fill and Midjourney zoom out. Furthermore, I explain and show what are Canny, Depth, Normal, OpenPose, MLSD, Lineart, SoftEdge, Scribble, Seg, Shuffle, Tile, Inpaint, IP2P, Reference, T2IA features of ControlNet.

Source GitHub Readme File ⤵️

https://github.com/FurkanGozukara/Stable-Diffusion/blob/main/Tutorials/ControlNet-Tutorial-How-To-Install-Extension-Become-A-Master.md

Our Discord server ⤵️

https://bit.ly/SECoursesDiscord

Auto Install Scripts (windows) ⤵️

https://www.patreon.com/posts/automatic-and-84875387

Auto Install Scripts (runpod) ⤵️

https://www.patreon.com/posts/84896373

If I have been of assistance to you and you would like to show your support for my work, please consider becoming a patron on 🥰 ⤵️

https://www.patreon.com/SECourses

00:00:00 Introduction to most advanced zero to hero ControlNet tutorial

00:02:55 How to install Stable Diffusion Automatic1111 Web UI from scratch

00:05:05 How to see extensions of files like .bat

00:06:15 Where to find command line arguments of Automatic1111 and what are they

00:06:46 How to run Stable Diffusion and ControlNet on a weak GPU

00:07:37 Where to put downloaded Stable Diffusion model files

00:08:29 How to give different folder path as the model path - you can store models on another drive

00:09:15 How to start using Stable Diffusion via Automatic1111 Web UI

00:10:00 Command line interface freezing behaviour

00:10:13 How to improve image generation of Stable Diffusion with better VAE file

00:11:39 Default VAE vs best VAE comparison

00:11:50 How to set quick shortcuts for VAE and Clip Skip for Automatic1111 Web UI

00:12:30 How to upgrade xFormers to the latest version in Automatic1111

00:13:40 What is xFormers and other optimizers

00:14:26 How to install ControlNet extension of Automatic1111 Web UI

00:18:00 How to download ControlNet models

00:19:40 How to use custom Stable Diffusion models with Automatic1111 Web UI

00:21:24 How to update ControlNet extension to the latest version

00:22:53 Set this true, allow other scripts to control ControlNet extension

00:24:37 How to make amazing QR code images with ControlNet

00:30:59 Best settings for QR code image generation

00:31:44 What is Depth ControlNet option and how to use it

00:33:28 Depth_leres++ of ControlNet

00:34:15 Depth_zoe of ControlNet

00:34:22 Official information of Depth maps

00:34:49 ControlNet Normal map

00:35:34 Normal Midas map

00:36:05 Official information of Normal maps

00:34:49 ControlNet Canny model

00:37:42 Official information of Canny

00:37:55 ControlNet MLSD straight lines model

00:39:08 Official information of MLSD straight lines

00:39:18 ControlNet Scribble model

00:40:28 How to use your own scribble images and turn them into amazing artworks

00:40:45 When to select none in pre-processor section

00:41:20 My prompt is more important

00:41:36 ControlNet is more important

00:42:01 Official information of Scribble

00:42:11 ControlNet Softedge model

00:43:12 Official information of SoftEdge

00:43:22 ControlNet Segmentation (Seg) model

00:43:55 How to modify your prompt to properly utilize segmentation

00:44:10 Association of prompt with segments the ControlNet finds

00:44:41 How to turn your wall into a painting with ControlNet

00:45:33 Why I selected none preprocessor

00:43:06 Official information of segmentation (Seg)

00:46:16 Open pose module of ControlNet

00:46:40 How to install and use OpenPose editor

00:50:58 Official information of OpenPose

00:51:08 ControlNet Lineart model

00:51:36 Preprocessor preview bug

00:54:21 Real lineart into amazing art example

00:56:34 How to generate amazing logo images by using Lineart of ControlNet

00:58:16 Difference between just resize, crop and resize, and resize and fill

00:59:02 ControlNet Shuffle model

01:00:50 Official information of Shuffle

01:02:36 What is multi-ControlNet and how to use it

01:04:05 Instruct pix2pix of ControlNet

01:06:00 Inpainting feature of ControlNet

01:07:49 ControlNet inpainting vs Automatic1111 inpainting

01:07:59 How to get true power of inpainting of ControlNet (hint: with tiling)

01:09:00 How to upscale and add details to the images with inpainting + tiling

01:09:30 The tile color fix + sharp to obtain even better results

01:10:35 Tile color fix + sharp vs old tile resample result comparison

01:11:20 How to use generative fill feature of Photoshop in ControlNet to remove objects

01:12:58 How to outpaint (zoom out feature of midjourney 5.2) image with ControlNet

01:14:17 The logic of outpainting

01:14:40 How to continue outpainting easily

01:16:06 Tiling of ControlNet - ultimate game changer for upscaling

01:17:19 How to turn your image into a fully stylized image with tiling without training

01:20:57 Reference only feature of ControlNet

01:22:29 Official information of Reference mode

01:22:39 Style Transfer (T2IA) of ControlNet

01:26:54 How to install and use ControlNet on RunPod

Video Transcription

• 00:00:00 Greetings everyone. In this video, I will show  you how to install Stable Diffusion Automatic  

• 00:00:05 1111 web UI from scratch. How to install the  most amazing and most advanced extension of  

• 00:00:11 Stable Diffusion community, ControlNet from  scratch. How to generate amazing QR codes by  

• 00:00:17 using ControlNet with a very easy workflow with  using only single ControlNet. What is the Depth  

• 00:00:24 feature of ControlNet and how to use it to  generate images like this. What is Normal  

• 00:00:29 map of ControlNet and how to utilize Normal maps  to generate images like these. What is Lineart of  

• 00:00:35 ControlNet and how to generate amazing pictures  from Lineart images like this. What is Canny of  

• 00:00:41 ControlNet and how to utilize Canny of ControlNet  properly. From this image to this image. What is  

• 00:00:47 Scribble of ControlNet and how to use Scribble  of ControlNet. What is SoftEdge of ControlNet and  

• 00:00:48 how to use SoftEdge of  ControlNet. What is segmentation  

• 00:00:56 of ControlNet and how utilize segmentation of  ControlNet. What is OpenPose of ControlNet and  

• 00:00:57 how to utilize OpenPose of ControlNet. What  is Shuffling of ControlNet and how to utilize  

• 00:01:07 Shuffling with Canny to generate amazing pictures  like this. What is instruct pix2pix feature of  

• 00:01:12 ControlNet and how can you utilize instruct  pix2pix of ControlNet. How to combine tiling  

• 00:01:19 and inpainting of ControlNet to improve images  resolution and quality significantly like this.  

• 00:01:26 How to use generative fill feature of ControlNet  like in Photoshop for free. How to use outpainting  

• 00:01:33 of ControlNet similar to the Midjourney or  generative fill of Photoshop as you are seeing  

• 00:01:38 right now. How to continue outpainting very  easily. I will show you that. What is tiling of  

• 00:01:44 ControlNet and how you can use it to turn yourself  into an avatar image like this very easily. What  

• 00:01:51 is Reference of ControlNet and how you can use  Reference to generate images. What is T2IA of  

• 00:01:57 ControlNet and how you can use style transfer  of ControlNet to generate amazing stylized  

• 00:02:03 images. How to use ControlNet on RunPod and how  to download ControlNet models on RunPod. How to  

• 00:02:11 update ControlNet on RunPod. I have prepared an  amazing GitHub readme file for this tutorial. All  

• 00:02:18 of the links and the commands that we are going  to use are posted on this GitHub readme file. This  

• 00:02:24 readme file will get updated if it be necessary.  So this is your number one source for this  

• 00:02:31 tutorial. The link for this file will be in the  description of the video and also in the pinned  

• 00:02:36 comments of the video. As some of you may already  know, I have Stable Diffusion GitHub repository.  

• 00:02:42 I am posting all of the useful stuff here. My  previous tutorials, the readme files of the  

• 00:02:48 other tutorials, and many other things. So if you  star it, fork it and watch it. I would appreciate  

• 00:02:53 that very much. So, we will begin with cloning  and installing the Automatic1111 web UI. For this  

• 00:03:00 tutorial, you need to have Python to be installed  along with Git. If you don't know how to install  

• 00:03:06 python. I have an excellent tutorial for that.  Just click this. It will open you this tutorial.  

• 00:03:12 Watch it and you will learn how to use python how  to install python at the level that is required  

• 00:03:18 to use Stable Diffusion. Installing Automatic1111  web UI is so simple. However, for your easiness,  

• 00:03:24 I prepared also automatic installer: open this  link. You will find some information and attached  

• 00:03:32 scripts here. Download control_net_install.bat  file and control_net_downloader.py file and  

• 00:03:39 auto_install_SD_web_ui.bat file. If  you are not my Patreon supporter,  

• 00:03:43 don't worry because everything that these scripts  doing are also documented here and I will show  

• 00:03:50 you them. So this is optional. This will make your  installation easier. I will make two installation  

• 00:03:57 in my E volume. So let's make a folder name it  as auto and also another folder as manual. So  

• 00:04:05 in auto we will do automatic installation and in  manual we will do manual installation. You will  

• 00:04:09 see both of them. So let's begin with automatic  installation. I copy the files downloaded, paste  

• 00:04:16 them into my auto folder and I just double click  the auto_install_SD_web_ui.bat file. It will ask  

• 00:04:23 you. Run anyway. You can right click this script,  edit, and see inside of it. It is so simple and  

• 00:04:30 let's also begin our manual installation. So  for manual installation we will begin with git  

• 00:04:36 clone. Copy this command, open a new cmd window  inside your manual folder, or wherever you want  

• 00:04:43 to install. Right click and paste the command and  hit enter. It will clone the repository into that  

• 00:04:49 folder. Meanwhile, automatic installation is  running. Then we will add xFormers. Because I  

• 00:04:55 prefer using xFormers. It is really good when  you are working with higher resolution images  

• 00:05:00 and when you are doing training. So enter inside  the cloned folder. Find webui-user.bat file. If  

• 00:05:09 you are not seeing the extensions of files like  me, go to view tab here and here check the file  

• 00:05:15 name extensions. It will show you the file  extensions. Right click, click edit. Add this  

• 00:05:20 as a command line argument here, save and then  just double click it. It will make a new virtual  

• 00:05:27 environment and start installing Automatic1111  web UI for Stable Diffusion. Automatic 1111 web  

• 00:05:33 UI is the best interface best ui for using Stable  Diffusion. It has over 85,000 stars on GitHub and  

• 00:05:44 this is magnificent. This is amazing. This is the  best UI that I am using for Stable Diffusion. I  

• 00:05:50 will show you what it is if you don't know it,  but I am pretty sure most of my viewers already  

• 00:05:55 knowing this. You will see it is installing every  libraries like this when it is running on the cmd  

• 00:06:01 window. Cmd window is our number one source  of information to see what is happening when  

• 00:06:07 using Stable Diffusion or when using ControlNet.  Also, if you want to learn more about set command  

• 00:06:13 line arguments like this, there is a link of wiki  page for Automatic 1111 web UI. Click this link,  

• 00:06:19 you will see all of the command line arguments  here. You can read this page and see what kind  

• 00:06:26 of arguments you can give. For example, you can  give the folder of models so that you can use  

• 00:06:31 single folder for multiple Automatic1111 web UI  installations. You can set a lot of folders here.  

• 00:06:38 There are also a lot of other options here. You  can look all of them and see which one of them  

• 00:06:43 are useful for you. For example, if you have a GPU  that is not strong, you can add --lowvram. It will  

• 00:06:52 optimize your Automatic1111 web UI to use lesser  vram. This will help you significantly if you have  

• 00:06:59 low vram having gpu. If you get out of vram memory  error, you can use lowvram or medvram. First,  

• 00:07:05 try medvram. If medvram fails, then use lowvram.  Moreover, if you have a low ram in your computer,  

• 00:07:12 you can use lowram option as well. So as I said,  check all of the options here. Since we did a  

• 00:07:18 fresh installation, it will download the default  1.5 pruned Stable Diffusion model file. However,  

• 00:07:26 if you have model file already, you can close the  screen at this point and copy paste the model. Let  

• 00:07:33 me show you. By the way, this is the automatic  installation. So I closed the window. I have a  

• 00:07:38 lot of models inside my F drive 0 models. From  here let's copy the 1.5 pruned.ckpt. This is the  

• 00:07:45 base model. You can also use other models. So I  copied it. Go to the auto, go to inside Stable  

• 00:07:52 Diffusion web UI folder go to inside models, go to  inside Stable Diffusion and paste it here. You see  

• 00:07:59 this is the previous file that it was downloading.  However, since we have closed the window, it was  

• 00:08:05 not completed so I will delete it and for starting  I just need to double click webui-user file.  

• 00:08:11 However, automatic installation automatically  upgrades xFormers to latest version so I will  

• 00:08:18 delete this command so that it won't reinstall  it again and double click it. Meanwhile, manual  

• 00:08:24 installation is also downloading the model file.  So let's also close it. For manual installation I  

• 00:08:30 will say that use this particular folder, let's  copy the path, go to our manual installation,  

• 00:08:36 open Stable Diffusion web UI, right click and  edit the webui-user file and here I will add the  

• 00:08:45 ckpt command. The command can be found here. ckpt  directory edit and then copy paste the path which  

• 00:08:53 is let's copy it again zero models like this.  Okay, don't forget to put quotation marks, save  

• 00:09:01 it. And let's also start our manual installations.  For starting it and for installing it we use  

• 00:09:07 webui-user.bat file and it will start the manual  installation of Stable Diffusion. Once it is  

• 00:09:14 started you will see a url like this: you need  to open this in your browser. So you can copy it,  

• 00:09:20 paste it into your browser and you see this is  our Stable Diffusion web UI interface. Now you  

• 00:09:27 can use text to image to generate images. You will  see the models are listed here, click refresh and  

• 00:09:34 you will see the new models if you download  them. For example, let's type as a cute cat  

• 00:09:40 and just hit generate. There are so many other  options. Hopefully I will make another tutorial  

• 00:09:46 that I will cover much more information about how  to use Stable Diffusion web UI, Automatic1111 web  

• 00:09:52 UI. In the very bottom of the screen you see the  user python version, Torch version, xFormers,  

• 00:09:59 Gradio version. By the way it was frozen because  when you click the command line interface it may  

• 00:10:06 get frozen. So just click the command line  interface and hit enter to unfreeze it and  

• 00:10:11 we got the result. But this is not good and we can  improve this with better VAE file. So I have also  

• 00:10:19 shown how to download VAE file in here. We will  do that. So open this link and this will download  

• 00:10:26 the best VAE file. The manual installation is also  loaded. Let's also open it. You will notice that  

• 00:10:32 this is running on the one increased port. So this  is the automatic installation. This is the manual  

• 00:10:39 installation and in manual installation I have  so many different models as you are seeing right  

• 00:10:45 now because we did set its model folder to another  folder and it sees all of the models there. Okay,  

• 00:10:53 VAE file is downloaded, copy it, you need to  put VAE file inside, Stable Diffusion, inside  

• 00:10:59 models inside VAE folder, copy paste it there.  Let's also do the same for manual installation  

• 00:11:05 as well and VAE is put. So this was the cat image  we got with the default VAE. So let's also check  

• 00:11:12 it out with the best VAE. So I copied the seed  and I don't change anything else and it should  

• 00:11:17 produce the same image. Okay, let's open the image  in a new tab, then go to the settings in here,  

• 00:11:23 go to the Stable Diffusion in here, refresh it  and you will see default VAE selection. Let's  

• 00:11:30 select it and apply and go to text image tab and  generate again and let's compare the result. Okay,  

• 00:11:37 this is the default way of the model and  this is the improved VAE. Improved VAE  

• 00:11:42 is definitely better and it will definitely  improve your overall image quality. Moreover,  

• 00:11:48 you may have noticed it in some tutorials. There  are some quick shortcuts in the above part of the  

• 00:11:55 screen. Go to the settings, in here click the  user interface tab, in here wou will see info,  

• 00:12:02 quick list settings. Click here and it will  allow you to add quick selections to your web  

• 00:12:07 UI interface. So from here I will select VAE, you  see SD VAE and then you can also select clip skip  

• 00:12:17 which is used in some of the models and then all  you need to do is apply settings and reload ui and  

• 00:12:24 ui is reloaded. Now I see quick VAE selection and  I see quick click skipping. You see. So as a next  

• 00:12:31 step I will upgrade also xFormers of the manual  installation. Let's close both of them. First you  

• 00:12:37 need to activate the virtual environment of Stable  Diffusion because whatever you do is installed  

• 00:12:44 inside the virtual environment in python with  Stable Diffusion. As I said, watch this tutorial  

• 00:12:50 if you don't know about virtual environments  and python. So copy this. Enter inside your  

• 00:12:56 installation folder. It will be inside manual  Stable Diffusion. Open a new cmd here. Now this  

• 00:13:03 cmd windows in this folder. It is very important  where your cmd window is open. Copy, right click,  

• 00:13:10 paste, hit enter. I have given incorrect version  why? Because I wanted to list all of the versions  

• 00:13:17 available to install and currently this is the  latest version. So hit enter and then type like  

• 00:13:23 this pip Install xFormers==0.0.21.dev553 and it  will install the latest xFormers version. This is  

• 00:13:37 how I find the latest version and install it. You  may wonder what is xFormers why we are installing?  

• 00:13:43 xFormers is an optimization library that increases  the speed of the generation significantly. There  

• 00:13:49 are also other optimization techniques. I added  the link here. Open it and you can read the  

• 00:13:55 available Stable Diffusion web UI optimizations  here. You can use other optimization techniques  

• 00:14:01 as well. So all of the optimizations are written  here. These are the command line arguments.  

• 00:14:06 I prefer to use xFormers over opt sdp attention  because it supports I think higher resolution and  

• 00:14:13 also it is better for training with DreamBooth  or LoRA. Okay, we have installed our Stable  

• 00:14:19 Diffusion web UI. We have installed the best VAE.  Now it is time to install the famous ControlNet.  

• 00:14:26 A lot of ControlNet information is found in  this link. I put it here. This is the official  

• 00:14:31 repository of the ControlNet. You can learn much  more about ControlNet in here. So let's begin  

• 00:14:37 with installation. There are two ways that you can  install. I will show both of them. For automatic  

• 00:14:43 installation I have a script that I have prepared  which I have shown in the beginning of the video.  

• 00:14:48 Just double click control net install. It will  clone it. If it already exists it will update it,  

• 00:14:54 then it will download automatically the missing  model files. After this operation is completed,  

• 00:15:00 all you need to do is restart your web UI with  webUI-user file. Let's continue with manual  

• 00:15:07 installation. So there are two ways of manual  installation. First one is manually cloning the  

• 00:15:13 repository. To do that copy this link git clone  enter inside Stable Diffusion web ui folder. Enter  

• 00:15:20 inside extensions. Open a new cmd here. Copy paste  it. It will clone the ControlNet into the correct  

• 00:15:27 folder. Then start your web ui and it will install  the necessary dependencies. The second way is like  

• 00:15:34 this: let me show you. To show you, first I will  delete the cloned extension. Start your web UI  

• 00:15:40 as usual. Sometimes this automatic installation is  not working. Therefore, learning how to install an  

• 00:15:47 extension manually is the best approach. Okay, web  UI started. Let's open it. Go to the extensions  

• 00:15:53 here. Click available and click load from and  in here search for ControlNet. You will see  

• 00:16:01 ControlNet with the name of SD web UI ControlNet.  When you click the name of the extension in the  

• 00:16:09 left panel, it will open the repository that it  is going to install and this is the correct SD  

• 00:16:14 web UI ControlNet repository. Click, install, and  look at the messages in the cmd window of Stable  

• 00:16:22 Diffusion web UI. Meanwhile, it is installing.  It also prevented us to click anything else. It  

• 00:16:28 says processing. If this screen is frozen, click  the screen of the cmd and hit enter. I see that  

• 00:16:35 it has already cloned the repository. However,  still, we are waiting. Maybe it is bugged again  

• 00:16:41 because it shouldn't take this long and it should  install the dependencies after restart. But let's  

• 00:16:48 wait a little bit more. Okay, it looks like the  installation procedure has been changed with this  

• 00:16:53 way, so it tried to install the dependencies of  the extension as well. However, we have OS error  

• 00:17:00 access denied. Because the virtual environment was  active. The Stable Diffusion web UI was already  

• 00:17:07 working. So I closed this window and I will  restart the web UI. Then this time it should  

• 00:17:12 install all of the requirements properly. You  see it says that it is installing requirements  

• 00:17:17 and this is the commit hash of my currently used  web ui. And meanwhile, the automatic installer is  

• 00:17:24 downloading all of the ControlNet models. You need  to download these models. I will show in a moment  

• 00:17:31 web UI started. Let's open it. And now you will  see ControlNet in the text image in bottom here  

• 00:17:39 under seed. This is the ControlNet extension. In  image the image you will see ControlNet extension  

• 00:17:45 as well. Pretty much wherever it is supported  you will see the ControlNet extension like this.  

• 00:17:51 However, now we are missing the ControlNet models  because we haven't downloaded any of them yet.  

• 00:17:57 So you won't see any models here. However, you  will see the preprocessors. So how are we going  

• 00:18:03 to download the models manually. You see there  is a link I have put here. This is the latest  

• 00:18:08 control net version 1.1. Click it. You will see  the model files here. The model files are the  

• 00:18:14 ones that are over 1 gigabytes. To download them  click this download icon. Let me zoom in more.  

• 00:18:21 You can download only the models that you need.  However, for demonstration, let's begin with  

• 00:18:27 downloading Canny for example. Click this. It will  start downloading. If you open your downloads,  

• 00:18:33 you will see the downloaded file like this. Okay,  the download has been completed. Cut the file,  

• 00:18:39 move into your installation, inside extensions.  This is important. This is where we put the  

• 00:18:45 ControlNet models. Inside sd-webui-ControlNet,  inside models. This is where you need to put  

• 00:18:53 your models. You see all of the yaml files are  here. Yaml files are configuration files. They  

• 00:18:59 are automatically downloaded with the extension.  However, you have to manually download the pth  

• 00:19:04 model files which are over 1 gigabyte. Then,  when I refresh here, I should see the model. Yes,  

• 00:19:11 now I can see the model and now I can use Canny  with this one. With this approach, you need to  

• 00:19:18 download all of the models that you want to use  and put them inside to that particular folder.  

• 00:19:24 Alternatively, you can use my auto downloader  script. It will download all of the models for  

• 00:19:30 you automatically into the correct folder before  starting, showing you how to utilize all of the  

• 00:19:37 different models of ControlNet. Let me also show  you how to use custom models with Stable Diffusion  

• 00:19:45 Automatic1111 web UI. You can download the models  from Hugging Face or from CivitAI. I prefer  

• 00:19:52 Hugging Face because in Hugging Face we can find  full precision models. For example, DreamShaper.  

• 00:19:59 This is one of the most famous models. By the way.  If you want to see famous models on Hugging Face,  

• 00:20:04 click the Stable Diffusion. It will list you  Stable Diffusion models sorted by most downloads.  

• 00:20:11 You can also sort them by recently updated or  from most likes. So from here you can see the  

• 00:20:17 most downloaded models and you can look each one  of them. So in DreamShaper when you click files  

• 00:20:23 and versions, you will see all of the model files  uploaded by the developer of this model. You will  

• 00:20:30 see different options such as pruned. I prefer  full models. Baked VAE means that this model  

• 00:20:37 has its VAE baked. Therefore, you can either  use its own VAE or the VAE file we downloaded.  

• 00:20:45 Let's download this model. When you click this  icon, it will start downloading. The model file  

• 00:20:50 has been downloaded. Whether you download it  from Hugging Face or from CivitAI, you need  

• 00:20:57 to put the model inside models directory, Stable  Diffusion web UI models, Stable Diffusion and put  

• 00:21:04 the models here. You see since the previous model  downloading has been terminated by us manually,  

• 00:21:10 it is not fully downloaded so I will delete it.  If your model download didn't complete, you will  

• 00:21:16 get an error while trying to use it. The automatic  downloader is still downloading the files because  

• 00:21:22 there are 14 different models. Let's say you  want to also update the ControlNet extension to  

• 00:21:30 the latest version. So for updating an extension  to the latest version, there are two approaches.  

• 00:21:36 The first one is manual updating which will work  every time. Go inside your extensions folder,  

• 00:21:44 enter inside the extension, open a new cmd when  you are inside the extension folder type git pull.  

• 00:21:51 This is manual updating. This will work always.  And the second way is let's open our web UI.  

• 00:21:58 Go to extensions tab in the web UI. Click check  for updates. It will check for updates and if it  

• 00:22:06 is not latest version, click apply and restart ui.  It will update the extension to the latest. Then  

• 00:22:14 you can restart the web ui by closing cmd and  reopening it. Automatic installation is fully  

• 00:22:20 completed, all models downloaded. I will continue  with automatic installation because I have shown  

• 00:22:25 everything with the manual installation by now.  Start the web UI with web user file. I just double  

• 00:22:31 click this bat file. I always use this file to  install and start my Automatic1111 web UI. Okay,  

• 00:22:38 the web ui has started. I know from this local  url. It automatically loads the VAE file we  

• 00:22:45 did set. Applying the optimization xFormers,  everything is loaded. Let's open the url. There  

• 00:22:52 is one crucial thing that you need to do before  start using the ControlNet. Go to settings. In  

• 00:23:00 here you will see ControlNet settings, click it  and in here you should select this option. Allow  

• 00:23:07 other scripts to control this extension. This is  really important because in many of the scripts  

• 00:23:14 that you may likely to use in future it will  require this option to be selected. There are  

• 00:23:19 also other options here you can use, for example,  passing ControlNet parameters with send to image  

• 00:23:26 to image. You can also check this and there is  one more thing. Model cache size. You can cache  

• 00:23:33 multiple models in your vram. However, use this  if you have sufficient amount of vram. Otherwise,  

• 00:23:41 don't use this because you may get out of vram  memory error. I have RTX3090, therefore I will  

• 00:23:47 use it. You can also allow max models amount that  you can use at the same time. By default it is 3  

• 00:23:54 and it is pretty sufficient. Apply it. After apply  we need to restart so I will close and restart. I  

• 00:24:02 also have updated the readme file and added this  section as well. You see I will update this file  

• 00:24:08 as it is necessary. Even after this video you can  make a comment to this video. I will update this  

• 00:24:14 file if it is being necessary. You can also open  issue or discussion in my GitHub repository. Okay,  

• 00:24:21 web has been restarted now we will begin. In left  up you will see the available models. Whatever the  

• 00:24:27 model you select here, it will be used in all  of the ControlNet actions that you are using  

• 00:24:33 and the VAE file you selected here will be used.  I will begin with showing you how to generate QR  

• 00:24:39 codes with ControlNet like you are seeing right  now. This one is working. First use this link to  

• 00:24:46 generate your QR code. Open it, select the action  whatever you want to generate your QR code for. I  

• 00:24:53 will generate for our channel. I am not using any  shorting. Make error correction level high block  

• 00:25:00 in pixel size. Now this is important. Depending  on your target resolution, make this bigger or  

• 00:25:06 smaller. I will make this as 30. When I generate  code the generated image is like this. When I open  

• 00:25:13 it in a new tab it shows me 1050 pixels by 1050  pixels. Save it as I save it as downloaded folder.  

• 00:25:22 Select your model, type your query. I will use a  simple query like this: portrait photo of naruto  

• 00:25:30 anime hero. Now there are so many different ways  to generate working QR code. Some of them are very  

• 00:25:36 complex and hard to use. However, I will show you  an easy one that won't use a lot of vram memory.  

• 00:25:43 Select DPM++ 2M SDE Karras make the sampling steps  like 50. Set your target resolution that you want.  

• 00:25:51 I will select it as 768. Open your control net,  load your downloaded QR code like you are seeing  

• 00:26:00 right now. Enable pixel perfect. We will use  inpaint model and from inpaint select inpaint  

• 00:26:07 global harmonious. This is the preprocessor  and as a model it will select inpaint so you  

• 00:26:13 must have downloaded this model to work. Then  the very important parameters that you can set  

• 00:26:19 is the control weight. As you increase control  weight, you will get lesser of your image and  

• 00:26:26 you will get more of an image like QR code.  Starting control step: as you start later,  

• 00:26:32 you will get a better image with lesser like QR  code. And if you start earlier you will get more  

• 00:26:39 like a QR code. Ending: you want this to be as  much as possibly lower. So starting control step  

• 00:26:47 will be as much as possibly higher, ending control  step will be as much as possibly lower. However,  

• 00:26:53 this totally depends on whatever the image and the  model you are using so therefore you have to try  

• 00:27:00 different values. If your QR code is not working,  try to increase this. Try to decrease this and try  

• 00:27:06 to increase this. Find a sweet spot that works  for you. Then let's generate eight images and  

• 00:27:13 see which one of them will work. Not every time  every image you generate will work. So you have  

• 00:27:18 to generate multiples and see which one of them  is working. By the way, this is a pretty simple  

• 00:27:23 prompt so you can use better prompts to get better  images. This is just for demonstration of how to  

• 00:27:31 generate QR code with ControlNet and I think this  is the easiest workflow that you will find on the  

• 00:27:37 internet. So the codes are being generated right  now. Since I have a powerful gpu, I made the batch  

• 00:27:43 size eight. However, if you don't have a powerful  gpu, then make the batch size one. Moreover,  

• 00:27:49 if your gpu don't have such amount of vram, make  this low vram. Okay, results are generated. Now  

• 00:27:57 what you need to do is click the image, open your  QR code application and start looking at them. For  

• 00:28:04 example. This is working very well. As you are  seeing right now this one is not working. This  

• 00:28:09 one is working. This one is not working. This one  is working. You see, we have got a lot of working  

• 00:28:15 images. Therefore, to get a better image, what we  can do, we can try to reduce ending control step  

• 00:28:23 and try again. And if it works then it is better  because we will get a better image. Let's try. So  

• 00:28:28 generating eight different images for RTX3090  is taking about one minute. For resolution 768  

• 00:28:36 768. Also, I am using xFormers, Torch version  2. Okay, the images are coming. So what does  

• 00:28:45 these values means? The control weight means that  how much should the ControlNet affect the output.  

• 00:28:52 This is the weight. This is the importance and  effect of the ControlNet. Starting control step:  

• 00:28:58 this means that when I said sampling steps 50,  the ControlNet will start after 20 step has been  

• 00:29:05 generated. An ending control step means that  it will stop using ControlNet when generating  

• 00:29:12 images after like 36 steps, you can multiply  them with this. Okay, images are generated.  

• 00:29:18 So let's see which one of them is working. I have  opened my QR code. Okay, this one working. Okay,  

• 00:29:25 this one is also working very well. Okay, this  one is also working. Wow. Amazing. This one is  

• 00:29:30 also working. Amazing and this one is also  working. You see, I have tried four and four  

• 00:29:35 of them and in all of them we have got a working  output. So therefore I can try to reduce ending  

• 00:29:41 control step one more time and repeat the process  until I get a non-working image. With this way  

• 00:29:48 I will have amazing QR code images. By the way,  as I said, this is not a good prompt. Therefore,  

• 00:29:54 I am pretty sure you will get much more amazing  results than me. And now we are seeing the results  

• 00:29:59 with ending control step 70%. Okay, the first one  is working okay, the second one didn't work. Let's  

• 00:30:06 try third. Okay, third one is working, the fourth  is working and the fifth is working. You see,  

• 00:30:12 we are still getting very well working images so  I can even try to further reduce this down like  

• 00:30:18 the 60 percent and I can try one more time. Okay,  time to test with 60 percent ending. Okay, first  

• 00:30:26 one is not working, the second one not working.  Third one not working. Fourth one not working  

• 00:30:32 fifth also failed. Sixth also failed. Seventh and  eight all failed. Let's try one more time. Okay,  

• 00:30:40 this time also all failed. So you see you need to  increase ending control step. One more time with  

• 00:30:47 60 percent and starting control step 40 percent  and control weight 60 percent. It didn't work.  

• 00:30:53 However, you can try. You understand the logic.  Before showing you all of the ControlNet control  

• 00:30:59 types. Let me show you the best settings for QR  code generation. I have used sampling steps 50,  

• 00:31:07 DPM++ 2M SDE Karras, resolution is 768 768 but  probably you can use any resolution. CFG scale  

• 00:31:16 7 probably you can use any CFG scale as well. So  used only single ControlNet, enable pixel perfect,  

• 00:31:24 inpaint, inpaint global harmonious, ControlNet  inpaint model, control weight is 60 percent,  

• 00:31:31 starting control step is 40 percent and ending  control step is 65 percent and control mode is  

• 00:31:39 balanced, crop and resize and this one you are  seeing right now is working very well. So let's  

• 00:31:45 begin with the first ControlNet option Depth.  Control Net works with maps in most cases and  

• 00:31:52 they have multiple maps options. So I will upload  my own picture this one and let's enable it pixel  

• 00:32:00 perfect and let's use Depth as an option. You  see there are several preprocessors that you can  

• 00:32:06 select and when you click this run preprocessor  it will show you what will this image turn into  

• 00:32:12 as a Depth map. When you first time you try to  use ControlNet, it will download the necessary  

• 00:32:18 model. Currently it is downloading Midas model to  be able to preprocess with Depth midas as you are  

• 00:32:27 seeing right now and on the screen you are seeing  it is downloading the Depth midas model. This  

• 00:32:32 will happen only one time for each new model.  New preprocessor that you use. So it will use  

• 00:32:39 this preprocessor to get the map of this image.  Then it will use the Depth model to generate  

• 00:32:45 image. So this is the preprocessor preview. Let's  generate an image based on this Depth model. So I  

• 00:32:53 have typed an awesome intricate portrait photo of  handsome actor. This is the result we got. So you  

• 00:33:00 see this result is based on this Depth map. So it  used this Depth map and it generated this amazing  

• 00:33:07 picture with just this simple prompt, no negative  prompts, sampling steps 20 and Eular a. I can also  

• 00:33:15 change resolution. Let's try with 768 and 768 and  now we are going to get another picture. So this  

• 00:33:22 is the result you see based on this Depth map. So  you can also try other preprocessors. For example,  

• 00:33:28 let's try Depth_leres++. If you want to see  what kind of map image it will generate,  

• 00:33:34 just click this preview icon. It will generate  the preview. Since this is also first time it  

• 00:33:40 is downloading 500 megabytes. It also shows where  these files are downloaded. It is being downloaded  

• 00:33:47 into sd web ui control that extension folder  into annotator downloads leres you are seeing  

• 00:33:55 right now. Okay, this is the result of Depth  leres++. Let's generate an image. So you see  

• 00:34:02 each Depth map will generate different maps and  it will affect your results differently. So this  

• 00:34:09 is another result we got with the same prompt and  different Depth map. There is also Depth leres.  

• 00:34:16 Each time it may use different models. Depth zoe.  So it is also going to download this Depth_zoe and  

• 00:34:23 it is 1.34 gigabytes. On the official ControlNet  GitHub page this is the information they provide  

• 00:34:32 for Depth 1.1 version. Okay here we see the result  of using Depth_zoe. Let's generate an image with  

• 00:34:40 Depth zoe. Each Depth map will be different and  based on whatever you want to achieve you can try  

• 00:34:47 them and this is another image we got. So the next  one is ControlNet Normal maps. Let's select Normal  

• 00:34:54 from here. So these are pre-made selections. When  you click them they will automatically load the  

• 00:35:00 pre-made selections. These are coming from the  developer. So when I click Normal it select the  

• 00:35:06 Normal_bae and Normal_bae model. Let's click the  preview and see what kind of result we are going  

• 00:35:12 to get. Again since this is a fresh installation,  it is downloading the model file only one time and  

• 00:35:18 this is the Normal map. Let's also use the same  seed to see the effect difference. Let's hit  

• 00:35:24 generate so with same seed with different map  we will see what kind of image we are going to  

• 00:35:30 get and this is the result of Normal map. Almost  completely different images based on your needs,  

• 00:35:37 you can try different control types. Let's look  for Midas. So this is the Midas result. Let's  

• 00:35:44 click, generate and see the difference. Okay, this  is the Midas result. Let's also try none. Okay,  

• 00:35:51 when I use none, that means that the image I  uploaded is like a map like this so it won't work  

• 00:35:58 properly. Let's try and this is the result we  got because it used this base image as a map  

• 00:36:04 which is not a map. And this is the description  the developers provided for Normal maps in the  

• 00:36:10 version 1.1. And the next one is Canny. I like  this model a lot. This is amazing one. Let's try  

• 00:36:16 it. So I click Canny and I click run preprocessor  and this is the Canny preprocessor result. You see  

• 00:36:24 this turned into this. In Canny there are several  options that you can set Canny low threshold and  

• 00:36:30 high threshold. This will determine your output  and it will affect it significantly. For example,  

• 00:36:36 let's try 1 and 1 and let's see the result. So  this is the result of 1 and 1. Let's first try  

• 00:36:42 with 1 and 1. This is the result we get when Canny  threshold is 1 and 1. Let's open it in another tab  

• 00:36:48 and let's make the threshold as 100 and 100 click  preview. So this is the 100. Let's generate and  

• 00:36:57 now we got completely different. By the way, you  see, the face is very similar to my original face.  

• 00:37:03 Original image because Canny threshold maps your  face exactly as it is and it makes the generated  

• 00:37:11 face similar to yours. So from this to this.  Let's also make it like 200 and 200 and this is  

• 00:37:19 the result of 200 you see 200, 100 and only 1.  Let's try another preprocessor. It says invert  

• 00:37:26 from white bg black line when I click preview this  is the preview. Let's try with it. By the way,  

• 00:37:32 also in Canny or other models you can set the  control weight and have different effects. Okay  

• 00:37:38 with invert we got completely unrelated thing.  And this is the description of the developers  

• 00:37:44 for Canny model for 1.1 version, just pause the  video and read it if you wish. The link for this  

• 00:37:51 page is here as I have shown in the beginning.  Okay now MLSD straight lines. This is another  

• 00:37:58 model. I never used this model actually so  let's select MLSD. Let's click preview. Okay  

• 00:38:04 this is what we get because it is trying to get  I think the straight lines and in this image we  

• 00:38:09 don't have much straight lines. So for testing  straight lines I will use this downloaded image  

• 00:38:14 which is this one. Let's click preview and now  when I use a room I get some straight lines as  

• 00:38:22 expected. I also need to change the prompt like  a room and let's try. Whenever you change your  

• 00:38:28 model it will cache it in the memory so it will  take some time. Okay now you see it is working.  

• 00:38:34 We got a room that is very similar in structure  and shape to the input image as you are seeing  

• 00:38:40 right now. So this is what MLSD is used for.  There are also different options for example,  

• 00:38:46 let's try threshold like this and let's see the  preview. Let's make it like this and this is the  

• 00:38:52 preview. As you increase the value threshold you  will get different results and this is when the  

• 00:38:57 both threshold is minimum. Let's try again and  we get this image. It is almost same with the  

• 00:39:03 same seed. It also has invert from white bg black  line. Let's click preview and this is what we get  

• 00:39:09 when we invert it. Completely unrelated. This is  the description that developers has written for  

• 00:39:15 MLSD if you want to read and Scribble. Scribble is  amazing. I will show you. Scribble turns scribbles  

• 00:39:23 into full images. So I loaded back my image. Let's  select Scribble. Okay, it selects Scribble pidinet  

• 00:39:30 let's click preview. It turned my image into  a scribble as you are seeing and let's try the  

• 00:39:36 result. I just typed a handsome man it is going  to use this scribble. This is the result we got.  

• 00:39:42 It looks pretty unrelated. It generated this image  from this scribble map that is why. Actually it is  

• 00:39:52 kind of matching. You see it found something  like this in here and it turned that into a  

• 00:39:58 face because I just typed a handsome man. So  what else we can do We can select Scribble hed  

• 00:40:04 as a preprocessor. It will change the map. With  Scribble hed I think we got more details than  

• 00:40:10 this one obviously. Let's try. This is the result  we get with Scribble hed. Let's try Scribble xdog  

• 00:40:17 this is the preview. I think this will be more  detailed, more similar. This is the result we  

• 00:40:24 got. Yeah. Let's say you have your own scribble.  So what you need to do: Upload your scribble into  

• 00:40:30 here. I will upload it from a downloaded photo and  make the preprocessor none because this is already  

• 00:40:37 like a scribble. And let's change the prompt. A  mighty dragon and see what we are going to get.  

• 00:40:44 So you see if your image is already preprocessed,  then you select none in the preprocessor and this  

• 00:40:52 is the result we got from this scribble to  this. Why this result is not very good because  

• 00:40:58 my uploaded image dimension is 612 by 428.  Therefore, I need to change this 612 by 428  

• 00:41:10 and now we will get much better results. Let's  see. Okay. You see now we got a very close to  

• 00:41:17 our scribble like this like you are seeing  right now. You can also change my prompt is  

• 00:41:23 more important. Let's try with that. My prompt  is more important than the ControlNet and this  

• 00:41:29 is the result we get. A mighty dragon from this  scribble. It is pretty much close actually. Let's  

• 00:41:36 also try with ControlNet is more important. Now  it will try to match the full Scribble into this.  

• 00:41:42 Actually it is not very like right now. Let's  see maybe we need to. Also let's also try with  

• 00:41:50 a scribble hed. Oh no. Because scribble is not  as strong as lineart which I will show. But this  

• 00:41:58 is how you can use scribble. Okay in Scribble the  developers gave this example also this example as  

• 00:42:05 you are seeing and this is the description of the  Scribble given by the developers. So the SoftEdge.  

• 00:42:11 This is another control. With SoftEdge it also  turns your image into like a scribble but with  

• 00:42:18 soft edges. So let's select SoftEdge from here.  Let's click the preview with SoftEdge. So this  

• 00:42:25 is the SoftEdge preview. It has soft edges. Let's  return back to older resolution and hit generate.  

• 00:42:32 Okay here we see the results of SoftEdge. This  is the map the ControlNet generated and this is  

• 00:42:39 the result. I think this is pretty cool. Pretty  amazing. There are also others options that you  

• 00:42:45 can try for example SoftEdge hedsafe safe. Let's  also try it. Okay, this is another one as you  

• 00:42:51 are seeing. Pretty cool. Let's try pidisafe  and this is the result we get with pidisafe.  

• 00:42:58 Okay finally we have hed. Let's try with hed.  I hope I am pronouncing the keywords correctly.  

• 00:43:04 These are hard to pronounce and this is the result  of hed. You see it is almost like stylized image  

• 00:43:10 of me. Maybe I don't know. It's up to you. This is  the description that the developers have written  

• 00:43:15 for SoftEdge 1.1 version and the next one is  segmentation. It will turn image into segments.  

• 00:43:23 Let's try it. So for segmentation I select this.  Let's see how it will segment my own image. By the  

• 00:43:30 way, in the first time, segmentation will download  additional model files. This is the result we get  

• 00:43:35 with the segmentation seg ofade 20k. Let's try the  result. From this segmentation we got this result.  

• 00:43:43 You see in the main subject we have a man, then  we have a background where I sit a chair and in  

• 00:43:51 the top we have another man. Why? Because in our  prompt we only have a handsome man. So if I change  

• 00:43:57 it like this sitting on a chair having a forest  picture behind the wall hanging, let's try it.  

• 00:44:03 Okay now you see everything pretty much changed.  We have the man, we have the armchair and we have  

• 00:44:10 the picture behind the wall hanging. So you see  it associated the prompt with the segments that  

• 00:44:17 we have. We have wall, we have picture, we have  man and we have armchair. This is amazing. So try  

• 00:44:23 to understand how this is working. Try to use  this according to your purpose. Let's also try  

• 00:44:30 other segmentations. Seg ofcoco. Each time it will  download a different model unfortunately. Okay,  

• 00:44:37 this preprocessor seg ofcoco generated pretty  different segmentation. This was the previous one  

• 00:44:44 and you see, this is the new one so it missed the  painting hanging on my wall. It added a different  

• 00:44:51 layer to my armchair. Let's try this one result.  For this one probably I also need to modify  

• 00:44:57 this prompt. Okay, you see now the back of my  wall became a forest painting and this is pretty  

• 00:45:06 amazing. This is wild actually. Okay, let's  try seg ufake 20k. Let's click the preview.  

• 00:45:14 Is also downloading another model, but this one  is smaller one. But this is amazing. This is just  

• 00:45:20 a thing that I have discovered. How to make your  background as a painting, your wall as a painting.  

• 00:45:28 So the smallest model performing the worst.  As expected. Seg ufade 20k it is just pretty  

• 00:45:36 useless. If you have your single image as like  this, then you select none from here. As I have  

• 00:45:44 shown in another example, this is the logic of  how preprocessor works. If your image is already  

• 00:45:50 preprocessed, then you select none from here.  For example, let me save this as into downloads,  

• 00:45:56 upload it, from here. Then I select none and I  hit generate and it should generate same image  

• 00:46:03 of this one. And yes, it generated same image of  this one because I have uploaded a map instead  

• 00:46:10 of a full image and this is the description that  developers have written for segmentation. OpenPose  

• 00:46:17 is another very powerful feature of ControlNet.  It has so many options as you are seeing. So I  

• 00:46:24 have uploaded one of my ai generated image. Let's  select OpenPose and OpenPose full. Let's see the  

• 00:46:31 result of OpenPose full and this is the result  of OpenPose full. There is also edit option,  

• 00:46:37 but it is requiring OpenPose editor so we can  also install this. So this is the link. Let  

• 00:46:43 me also show you how to install it. I copied the  link, then I go to extensions, install from url,  

• 00:46:49 paste it, and install. Okay, it is installed.  Then I need to apply and restart ui. Okay,  

• 00:46:56 I am doing that right now. It is restarted. Let's  click this to get the last values, and then let's  

• 00:47:03 open ControlNet. Let's open my ai generated image.  Enable, pixel perfect. OpenPose, OpenPose full,  

• 00:47:12 click preview it generated OpenPose. Okay,  I think it requires restart of web UI. Yeah,  

• 00:47:20 it was supposed to be added in the extension  it is displaying here. Okay, let me restart it.  

• 00:47:27 So I closed cmd and restarting the cmd from  web UI to start it. When restarting web UI,  

• 00:47:34 you see it displays whatever the arguments used.  Whatever, the other options are loaded. Okay,  

• 00:47:41 web UI restarted. Let's see if it is available  now. Let's load, enable pixel perfect. OpenPose  

• 00:47:48 preview okay, now I have edit option so probably  we needed to restart. Okay, but it is not working.  

• 00:47:56 Weird so looks like this extension is not. Oh,  when I click it again edit: it appeared so from  

• 00:48:04 this edit you can change the pose. So this is  nose. Wherever you want nose to be positioned,  

• 00:48:10 you can change it. These are the eyes positions.  You can change them. This is the ear. This is the  

• 00:48:16 body torso, the shoulders, the torso. So this is  what it is useful for. Send pose to ControlNet.  

• 00:48:25 Now this is the changed pose and let's try. By the  way, this prompt won't work so let's make it like  

• 00:48:32 this. Okay, let's generate another one with this  post. Okay, so this is the result we are getting  

• 00:48:37 with this OpenPose like this. However, this is not  matching this one because of the resolution. For  

• 00:48:45 this to be matching, I think I need 768 by 1024.  Let's try again. Okay now it is more matching. As  

• 00:48:53 you are seeing it is trying to match to this pose.  So this is full pose. There are also face only.  

• 00:48:59 When I click face only it will only put the face  not the body position. Let me show you the result.  

• 00:49:06 So it will try match the face not the body. It  is still similar. Let's also try with hand but  

• 00:49:14 we don't have any hand in this picture so this  is another good example. Okay, I think we can try  

• 00:49:19 with this one. Let's see. Okay, it shows hands as  well right now let's try the pose like this. Okay,  

• 00:49:27 it matched pretty much. Well. Yeah, so from this  pose, this original image, we generated this image  

• 00:49:35 and it is looking amazing. quality Yes, looking  pretty amazing. Pretty accurate as well. So this  

• 00:49:41 is how you can use OpenPose control. I can click  this edit and I can edit the generated pose from  

• 00:49:49 here. I can set it whatever I want. For example,  let's make this like this: let's see what we will  

• 00:49:55 get. So I click send pose to ControlNet. Okay, now  the hands are not matching though. The hands are  

• 00:50:01 gone. Okay, I think hands are shown here. So okay.  Let's reset zoom and okay it says space f drag  

• 00:50:09 mouse so I will click space f. Okay, currently I  am hitting space and f keys so that I am able to  

• 00:50:17 drag and move the picture like you are seeing. So  let's also move the hands. Okay, oh, each hand is  

• 00:50:25 like this. Let's reset zoom. No. Okay, okay. Space  f zoom in, select all hand, move hand to here and  

• 00:50:35 then select this hand and move hand to here as  well. Send pose to ControlNet and generate. I  

• 00:50:41 wonder what kind of result we will get from this  pose. Wow. Amazing. You see the hands are now here  

• 00:50:47 like this. They are not completely same, but it  matched the hand to the position of the arm and  

• 00:50:54 it is amazing. This is another cherry pick as you  are seeing right now. This is a live action. So  

• 00:50:59 this is how you can use OpenPose and this is the  description that the developers has written on  

• 00:51:05 their page and the next one is Lineart. Lineart is  actually the amazing one for turning sketches into  

• 00:51:13 full arts. So let's load again our dragon image.  I set its original resolution. Mighty dragon and I  

• 00:51:21 select the lineart from here. But I will make the  line art preprocessor none and let's hit generate  

• 00:51:29 and see what kind of result we are going to get.  Okay, it didn't work interestingly. We got a very  

• 00:51:35 different oh oh it is using my other preprocessor  probably I need to. Okay, it looks like this is a  

• 00:51:42 bug because it used the previous preprocessor.  Let's click the preprocessor preview so it is  

• 00:51:47 changed and try again. We just discovered a bug  in the control net. Okay now you see it is keeping  

• 00:51:54 the entire shape and the structure very well. This  is the result we got. This can be improved a lot.  

• 00:52:02 Let me add some beautifying prompts. I added some  beautifying prompts but still not very good. Let's  

• 00:52:08 also try high resolution fix. By the way, still,  I am using the same seed. That is why I am getting  

• 00:52:13 same image when I generate multiple. Okay, I will  change the preprocessor lineart anime and this  

• 00:52:20 will turn into let's see. Okay line art anime is  downloading a preprocessor model and let's change  

• 00:52:27 the high resolution fix and I will make this  double size like this. This is what we got from  

• 00:52:33 line art anime. Let's try like this: double size  of the original resolution. Okay, we are getting  

• 00:52:40 yeah, this is the result we got from line art  anime. Still not very good. Let's try line art  

• 00:52:46 anime denoise I wonder what kind of output this is  giving. This is also downloading another model. It  

• 00:52:53 is almost looking like same as the previous one.  Okay, we got this result. Not very good either.  

• 00:53:00 Lineart coarse, each time it is downloading new  models. Okay, let's try this one. By the way,  

• 00:53:06 the preview resolution is half of what we are  generating right now. Also, this is a pretty  

• 00:53:12 big resolution. This time I will try line art with  a real image that is from twitter. Let's download  

• 00:53:18 it. Let's upload it. Let's select our line art  realistic and click preview: okay, this is the  

• 00:53:25 realistic preview we got. By the way, we should  get the resolution of this image so it is 736 to  

• 00:53:32 1104 and then we need to change also the prompt.  A man is against a mighty dragon. Okay, let's hit,  

• 00:53:43 generate and see what kind of result we will get.  It is pretty amazing, but it is not as good as the  

• 00:53:50 original image of course, but it is pretty amazing  when considered. We can also try several different  

• 00:53:56 seeds. Let's make it batch size 8. Okay, we got  the results. Let's look each one of them. This,  

• 00:54:03 this, this and this one. Wow. Pretty amazing.  This one. So you see you can turn your line arts,  

• 00:54:11 sketches into amazing pictures, drawings before  moving into next one. I want to show you an  

• 00:54:19 example from a real line art. Free line art from  deviantart for example. This one. Let's download  

• 00:54:26 it. This is a free dragon line art so you can  use this, generate image and use it as you  

• 00:54:31 wish. The resolution is 866 to 684. Okay, I did  set the resolution. Let's load it. Let's select  

• 00:54:40 preprocessor none from here. And how would you  describe this? I typed this: masterpiece amazing  

• 00:54:47 colorful flying dragon creature. I wonder what  kind of result we will get. Right now producing  

• 00:54:53 eight of them with page size 8. Again, we got  the same results as the previous one with the  

• 00:54:59 new resolution because it is still using the  preprocessor preview. You see when I changed  

• 00:55:04 the prompt it changed the result significantly.  However, it is using the previous dimension and  

• 00:55:10 this preprocessor. So let's click the preprocessor  here to refresh it and try again. Okay, we got the  

• 00:55:16 results. So this is the used line art as a map.  This is the first result. Second one, third one,  

• 00:55:24 fourth, fifth, sixth one. Oh this is a pretty  cool one actually, it's just missing some of the  

• 00:55:30 colorings. So you see you can change your prompt  and try different ones. Oh this one is also pretty  

• 00:55:36 amazing. I wonder what we would get with high  resolution fix: use this seed and just only one  

• 00:55:43 batch size. Okay with high resolution fix: this is  the output and this is looking amazing. But in the  

• 00:55:50 wing of the creature we have a repeating issue.  I wonder if this can be fixed. Probably hard,  

• 00:55:57 maybe with lower denoising strength. This is what  we got with lower the noise but this was a too  

• 00:56:03 low. So let's try like this. You see when you go  below certain denoising strength then you lose the  

• 00:56:10 clearness, sharpness or maybe focus of the image  we can say. It is not anymore like fully focused,  

• 00:56:19 it is like blurry image. Okay this is another  image we have. So you see developers are showing  

• 00:56:25 also. Same if you have a line art like this a  sketch then you select none as a preprocessor  

• 00:56:31 and you generate images like this. You can use  line art for making amazing logos as well. Let  

• 00:56:38 me show you. So I have downloaded the ChatGPT  logo here. Select lineart from here as usual.  

• 00:56:47 You don't need to change anything. Enable. Pixel  perfect, select resize and fill because we will  

• 00:56:54 make it expanded. Select your resolution from here  like 768 by 512 and type such as aerial glacier.  

• 00:57:04 Like this hit generate. It is totally up to your  imagination that how you prompt for example this  

• 00:57:11 is the first example. Let's make it batch size 8.  Okay we got some amazing results. First second,  

• 00:57:19 wow. Third one amazing. Fourth oww fifth one.  This is also nice. Sixth, seven and eight okay  

• 00:57:27 here another one. Aerial desert. First  one oh second one pretty good. Third one,  

• 00:57:33 fourth one fifth one. This is looking pretty  decent. So these are the other results. It is  

• 00:57:40 totally up to you as I said and here we see  aerial jungle. So these are the results. Wow  

• 00:57:47 amazing one. This is also pretty cool. Wow. This  is also good. This is also good. Some very good  

• 00:57:54 results. It is important that you select resize  and fill. Because this logo this scribble is 512  

• 00:58:03 and 512 which is one over one aspect ratio. It is  important that to select resize and fill. So this  

• 00:58:11 is resize and fill result you see. This is the  map it generated. Let me show the crop and resize  

• 00:58:19 difference using the same seed. So this is crop  and resize difference as you are seeing right now:  

• 00:58:26 resize and fill, crop and resize. Let me make it  just resize and this is the result of just resize.  

• 00:58:33 The just resize is worst. I think resize and fill  is the best and this is crop and resize so it will  

• 00:58:41 affect the result that you are going to get when  you are working with non-matching aspect ratio.  

• 00:58:47 So this is the aspect ratio that I am generating  and the aspect ratio of the logo is not matching  

• 00:58:53 with this one. This can be applied pretty much  anything. To make good text to make to make logos.  

• 00:59:01 This is another logic of using line art. So the  next option is Shuffle. This is a new feature that  

• 00:59:07 comes with ControlNet 1.1. It shuffles the image.  It is pretty interesting so let's see what kind of  

• 00:59:16 result we will get. I will select the Shuffle  from here. I will click pre. Oh, there is no  

• 00:59:21 preview but I need to delete this preview. Maybe  close? no. Okay, let's refresh the page to be  

• 00:59:28 sure. Let's upload our shuffling image. I will use  this image for shuffling. Let's select Shuffle and  

• 00:59:37 I entered the original resolution of the  image and how should we shuffle it? Okay,  

• 00:59:42 I just typed. Amazing dragon oh I forgotten to  enable. Let's enable pixel perfect and let's  

• 00:59:49 Shuffle again. It is loading the model, it is  setting the proper resolution. Okay, I see. So  

• 00:59:57 you see when we used Shuffle it almost uses the  same colors and it Shuffled them like this. So  

• 01:00:04 this is the image it generated based on the colors  used in this image and this is the completely  

• 01:00:11 Shuffled output. Actually, this is pretty amazing.  Maybe we can try to generate a lower resolution to  

• 01:00:18 get a better one. Wow, I see this is pretty  amazing. Not the dimension is matching so we  

• 01:00:25 can try as half of the original dimension. We can  keep the aspect ratio. So this is the half of the  

• 01:00:31 original resolution image. So here I have reduced  the dimension according to the aspect ratio of  

• 01:00:38 the original image. This is the result we got and  this is the shuffling. You can play with this and  

• 01:00:43 if you find it useful let me know in the comments.  This is pretty interesting feature. This is also  

• 01:00:49 pretty new. So this is the description of the  developers written. For example, they have given  

• 01:00:55 this image and Shuffled it as ironman. Wow, this  is pretty interesting. Let's try this. So I will  

• 01:01:02 test it with this image. Let's save it as ironman  png: let's refresh the page. I will turn this into  

• 01:01:09 spider-man. Spider-man select the image from here.  Enable pixart perfect Shuffle. By the way, Shuffle  

• 01:01:16 has none other preprocessor. We have a preview.  Okay, this is the shuffling preview and let's set  

• 01:01:24 the resolution same and let's hit generate.  With shuffling ironman into spider-man okay,  

• 01:01:30 it is not very like as the developers show. Maybe  I can try superman. I wonder what we will get. The  

• 01:01:37 spider-man is not very like. So this is Shuffled  into this superman. I don't know if it is working  

• 01:01:43 or not. It doesn't look like. Yeah, maybe uncheck  this and try again. Okay, without pixel perfect,  

• 01:01:49 it still doesn't look like shuffling as shown  here because it is supposed to turn this image  

• 01:01:56 into ironman. However, in our case it doesn't  look like turning this image into this one. It  

• 01:02:03 is a little bit similar, but not like yeah. Maybe  I can try with my own image. So let's turn myself  

• 01:02:09 into superman. Okay, so it turned this image into  this one. Pretty different. Maybe because of the  

• 01:02:16 aspect ratio let's try it. So let's try with 768.  Okay, so it tries to match the colors and turn it  

• 01:02:25 into this one. This is very interesting. This is  the Shuffled image and this is the other image.  

• 01:02:30 Anyway, I never used shuffling in any of my daily  life, but it is supposed to work like this. Okay,  

• 01:02:36 multi ControlNet. Multi ControlNet is extremely  important feature. So they used this image as a  

• 01:02:45 base image and then they used Shuffle image as  a this image. Let's also download it. Let's load  

• 01:02:50 them. So the first image will be used as Canny. It  is enabled. The second ControlNet will be used for  

• 01:02:57 shuffling enable, pixel perfect, shuffle. So we  got this one and we got canny. Let's try ironman.  

• 01:03:05 I wonder what kind of result we will get. Now  it will use two images together. Okay, I see it  

• 01:03:12 Shuffled ironman with this skin and it generated  this image. Pretty pretty decent actually. Pretty  

• 01:03:21 amazing also. So this is the resolution of line  art. Let's select proper resolution. So I changed  

• 01:03:29 the resolution according to the dimension of the  ironman image. Yeah, we got a pretty decent output  

• 01:03:36 from shuffling and from Canny. Sorry I said that  line arts but they used Canny. Not line art. So  

• 01:03:43 let's try also with line art and see what kind of  result we will get. This is Canny plus shuffling.  

• 01:03:49 It is pretty amazing actually and this is line  art plus shuffling I see. Wow. Interesting. I  

• 01:03:58 think this is still early days of shuffling  so it may get better over time. These are the  

• 01:04:04 outputs they had. Now instruct pix2pix. This is  another feature of ControlNet. So let's refresh  

• 01:04:11 the page. I will try with my own image. So  I have uploaded my image. I click enable,  

• 01:04:17 select IP2P. This is instruct pix2pix. It doesn't  have any preprocessor. This works with giving  

• 01:04:25 natural commands. So what can we say: we can say  that turn hair color into red. I wonder if it  

• 01:04:33 will work? Let's hit generate so it is loading the  instruct pix2pix model. Okay, it didn't work very  

• 01:04:40 well. It turned entire image into red. What else  can we try? Okay, make the man in picture have  

• 01:04:46 longer beard but it changed the face completely.  It is supposed to obey our commands and make it  

• 01:04:54 on fire. For example, they have provided in their  page. So I have uploaded an image of a horse and  

• 01:05:01 I said make the horse wearing an armor. It is  pretty much turned it into correcting but it  

• 01:05:09 changed the background. The quality decreased it.  So instruct pix2pix of ControlNet is not working  

• 01:05:15 very well. There is actually an original  instruct pix2pix and it was working better.  

• 01:05:20 This is the original paper of instruct pix2pix.  This was better than ControlNet instruct pix2pix  

• 01:05:27 and I think it is still like that. So ControlNet  instruct pix2pix is not as good as the original  

• 01:05:33 instruct pix2pix paper. It is supposed to work  very well. Let's try make it winter. This looks  

• 01:05:40 like a another prompt. Make it winter. By the way  the results will depend on our selected diffusion  

• 01:05:46 model. Okay, this is what we get with dream  shaper. Let's also try with the base model. Okay,  

• 01:05:52 let's hit generate with the base model. Maybe  it will work better. I'm not sure. Okay,  

• 01:05:57 it is not working good yeah. Inpaint. This is a  new feature of ControlNet. This is the description  

• 01:06:04 they have written. There are several options of  inpainting with control net. So they have some  

• 01:06:11 guidelines linked here. I am going to use this in  my upcoming full realism model. Let me show you an  

• 01:06:20 example. So you see on the left I am. This middle  one is trained model generated ai image and on the  

• 01:06:27 right you see the output of ControlNet inpainting  and tiling output. I will show this. Hopefully in  

• 01:06:35 my upcoming tutorial I will show how to generate  studio quality images so stay subscribed. For  

• 01:06:43 testing inpainting. I generated an ai image a  man yielding a mighty sword like this. Let's  

• 01:06:50 send it into send to inpaint. And let's select  from ControlNet and select inpaint mode. Enable,  

• 01:06:58 pixel perfect and let's modify the face to improve  it like this. Let's inpaint. I will also inpaint  

• 01:07:07 without control net to check out the difference.  Okay, it says that a1111 inpaint and ControlNet  

• 01:07:13 in paint duplicated. ControlNet support enabled.  Okay so this is the output we get with in paint of  

• 01:07:21 ControlNet based on this mask. Actually it doesn't  look good to me. inpaint global harmonious. Let's  

• 01:07:28 try with it. Okay, this is what we have still not  looking very well and this is the result. With the  

• 01:07:35 third inpainting option inpaint only + lama. Okay  let's try without enabling the ControlNet and this  

• 01:07:43 time I will set only masked because this is the  inpaint of the automatic1111. It is 75 percent.  

• 01:07:51 Maybe we should try like 45 and let's see the  result. It looks like Automatic1111 inpaint is  

• 01:07:57 working better at this example. You can get the  true power of inpainting of ControlNet when you  

• 01:08:04 utilize it along with tiling feature. So let's  open our image, send to inpaint. Select inpaint  

• 01:08:12 not masked. Select resize by 2x. You can also  increase this. Increase denoising strength to 1  

• 01:08:19 because we will utilize tiling. Open ControlNet  enable, pixel perfect, inpaint, inpaint only is  

• 01:08:27 selected. Then get your controlnet unit one enable  pixel perfect select tile and let's hit generate  

• 01:08:34 and here we got the amazing results. You will be  amazed by the results. So this is the result of  

• 01:08:40 inpainting + tiling of ControlNet. Let's compare  them. This is the new result we got and this is  

• 01:08:48 the original image. You see it has significantly  improved the face quality the hands quality. I  

• 01:08:56 see some errors in the feat so we can generate  multiple ones, but with this feature you can both  

• 01:09:03 upscale and improve the quality significantly.  This is where comes the power of ControlNet  

• 01:09:10 inpainting when combined with tiling. You see from  this image to this image. As some of you may have  

• 01:09:17 noticed, we have lost some of the coloring when  we have used inpainting plus tiling feature. The  

• 01:09:25 developers also were aware of this issue and they  have released a new preprocessor. The tile color  

• 01:09:32 fix plus sharp. So go to your tiling ControlNet  in the preprocessor section. Try tile color fix  

• 01:09:40 plus sharp. Let me demonstrate you the its  results. Moreover, you can change variation  

• 01:09:45 and sharpness with this preprocessor and it will  affect your results. The developer has shown some  

• 01:09:52 comparisons. So this is the input image they used.  You see it is only 480 pixels by 640 pixels and  

• 01:10:00 then they have used tiling. 100 percent denoising  strength, no extra upscaler, super resolution  

• 01:10:07 or other things. Old man best quality. If you  wonder what is T2I and I2I. This is text to image  

• 01:10:15 and this is image to image. With ControlNet Tile  resample, this is the result. With color fix this  

• 01:10:21 is the result and with color fix plus sharpness,  this is the result. Here our new result. Let me  

• 01:10:27 compare. This is regular Tile resample. This is  Tile resample plus sharpness correction. Let's  

• 01:10:34 look at them side by side. So here right now on  the left we are seeing the original raw output on  

• 01:10:40 the middle Tile resample. On the right we see Tile  resample plus color correction plus sharpness fix.  

• 01:10:49 You see it is much more colorful. It is looking  much more sharp. So you can also change the  

• 01:10:55 settings here and see how they are affecting.  For example, let's make some experimentation:  

• 01:11:00 variation 16. So here we are seeing the variation  8 and here we are seeing the variation 16, 8, 16,  

• 01:11:08 8, 16 so it changed the colors position looks  like. It is up to you to test and see whichever  

• 01:11:17 one you like more. I think variation 8 has  more details. Now I will show you the famous  

• 01:11:23 generative fill feature of ControlNet which has  arrived recently to the ControlNet. I am pretty  

• 01:11:29 sure you have heard the amazing generative fill of  Photoshop. Now we have similar feature with Stable  

• 01:11:36 Diffusion automatic1111 web UI with the extension  of ControlNet. This is their new preprocessor for  

• 01:11:43 outpainting and inpainting as a generative fill  of Photoshop. So they have used this as an example  

• 01:11:50 image. I will also use it. Let's save image as  into our downloads. Let's refresh our web UI.  

• 01:11:56 Load image into your ControlNet. When you are  using generative fill feature of ControlNet,  

• 01:12:03 you don't type any prompt. Make the cfg like 4.  So let's say you want to inpaint and remove an  

• 01:12:11 object. How can you do that. Mask the object that  you want to remove like this. Then in the bottom  

• 01:12:18 enable pixel perfect. Inpaint. Select inpaint  only + lama. This is better than inpaint only  

• 01:12:28 for generative fill feature of ControlNet for  removing objects or outpainting image. I will  

• 01:12:34 also show outpainting logic So that's it. Let's  hit generate. By the way, this is not the correct  

• 01:12:41 dimension of the image so it will get changed but  still working very well. You see, we have masked  

• 01:12:47 this area. It has removed that image completely  and perfectly fine. So this is generative fill  

• 01:12:53 feature of ControlNet. You can use this to remove  images. Then let's also check out how to outpaint.  

• 01:13:01 The original image we downloaded is 500 by 750  pixels. So we are going to change the output  

• 01:13:08 resolution to the opposite. Like this, this means  that it will expand the image in width and it will  

• 01:13:16 crop the image in height. For outpainting the  work you need to select resize and fill option  

• 01:13:21 in ControlNet. This is really important. You  don't need to type anything. Okay, this is the  

• 01:13:27 first result we got as an outpainting. It is not  fabulous, but we can do more trials and see if we  

• 01:13:34 will get better. By the way, this is DreamShaper  model. Whatever the model you use will make a  

• 01:13:38 lot of difference. After changing resolution to  768 and 512 making the model realistic vision,  

• 01:13:46 I have a very decent result. So on the left you  see the original image and on the right we see the  

• 01:13:54 expanded image. This is how we were able to expand  outpaint this left image into the right image. By  

• 01:14:04 the way, if you have noticed that the toothbrushes  are gone, why? Because I have masked them,  

• 01:14:13 therefore they are removed when the image were  being expanded. So the logic of outpainting is  

• 01:14:19 that the changing the aspect ratio of the image to  expand it and also selecting resize and fill. You  

• 01:14:28 don't type anything, you select your base model,  you select your cfg scale and that's it. With that  

• 01:14:34 way you can outpaint images in Stable Diffusion.  Here another example and let's say you want to  

• 01:14:40 continue outpainting. How can you do easily.  Drag and drop the image here and then change the  

• 01:14:48 resolution and the aspect ratio. So for example,  let's make it as 1024 height and keep the width  

• 01:14:58 same. So with this way you can continue doing  outpainting very easily. And here we got a decent  

• 01:15:06 example result when we further expanded, further  outpainted the image. Because I have used very  

• 01:15:15 high resolution. You should try to increase width  and height with small increments. Otherwise, you  

• 01:15:22 will have hard time to get decent images. So try  to increase width and height in small increases.  

• 01:15:30 For example, then I can drag and drop this one  here and I can increase like 920. So with this  

• 01:15:39 way you can continue outpainting and expand your  image. When you continue outpainting your image  

• 01:15:45 like this. It is being similar to the mid-journey  new zoom out feature that has just arrived this  

• 01:15:53 week. So you see ControlNet is getting better and  better. Competing with the millions of dollars  

• 01:16:00 having companies. So we should keep supporting the  ControlNet developers. So the tiling: this is a  

• 01:16:07 new feature of ControlNet and it is amazing. When  you use tiling, it will keep the original image as  

• 01:16:15 much as possible and it will also apply the other  rules that you are doing. This is super useful for  

• 01:16:21 upscaling images. For example, they show that they  turned this image into this image with upscaling,  

• 01:16:28 it keeps the original image. It keeps the shape  of the original image and yet it upscales it. It  

• 01:16:34 adds it a full details, new details. This tiling  can be used for so many things. They have shown  

• 01:16:42 an example here. For example, with tiling, they  turned this very less detailed image on here you  

• 01:16:50 see into a very detailed image like this. Tiling  is a very powerful feature of the ControlNet.  

• 01:16:57 From this image to this image you see. It adds  so many details from this image to this image.  

• 01:17:05 Can you see: this is the original image and this  is the tiled upscaled image. It is amazing and I  

• 01:17:13 also just demonstrated how to use it to you. You  can use tiling with many stuff. Actually you can  

• 01:17:19 use it to stylize our original image. Let me show  you that. Let's refresh the page. Let's go to the  

• 01:17:26 image to image. I will upload my own image. Let's  add this one. Okay, I just typed a simple prompt,  

• 01:17:33 a handsome man. Then I will select my image from  here. Enable, pixel perfect, Tile and let's make  

• 01:17:41 denoise 1 because I am using Tile and let's hit  generate. So now I get a stylized image of myself  

• 01:17:48 like this. It is not very like me, but let's also  try to combine it with another ControlNet model  

• 01:17:56 such as Canny. Let's make the Canny weight as 50  percent and try again. Okay now it is more like me  

• 01:18:04 from this to this. Actually we can make the Canny  weight 1 and try again. So with tiling it will  

• 01:18:12 keep the original image as much as possible. Okay  now you see you see without training and without  

• 01:18:18 anything it almost generated a stylized image  of myself. Tiling will keep the original image  

• 01:18:25 and with also Canny we keep the shape and we can  get like this. We can add some more details. Okay  

• 01:18:31 let's try. Awesome intricate cgi rendered man.  Okay, now you see I am rendered as cgi. Let's try  

• 01:18:38 a picture of a man in disney a style and let's  try. It is also keeping the background image,  

• 01:18:46 background objects. Okay, like this. Not very  good. Maybe try as pixar anime style. I wonder  

• 01:18:53 what we will get. So you see, this is how you can  utilize multi-ControlNet styling, multi-ControlNet  

• 01:19:00 tiling. Let's try this: cgi 3d rendered portrait  photo of a man. Okay, not very good. Let's try  

• 01:19:06 like this. It will totally depend on your input.  Let's try a simple thing: photo of a man. So it  

• 01:19:12 will be in the style of the DreamShaper. So  this is it. Maybe we should add some styling.  

• 01:19:17 okay here. A man sitting on an armchair in  gta style. Maybe we should add gta 5. Okay,  

• 01:19:24 now you see from this to this, perhaps we can  make it obey our prompt more. Let's say my prompt  

• 01:19:32 is more important with the Canny. Wow. Nice from  this to this. So you can play with these values.  

• 01:19:40 Perhaps you can reduce the weights of the Canny  and other things. For example, let's make this 50.  

• 01:19:47 It will add more details. Let's make the control  weight 75 percent and try again. But this is fully  

• 01:19:53 stylized image. Oh yes, there's another stylized  image. Perhaps. Let's try like this. Let's make  

• 01:20:01 this like 90 percent and try again. Let's also  add our eyeglasses. A man with eyeglasses. Okay,  

• 01:20:11 like this. By the way none of these are  cherry-picked okay looking pretty well. Yeah,  

• 01:20:17 so you can turn yourself into any style. Perhaps  let's make this one. Let's make this as 200 and  

• 01:20:24 200 and try like this. Okay, nice, very nice.  You can also reduce the tiling power. Let's make  

• 01:20:32 this as 80 percent and try again. And this is the  output we get. So if you reduce the tiling power,  

• 01:20:39 you will get away from your original image. Let's  try with 1.2 tiling power tiling weight. Okay,  

• 01:20:48 it is now more like me I think. So you can play  with this and see what you can get. Okay, so we  

• 01:20:55 have got two options left, which is Reference only  the first one. So the Reference only is similar  

• 01:21:02 to in-paint based Reference. But it does not make  your image disordered. So this is the description  

• 01:21:08 that the developers has given for Reference only.  Let's test it out. I will refresh the page. A man  

• 01:21:15 sitting in arm chairs. Let's upload our Reference  image here and let's select the Reference. It  

• 01:21:24 selected reference only. Okay, let's try. This is  the first time I am using this. Actually, I never  

• 01:21:30 used it. Okay, it is not very like me. I wonder  how we can improve it. Okay, so this is used in  

• 01:21:37 yes, this is used in text to image. I think we  need a realistic model for this to work. I wonder  

• 01:21:45 what else we can use? Yes, Reference only. Let's  try one. Okay, nothing like let's try zero ten.  

• 01:21:54 Let's try with a realistic model like realistic  vision version two. Okay, I make the prompt photo  

• 01:22:01 of a man. Wow. Now we get somewhat similar thing.  Let's try with style fidelity unbalanced to zero.  

• 01:22:09 I wonder what we will get. Okay, this is with  zero. Let's make it as five percent. Okay like  

• 01:22:16 this. Let's change the prompt like this. Okay,  I don't know. Let's try again. Wow. It tries to  

• 01:22:22 make something similar but the results are not  very good. Okay with Reference only. So far we  

• 01:22:28 didn't get anything good, but you understand the  logic, you can try it. So this is the description  

• 01:22:34 of the developer. The last remaining model  is style transfer T2IA adapter models. It is  

• 01:22:42 downloaded from this url. I also have updated the  GitHub readme file style transfer models T2IA. In  

• 01:22:50 the readme file you will find this link. So from  here you need to download the models that you want  

• 01:22:56 to use. Let's try style transfer model which is  this one co-adapter style transfer and also let's  

• 01:23:07 try this T2IA adapter style. In the description  it says add style adapter and in the other one  

• 01:23:14 it doesn't say. So these two are style adapters.  Okay, it says that co-adapter is not implemented  

• 01:23:21 yet so it won't be useful. So the style adapter  is used like this source image, input and output.  

• 01:23:28 Okay, let's give it a try. So I cut the copied  model, paste it into extensions, ControlNet,  

• 01:23:35 models. Yes, then let's refresh our page. Okay now  I will show you how you can use style adapter. Let  

• 01:23:44 me refresh. Go to text to image, type your prompt.  A mighty dragon like this. Select ControlNet,  

• 01:23:51 upload your style image. I want to generate a  dragon with this style. Enable. Select t2 from  

• 01:23:59 here. The model is selected as T2IA adapter style.  Select your preprocessor as style clip vision and  

• 01:24:07 then select your desired width and height. For  example, let's make it like this: 1024, 768 and  

• 01:24:15 this is the output we got. It is similar to the  style, not perfect of course. And if you wonder  

• 01:24:22 what would happen without selecting style adapter,  let me show you. So this is another image. Let's  

• 01:24:28 try with high resolution fix. Okay we got so many  repeating maybe try like this. Okay so this is the  

• 01:24:35 image. Let's also make the seed same and when I  disable this style transfer, style clip vision,  

• 01:24:42 let's see what result we are going to get. You see  we get completely different thing not even related  

• 01:24:49 to this style. Let's enable again and with style  transfer we get somewhat similar style. I think  

• 01:24:56 it is working decent. If you prefer you can also  download color transfer. It is working like this.  

• 01:25:02 This is the color source, this is input and this  is the output. It tries to match the colors of the  

• 01:25:08 picture and in style it tries to match the style  of the source. You see it didn't use any image as  

• 01:25:14 a input in the color, we use an image as an input.  So the other says that if you have 4 gigabyte vram  

• 01:25:22 having gpu and if you enable low vram, you can  go up to 768 by 832 pixels resolution when using  

• 01:25:32 ControlNet. This is all for today. Please join  our Discord channel. When you click this link,  

• 01:25:37 you will get to this page. Join to our server.  You can follow me from twitter. When you click  

• 01:25:43 this link, you will see my twitter account. This  is our youtube channel as you already know. If  

• 01:25:49 you support me on Patreon, I would appreciate that  very much. When you click this link, you will get  

• 01:25:54 to this page. Your Patreon support is tremendously  important for me. Currently we have over 200  

• 01:26:01 supporters. I appreciate them very much. I am  posting a lot of special content here. Actually,  

• 01:26:07 these are content that I show in my videos,  but instead of writing the code yourself, you  

• 01:26:14 are able to download the code from here directly.  So it will make your life easier and it will help  

• 01:26:21 me to continue producing more higher quality  content. It will allow me to do more research,  

• 01:26:26 more work for you. Also, please, like subscribe,  leave a comment. Tell me your ideas, tell me your  

• 01:26:34 opinions. If you also support me by joining on  Youtube, I would appreciate that very much. On  

• 01:26:39 our Youtube page you will find our Patreon Discord  and my linkedin link. In the description of this  

• 01:26:46 video, you will find the GitHub readme file  link like this and also in the pinned comment  

• 01:26:52 you will find the GitHub readme file like this.  If you don't have a good gpu, then you can use  

• 01:26:57 this software on RunPod as well. Everything is  same. Let me show you how to use this on RunPod.  

• 01:27:04 To use the ControlNet on RunPod. Let's go to our  RunPod. By clicking this link, you will get RunPod  

• 01:27:10 page. You can either sign up if you don't have an  account or login. Let's login. Okay, I have logged  

• 01:27:16 in my account, community cloud. Let's select a  pod like RTX3090. Look for Stable Diffusion web  

• 01:27:24 automatic. This is important. This template comes  with ControlNet. Continue, deploy. If you don't  

• 01:27:30 know how to use RunPod, I have an excellent two  tutorials. Watch both of them. You see the links  

• 01:27:36 are here in the end of the readme file. So the  logic is saying you just need to download the  

• 01:27:42 model files. You can download them one by one or  you can use my automatic script which is posted  

• 01:27:48 here. So let's download our automatic downloader  script first in the attached cdnRunPod.py file.  

• 01:27:55 Let's connect to jupyter lab. The models will be  downloaded into the same folder. Let's open the  

• 01:28:01 models folder. For example let's download this  model file. Right click this arrow, copy link  

• 01:28:07 address. Enter inside the extensions. ControlNet,  models ControlNet models, open a new terminal  

• 01:28:15 inside. Here you see currently I am inside models.  wget Paste the link and it will download the model  

• 01:28:21 into models folder. Alternatively, you can upload  my script to here, open a new terminal and copy  

• 01:28:31 the name of the file. You can right click,  rename, select all copy type python, the name  

• 01:28:37 of the file hit enter. It will skip installation  of the ControlNet. It will then update it to the  

• 01:28:44 latest version with git pull. Then it will start  downloading all of the model files. If one of the  

• 01:28:50 model file exists then it will skip it. If not  exist, it will download it. So this is how you  

• 01:28:56 use ControlNet models on RunPod. Then just click  connect. Click connect to 3000 port. You see  

• 01:29:02 it comes with ControlNet extension. Click here.  Refresh the models. As the models get downloaded,  

• 01:29:08 they will appear here. You just need to click  refresh. The rest is same. The usage is same  

• 01:29:14 as I have shown in this video. Thank you very  much for watching. Hopefully see you later.