A1111: ADetailer Basics and Workflow Tutorial (Stable Diffusion)

hi I am Seth welcome to the channel automating the in painting process in your workflow can save you time take this image for example in one step you can have the AI detect the face and add some light makeup in the same step get her clothes and painted and change the clothing via prompt using a different check point and for the final step select the eyes and enhance it with double the resolution in fact this extension is so powerful you can take a wholly distorted image like this and restore the face all this is possible with the after detailer extension for automatic 1111 let me show you some hacks and my unique workflow using a detailer and stable diffusion I will start the tutorial with the basic installation then go on to explain some of the settings I cannot cover all the settings in one go so I will explain some of them along with the workflow examples this is the automatic 11-11 interface you need to be familiar with this interface to understand the tutorial if you are new to stable diffusion and don't know anything about this UI then you can go to the channel and check out the stable diffusion playlist to install after detailer go to extensions then open a new tab and go to this website scroll down to the install section and copy this link go back to extensions and click on install from URL here paste the link you just copied and click install wait for the installation to complete then go to installed and click on apply and restart UI when you install this extension it automatically downloads the basic models you can verify by going into the ad detailer tab these are the default models that it installed for this tutorial workflow I would use two extra models that need to be downloaded manually go go back to the web page scroll down and you will find a hugging page link click on it download the Han YOLO V8s and the Deep fashion 2.pt files take note of the location you should save the two files in this folder after saving restart stable diffusion from the command prompt before I proceed further you should probably change some settings for after detailer over here increase the max models to three also make sure the save mask previews and images before a detailer are ticked the save mask preview saves images showcasing what the model has detected and masked whereas the save images before a detailer will save your original generated prompt image before the model applies the changes this this comes in handy for doing a comparison of before and [Music] after when you expand the after detailer tab there is an enable and disable option you can also see the first second and third tabs here similar to control net units you can have multiple detection and settings for the generation in one go for example you can have the face selected on the first hand hands on the second and the eyes for the third and you can customize each and every setting individually as per the requirement these are the models for the ad detailer YOLO stands for you only look once you have two YOLO and four media pipe models for the face in the YOLO models you will find versions ending with n and S the N version is about three times smaller but the S version is stronger and more accurate for this tutorial I am only using the S YOLO versions the face YOLO detects the whole face or multiple faces depending on the image the YOLO version works for both realistic and 2D images this model has a better detection algorithm and can also detect side faces media pipe is weak for 2D images and has difficulty detecting faces especially Small Faces it can detect large and real faces easily media pipe face uses points along the face to detect the images the face short detects faces closer to the camera on the other hand face full will detect faces further away from the camera this is what the technical explanation says however I found it inconsistent the image on the right is full and the left is short whether you select full or short the center image is detected almost always here is another example basically there are settings that you can change to select specific faces above a given threshold media pipe face mesh detects the face with a mesh algorithm I did not find much use for it but the detection model is different the eyes only model is excellent for detecting and post-processing the eyes of a realistic photo I did try it on anime and the post-processing doesn't work that well the hand YOLO is a bit tricky because it's a hit or miss postprocessing at at no point can you use the hand model to fix extra missing or deformed fingers it can correct slight deformations or just enhance a clearer hand generation however if you have a clear hand with five fingers it will just add details to it if you try to completely denoise and regenerated the AI cannot accurately blend the regenerated hand along with the subject the person YOLO models detects and imp Paints the whole or multiple subjects it works for both 2D and realistic images the Deep fashion YOLO is for clothing and works well for realistic clothes I did test it on an anime image and it was able to detect it correctly these are the support IDs for each style of clothing it automatically detects and Associates the ID when saving the image for reference you can input positive and negative prompts over and above the main prompt if if you leave it blank it will only use the main prompt I will explain this in detail with a workflow example later in the tutorial when the image is generated the AI analyzes the image and detects as per whatever model you have chosen in this example you can see the values it attaches to each face the more confident the model is the higher the value confidence here implies the ability of the AI to correctly and accurately render details or corrections to the detection in this image if you want to make changes only to the center and right face you can increase the threshold to 0.86 the model will then apply the details or changes as defined to anything above the value of 0.86 only I will later explain the mask Min and Max area ratio during a workflow Example The Mask X and Y offset moves the Imp painting mask along the X and y-coordinates this won't show in the save preview but when applying the effect it will go as per the coordinates defined why would you use this for example you generated this image look at the area of selection for the face this is a low resolution image and say you want to enhance the details using the normal selection you get this image however if I change the X and Y coordinates to POS 50 the m asking area will move towards the right and up by 50 pixels this will improve the details on the subject's left ear as well this is done using the same face model twice only the second setting will have different X and y-coordinates the mask erosion and dilation slider is set at four as a default if you increase this value it expands the masking area however in doing so the center of the Mask makes the image blurry reducing ing the slider shrinks the masking area I did not find any use of these three settings for my workflow examples this is because I used another method to achieve what I wanted more accurately which I will showcase later in the tutorial unless you have very specific needs just keep these settings to default The Mask merge takes multiple detections and merges the in painting before applying again I found no use for this setting however the merge and invert option is unique and we'll show you how this option works in a workflow after detailer works by blending the mask with the surroundings through blurring the inpaint masked blur slider does precisely that increasing it makes the blur wider I strongly suggest keeping this value at default the denoising strength tells the AI how much to change the image in the massed area a lower value means the image will be close to the original and a higher value would result in a very different image I do show how to use this option in the workflow correctly it has a bit of a learning curve keep this option ticked otherwise it will imp paint and regenerate the whole image and may give you undesired results this option defines the padding on the mask if you want to increase or decrease the mask padding in all directions change this value accordingly by default the model will use a predefined resolution to generate the massed area and resize it to the resolution of the output this setting has to be used carefully when you enable this the AI will use a separate resolution as Define and resize it this does not mean that using this setting at a higher resolution will always result in sharper images after exper experimenting I found out that this does not work for the faces at all and in fact makes them blurry however there are some instances where you can use this and it gives fantastic results also using a higher resolution will take up a lot of vram and more time for the generation this setting is completely GPU intensive enable and Define these settings if you want the steps and CFG scale to differ from the original you can use a separate checkpoint matching VA AE and a different sampler for the mask regeneration I found these settings extremely useful and you can get very creative with this I will show you some workflow examples on how and when to use this correctly keep this option disabled as using the restore faces function after applying a detailer makes no sense after detailer does a much better job than restore faces there is support for in paint scribble line art open pose and tile control net models for both SD 1.5 and sdxl when using this option make sure that the main control nit is not enabled moving on to the first workflow I am using a newly released checkpoint model called think diffusion XL I will be doing the after detailer settings one step at a time so that it's it's easy for you to understand what happens at each stage you can do everything at once in one step that is the main point of using after detailer to start let me generate a basic image even though this model is one of the best realism models on civid AI it produces lowquality faces when the subject is zoomed out in such cases after detailer works best I am selecting the pholo model and will generate the image without changing anything else okay I am going to interrupt the generation process because it has detected too many faces it did save this image so let me bring it up what I want to do here is to correct the center face only so I need to change the confidence threshold to 0.82 or higher this way the model will select only the center face and add the details to it perfect let's work on the [Music] hands I am selecting the hand YOLO as the second model 4A detailer without changing any settings let's let's hit generate and see what happens let me bring up this image and show you a crucial limitation you can see here that there is a hand in the background out of focus but more prominent than the subject's left hand the model has put a confidence value of the left hand to 0.35 this means that by changing the confidence threshold there is no possible way to to select only the right and the left hand neither is this possible by any other setting given in a detailer you might not face this situation but I got it the first time I tried it so in this case I will let it continue and add details to all three hands since the second hand is out of focus and in the background I use the same checkpoint model to generate the new hands even at the default setting they look a bit fake what I will do here is use a separate checkpoint model called realistic Vision only for the hands and see if it produces better [Music] results also am increasing the Deno strength to 0.42 this completely spoiled the hands the mistake was that I did not input any negative or positive prompt for this checkpoint if you check out this checkpoint on civit AI this is its standard negative prompt I just copied and pasted the same here for the positive prompt I will just put [Music] hand the hands are now much better this is one way you use multiple checkpoints in after detailer you can also add a prompt in the first model for example 60-year-old American male and generate the image without changing any other setting to recap I use one checkpoint to generate the base image then I used the face YOLO with a prompt to make the model older for the second model I used the hand YOLO and another checkpoint to regenerate and fix the hands this entire process is done in one generation let's move on to the next workflow I will be using realistic vision for this work workflow let's use a high resolution fix this time and upscale the Generation by 2X this is the base image I will try adding light makeup using the face YOLO and a positive prompt The Prompt is light makeup Coral pink lipstick remember there is already a main prompt adding a positive prompt in a detailer will only reflect the inpainted mask without changing the entire image nice now I will use the Deep fashion YOLO to change the clothing style this is one instance where you can use the denoising strength value at one when you do that the model replaces the mask with a newly generated image and we want that because we are not adding to the clothes we want to change the clothes also I am using a separate checkpoint and sampler when I do this what happens is that the Regeneration of the clothes sometimes causes some blending issues to avoid this it's best to use control net here for this instance the tile control net model works [Music] best for the last step I use a third model to fix and enhance the resolution here I saw better results when I used a separate higher resolution only for the eyes I did try it on the face and it doesn't work well it makes the face blurry this may be checkpoint dependent you might want to experiment with the face using different checkpoints I tried a couple of different ones and it didn't not work for me so this is the final image but let me zoom in on the eyes so you can see the difference the bottom image is with the standard resolution and the top is with the 2048 resolution I will show you another workflow quickly using Everything at Once in one generation I am using the than diffusion XL checkpoint to generate the base image for the first after detailer model I am using the face YOLO and the realistic Vision checkpoint to change the face completely and after detailer did an excellent job blending it for the second model I am using the design YOLO to alter the clothes [Music] [Music] finally for the third model I changed the eye color using a ProMed and enhanced the [Music] resolution [Music] [Music] one reason why you might want to use multiple checkpoints is purely for Aesthetics in my example the base image produced a beautiful bouquet and hair however the other checkpoint enhanced the face the eyes and the clothing this is the last text to image workflow using after detailer for this workflow I am using the law mysterious [Music] checkpoint for this anime image I want to enhance the entire image without touching the clothes to achieve this I would use the person and the face YOLO models when you select the person model this is how it masks the subjects the thing to note here is that if you apply a detailer using the person YOLO The Mask area becomes blurry after regeneration you can solve this by increasing the resolution to 248 and in painting when you do that the face and the clothes in this example get much more depth and the Shadows become too dark overall the image becomes too contrasty which does not blend well with the flat anime look this is why I am using the person YOLO to select the two subjects and at the same time using the merge and invert option so it add adds details to the entire image except the face and clothes the second model is the face YOLO and in the prompt I will just add the text smile let me show you a comparison so you can understand the [Music] [Music] difference [Music] the image to image function with the ad detailer does give random results the process is simple after you generate the image to image first copy the seed using the same seed try and apply your after detailer settings to get the desired result if you don't then rinse and repeat this image is ridiculous but after about 8 to nine tries I could fix the face to an extent the purpose of doing this was to test how robust the model can be if you know how to use [Music] it for image to image always use the denoising strength from 0.1 to a Max of 0.3 otherwise the image will change drastically the problem here is the model detected both faces since I want to change the smaller face not the big painting behind the confidence threshold won't work here also the smaller face is not correctly regenerated this is because the area it has to redraw the image for the smaller face is too big since the AI does not have anything to blend the Regeneration with you will always get an unblended regeneration to have a lower confidence threshold selected we should change the mask m and Max area ratio the value of one for the max ratio means 100% of the pixels so we have to do some guesswork here the model should actually have a confidence threshold Min and max value however it does not therefore we have to estimate the percentage of the smaller face pixels and ratio to the entire image a value of 0.01 would mean it is 1% of the whole image so you start with 1% and increase the value till it selects and imp paints your subject fortunately for my image 0.01 worked and I did not have to keep increasing it when I selected a lower percentage of the masking area it Blended the image very well I hope you learned something new today in automatic 1111 until next time [Music]

Channel: Control+Alt+AI

Views: 10,092

Keywords: adetailer, after detailer, Stable diffusion, a1111, automatic1111, adetailer extension, automatic 1111, a1111 adetailer, stable diffusion adetailer extension, stable diffusion extensions, inpainting, restore faces, fix hands, stable diffusion tutorial, a1111 tutorial, adetailer stable diffusion, stable diffusion adetailer anime face, adetailer tutorial, adetailer img2img, adetailer controlnet, stable diffusion tips and tricks, after detailer stable diffusion

Id: Fef5ia8ly0o

Length: 30min 17sec (1817 seconds)

Published: Fri Oct 27 2023