AI Image Editing - Stable Diffusion Inpainting

what is going on guys welcome back in this video today we're going to learn how to edit images using stable diffusion or to be specific using the in-painting feature of stable diffusion we're going to learn how to add new things to the image we're going to learn how to remove things from the image or how to change certain aspects of the image entirely using AI so let us get right into it [Music] alright so the first thing we want to do is we want to get stable diffusion running on our system and the simplest way to do that is by using the stable diffusion web UI for this you will find a link in the description down below to just get up repository and all you have to do here basically is scroll down to the installation section and then follow the four instructions here for Windows order for instructions here for Linux now in my case I was not able to just run this command here so the first step is to install the dependencies on Linux and the second one is to run this command for some reason it didn't work for me I had to download the repository and then to run the web UI shell script file but maybe it works for you out of the box for Windows I actually already have a video on this channel where I show you how to set it up and on Linux as I said it's not too complicated you either just install the dependencies and run the command or you install the dependencies and you download the repository and then you run the web UI shell script so in my case I already have this installed or downloaded so I'm going to open up my terminal here and I'm going to run the web UI shell script and this will then open up stable diffusion here on localhost so I can click on it it opens up stable diffusion and what we're interested here in particular is not stable diffusion in general not the image generation with the text to image feature not the basic image to image but the in painting feature from image to image so in painting basically means we take an image we select certain areas and then we change them we add something to the image we remove something from the image very similar to the new Photoshop feature that I saw in the video recently now I myself am not using Photoshop so I don't really know what I'm talking about here but I saw that photoshop has this new feature where you can use generative AI to add stuff to the image to change stuff in the image to remove something from the image and so on and this is very similar to that but it's free and open source so to do proper in painting we don't want to use the basic stable diffusion checkpoint we want to use the in painting checkpoints so we want to go to hugging phase is to get this sdv15 in painting checkpoint to this you will also find a link in the description down below you just open it and you download it where is it this is the download button here we're gonna save this to the desktop and this has four gigabytes so it will take some time here while it is downloading I'm going to explain basically uh what we're going to do we're going to get some random images now we're going to get them from pixabay for example uh and we're going to change things about them so for example I'm going to say person standing or something and then we're going to just pick any of the people here that are standing some image that looks uh not too complicated to edit maybe this guy here actually I use this image when preparing the video so we're going to just download a basic version of that now stable diffusion works better on smaller resolutions so you can go with whatever resolution you want but I'm going to go with the smallest one here because that is going to give us probably the best results so we're going to save this on a desktop man.jpg and maybe we also want to have something like Doc at the beach or something if we if we can find this so this is sponsored this is royalty free so maybe we can take um what are we going to take maybe this one here also small resolution doc.jpg and we're going to play around with those two I'm not sure if they're going to work or not but we're going to try to change third certain things about them so for example for the man what we're going to do is we're going to try to replace this black leather jacket with a red jacket for example or a golden jacket or we're going to replace the shirt with another color with another type of clothing the shoes maybe and what's a little bit more difficult and I'm not sure if this is going to work because it usually doesn't work when I try it is we can try to add something into the hand so something that doesn't exist at all here something like a knife for example usually this doesn't work or you have to try a couple of times until it works but we're going to try it and for the dock maybe we can add some item here maybe we can change the water to sand so it's a dock in a desert or something we will play around with that as well now the checkpoint is downloaded what we're going to do now is we're going to take this checkpoint uh we're going to take it and we're going to into stable diffusion and you want to going to your models but for some reason I think in my case uh it created stable diffusion inside of stable diffusion so actually the directory that I have to go in is I have to go into the stable diffusion web UI inside of the stable diffusion web UI and here I have to go into models whatever just go to models go to uh stable diffusion and just paste your in painting checkpoint here so this is the painting checkpoint now in stable diffusion you can now click this reload button and you should be able to select the checkpoint which is optimized for in painting meaning that this is a checkpoint that is powerful when it comes to in painting while this is loading here what we're going to do is we're going to load an image of the man for example and we're going to provide a prompt so for example uh man with golden jacket maybe also just golden jacket something like this um and then what we're going to say here is a negative prompt as something that the image should not be we're going to say that this is uh a black leather jacket or just a black jacket um so right now we didn't select anything for in painting so what we're going to do is we're going to adjust the brush size and we're going to select the areas of the image that we want to change and I'm just going to draw here roughly over the leather jacket part I'm going to fill this with a mask I'm going to mask all of this here I'm going to do the same thing here on the left side and you don't have to be super precise with that you can also just um paint over stuff that's not necessarily the jacket so if I also paint over this year it won't cost too many issues the important thing is if you don't paint over something it will stay the same so if you forget to paint over pixels of the black jacket the black jacket will not be replaced entirely because you can only replace what you paint over so this is now the jacket selected and now we have some settings down here so the sampling steps are basically how many iterations how many steps you want to do while sampling I usually set this to something between 60 and 80. let's go with 80 now or let's go with 70 maybe the width and height is important because you want to keep the ratio usually so in this case the height is greater than the width and you can actually see this ratio when adjusting you can see it at the top the red box here you can see when the height fits or when the ratio fits so we can also adjust the width so we can go for a lower resolution there you go so this is now the perfect resolution and the other parameters here are CFG scale I don't know exactly what CFG stands for actually it tells us here classifier free guidance scale basically the lower this number is the more creative the model is going to be the larger the number is the more it's going to focus exactly on your prompt and I think it actually also says that here so lower values produce more creative results I usually leave this at seven and then we have the denoising strength uh the more the higher this number is the more the closer it is to one the more it will replace the image that's currently here and the smaller the number is the more it's going to focus on what's already there and I think this is also saying that here at zero nothing will change at one you will get an unrelated image so everything in between is orienting itself more to something new and unrelated or to what is already there that's the basic idea so let's just give it a try the important thing is that you set this to fill when you're trying to replace something completely um and then we're going to generate and see what happens now just a little bit slower than usual because I'm recording and you can already see this one doesn't look too good this doesn't look like it worked uh and we can try now a couple of times because sometimes it's just not going to work the first try sometimes it's going to give you the funny result so in this case for example it gives me a blue jacket um so not exactly what I'm looking for we can try a couple of times and once you find something that you like you can also copy the seed and you can try to change it slightly so let's just repeat this a couple of times until we get something like gold golden maybe we can also do yellow and here we can say black dark and now it adds some more gold to it but it's still a blue jacket fundamentally maybe we can increase this to like 0.9 to be less related to what's already there and you can see hopefully this is not influencing the recording too much probably it is let's see so I'm not sure if you hear everything I say while this is uh generating because it seems like my recording is lagging while I'm generating this uh so I'm going to try to only speak when it's not generating but the idea is that the higher you set this denoising strength the less it will focus on what's Already There usually so we can also try to set this to one and it will try to replace it completely you can also increase this to make it focus more on what you're writing and you have to play around with this so even if you have the perfect settings you're not going to get the best results every time sometimes you have to try 50 times to get a good replacement so we can also just skip if you see that this is not what you're looking for you can also just skip immediately and you can do this a couple of times here now if this does not work with the next couple of tries I'm going to actually do something here I'm going to all right so let's try to adjust some settings here maybe let's try to increase this a little bit maybe it helps or maybe we have to keep the prompt simpler so maybe we just have to say black or actually golden jacket black jackets maybe this is enough oh it looks better and now it's silver or blue okay let's try again oh there you go a little bit better and the good thing now is also what you can do is you can instead of working with this image here you can also just take this one and you can send to in paint to continue working on this one so for example I can now use the same prompt the same settings here on this one maybe I can also adjust this to be 0.85 and now I have something golden as a basis and it will probably make this better there you go so now you can see we have a golden jacket instead of a uh instead of a black jacket and of course we can also try to send this to inpaint to select the area and now since this is already pretty much what we're looking for we can also change the mode from fill to original and we can turn this down to something like 0.4 and maybe you want to turn this back to seven and this will then optimize the image for what we already have hopefully so it will most likely let me just work a little bit more should look a little bit better there you go looks like a golden jacket now so we can send this to in paint and now we want to change the white shirt to something else so maybe I want to mask all of this here there you go and I want to say from a white shirt change this to a red shirt for example actually the other way around this would be a white shirt and this would be red shirt and now of course we want to use fill we want to turn this up again to 0.9 or something and we want to see if this is enough oh okay now it's replacing it cannot replace the jacket though actually unless the mask is invalid yeah I think it still uses the old mask this is a buck so we need to rerun this again it still had for some reason selected the jacket so select all of this generate and you can see it works immediately we have a redshirt below the yellow or below the golden jacket and it looks kind of realistic it looks like he actually wears a red shirt and of course you can also adjust this you can play around with this if you want to use the exact same seat you can use this button here to reuse the seat and then you can change the settings slightly if you just want to have slightly different results um and now we can do something else like we can send this to end paint again we can then select the shoes this is the last thing I'm going to do with this guy here we can say I want to have white shoes instead of gray shoes and I think this will be a little bit more difficult but let's see uh there you go okay now he has oh he has white shoes works first try it's not more difficult there you go so now we have a guy with a golden jacket a red shirt and white shoes uh and of course this is now not the most high quality result that you can get this was a very first try approach if you want to have realistically looking perfect results you can just try again and again and again you can reuse the image you can go into original mode with a denoising strength of 0.2 or 0.15 to just slightly adjust the image and stuff like that this is very powerful but this already looks quite good if you think about what we had in the beginning which is this image here so let's try the second one let's try the dock um usually it's quite difficult to spawn objects here but maybe you want to remove this one here so this should be quite simple uh maybe I want to just select this and also this line here something like that uh and I want to say something like Doc at the beach C sand and then objects yellow or maybe just yellow object maybe this is enough to remove this so in painting we want to fill we want to maybe use 0.8 or something um I may want to use zero or 60 sampling steps here yeah let's see if that works now we need to also change the resolution [Music] for some reason this is not shown correctly but let's see if it works anyways there you go it actually works it seems to remove the object and the rest of the image is the same so we can send this to impainting now again and now we can change other things about the image so for example maybe I want to spawn a treasure chest here now this is where it gets difficult I think I don't think this will work too easily but I was wrong before maybe it works first try but I don't think that this works too easily here because now I have to spawn something that's not there at all so actually I would have to go with a one here um and I think it will be quite difficult to be honest but let's see maybe I'm wrong about this so let's go with treasure chest and I'm going to just say c as a negative here maybe we get a treasure treasure chest but as you can see in the first try it doesn't work nothing spawns there uh we can try again uh usually it works randomly at some point but most of the time it doesn't work so maybe we can increase this year to just force it to be less creative and just force the The Prompt but this also doesn't work in this case um so we can try a couple more times yeah difficult um maybe we can uh set this back to seven but maybe we can set this to original here which I don't think makes a lot of sense but maybe it can produce some results no it doesn't look like it uh maybe we can give it more area to use so that it can adjust the surroundings better but if this doesn't work I can show you at least some results that I already had before making this video but let's try maybe two more times oh this actually seemed to try to spawn something but doesn't look like it's gonna work maybe we can try one more time with a different prompt like chests uh treasure wooden box and then we can say here's sea water sand that's a negative and maybe we can somehow force it uh to create something here but it doesn't look like it um before I show you what I managed to do yesterday maybe we can do it with a sketch so now we have um actually let me just send this to in painting again uh actually not this one send us to impaint sketch okay forget about it we're going to just manually open this we can go to stable diffusion and then to Output whereas outputs there's outputs image to image and uh here we can pick this image here as a base image and we can sketch a chest maybe I don't know I don't think that this will work but you know we can try here so I can just fill this with some brown color representing wood and then maybe some gold gold light colors to I don't know make it look like a chest or something I don't know we can try like this um yeah so that would be that maybe let's go with a nine here and let's see if it works with the fill mode oh there you go now it looks like something is happening at least a little bit there you go it generates an actual chest okay this looks promising let's try again let's see if this was a coincidence ah this is not too good let's try again this looks better okay so sketching seems to have um seems to yield some results here now it doesn't look perfect but what we can do now is we can send this to in paint so this this image here we can send it to impaint we can select this area here and we can then say um okay let me select this again and then say that what we have is already quite good but we want to change it so we want to go for original I want to go with something like 0.4 again here and want to keep the prompt so this could actually improve the chest oh looks pretty good now let's increase this maybe to 0.55 to have more changes to the chest so that it actually looks like it's it's actually a chest and this looks already quite promising so Center in paint again maybe do one more iteration and maybe we can change it also to say C and water as a negative but your chest in sand something like this and this now looks way more realistic looks like there's actually a chest besides the dock and we can send this now to impainting and we can do something like change the entire background we can say it's no longer a dock at the beach if I select all the water I can actually do this and now we want to say okay this is not a c uh or actually this we have to put it down here C water Ocean Beach this is dark in front or actually let's let's forget about the dog let's just say sunset Blue Sky and let's see what we get here as a result uh actually what are the settings where let's go with fill and let's go with 0.9 for this one and now what we get is something that doesn't look too I think it's still selecting the chest I have the same problem as before and it sucks alright so now the chest should no longer be selected so let's go and try to generate this doesn't work first try okay the problem is since I'm so close to the dock it changes the dock and it changes the chest so maybe uh let's try to just say 0.6 doesn't seem to work too well let's go with blue sky let's go with sea ocean water as a negative so maybe it can just change water to Sky and this would already be some progress that we can use for another in painting but it doesn't seem to work too well and maybe we can get better results with original here so maybe we can uh set the denoising strength to 0.45 or something it will change the water to Sky this is what I hope for at least no it still looks like water so let's try [Music] um 0.65 maybe which is quite a bit more okay but it doesn't seem to work too well now I'm going to show you something that did work just so you believe me that it actually worked I did something similar with a different image I'm going to show you that so this is how it started this is a dock at the beach and I added a chest to it so I had to try a couple of times as you can see so then we got this chest and then I changed the sky to a clear sky and then I changed it to a sunset a couple of times and then I change the dock to just a dark Line in the Sand uh then I change it back and yeah you can see that it works but it doesn't work every time so this here looks actually quite realistic you have a dog in the desert with a sunset uh and the original image was was this one so it does work but it doesn't work always you have to try on a couple of times you have to change the prompts uh you have to adjust the parameters you have to try multiple times with the same problems and the same parameters it is not predictable and it is AI it's not an algorithm so you will have to just try and fail and try and fail again um yeah but this is how you do in painting with stable diffusion so that's it for today's video I hope you enjoyed it and I hope you learned something if so let me know by hitting a like button and leaving a comment in the comment section down below and of course don't forget to subscribe to this Channel and hit the notification Bell to not miss a single future video for free other than that thank you much for watching see you in the next video and bye

Channel: NeuralNine

Views: 15,575

Keywords: stable diffusion, image editing, image inpainting, stable diffusion inpainting, generative ai, ai generation, image generation, ai image editing

Id: 1ZmvRx1tN3A

Length: 26min 58sec (1618 seconds)

Published: Sat Jul 29 2023