Mastering Image Manipulation in Stable Diffusion | Inpaint Anything & Control Net Techniques

foreign [Music] welcome to Alchemy of zero phase my name is Eric and in this video I want to walk through my workflow I did a couple videos short videos a little while ago where I had a person and I showed how you can just change the scene and the clothing and keeping the same person I kind of synced it with some music anyways a lot of people love that and uh uh one person was even like uh what do you say bro do a tutorial and so uh yeah anyway we're going to walk through doing uh something similar to that and I hope you enjoy this I hope you learned something from it uh so what we're going to do is uh select that we just kind of get things set up here um what we want to do we'll do something similar we're going to just do a scene with you know uh maybe somebody modeling or something like that so let's come over here let's grab a uh let's create a prompt so we're going to do let's just do you know same thing here digital illustration that's fine what we're going to do is a beautiful woman stinged centered on a city street trending get a couple of prompts here for those of you who are just seeing this for the first time this is my prompt generator that I have developed to use chat GPT to develop to generate amazing prompts it's very intuitive very flexible this is this is uh a thousand times more advanced than than the ones you see on Huggy face or the spreadsheets that people are the really cumbersome spreadsheets that people have tried to use you can get it on my store go to shop.zero phase.com it's in a link to below but yeah so let's say captivating digital illustration portrays and elegant home employees and kind of standing in the heart of a bustling City street okay so we're just gonna see if we can get an image from that if we want to picks a stunning moment as the focal point yeah we'll let you know let's grab that one I like that one better let's run that over here we're going to throw that in and we're going to do portraits on that one as a negative prompt but just a few minor adjustments that in the prime never negative prompt that it does so we're going to set this up as kind of a portrait uh Dimensions so aspect ratio 4x5 we're going to generate a few images here just to see if we can get one that fits what we want okay I am going to up the scale on this little this model does pretty good when you're dealing with higher resolutions so we're just going to see what that gives us here the RPG artist tool model you can get that on civit AI it's an amazing model and uh just like especially for doing like fantasy and and uh illustrations and and you can do photo realistic stuff but it still has that slight Illustrated look to each photo um pretty cool model though so I'll let that go through it looks like we're already getting a couple images here that we'll be able to use for what we're doing so once we get an image here we'll go over to in paint anything I'll show you how to create the mask and the adjustments you need to make for this to work really effectively okay foreign a second to go through here okay so let's see what we got that one's not bad we could work with that one yeah not so much not so much that one's not bad too I think that one is what we got here that one might be easier to work with actually yeah let's try that one let's try that one okay so let's grab that we're gonna send this over to actually we're gonna have to um save this I'll give you just a second here okay and then we're gonna come over here to in paint anything and um in paint anything is a pretty fantastic extension again you can go to your extensions go to uh available look for it should show up as uh let's see where is it in paint so sdwebui in paint anything the link is right here for the GitHub but I'm pretty sure this one is available in the uh the repository if not uh just grab you know I'll link the extension down below um and you just grab that and do install from URL okay a couple things to note uh when you get this installed you'll need to download the model so there are several models here depending on what you want to do the defaults on this one you just hit download model give it a few minutes It'll download the model depending on your internet connection it may take a little while okay um and then once you get those downloaded we can start moving forward here we're going to grab this image drop it in here okay all you do is hit run segment anything oh they got some new options here I just updated a little bit ago so scale width Left Right balance I'll have to look at these little layer I'm just going to leave them on default we're going to run segment now what it's going to do is go through the image and identify individual separate objects and items just takes an initial like when you first use it just takes an initial minute to kind of load everything up so as you can see we have uh like a lot of the buildings identified we have her and uh we can actually do some pretty cool stuff with us so uh here's something you need to know you can see the little cursor moving around right there let's uh let's increase its size just a little so you can see it a little bit better okay and what this does in this particular screen all this cursor does is you want to Mark the areas that you want to mask I usually like to keep them the cursor small because you don't need to mask it out like draw over everything which you can do but the way this works is if you mark in an area once like that that entire area is flagged to be masked okay and so what we're going to do in this video is we're just going to be changing clothing okay we want to put her in some different clothing and and shift what she looks like so we're going to mask that out we're going to mask the belt here now if you if you mask an area that crosses both colors it'll flag both those areas so we're just going to mask this all out we're just going to flag that um it does look like there's slight a slight visibility of the skirt behind her legs there uh let's just play with it and see what it does we're going to mask that okay let's shrink this down let's see if we can grab that area just right behind our lives there let's just see what that looks so once you have the ones flagged that you want flagged then just hit create mask come down here and this will be where you are going to be able to figure out if you need to make adjustments if you masked too much sometimes it'll flag other areas you get outlines around things you didn't want so it looks like there is a green here that I'm not seeing well no I see it okay it's just really faint so we're gonna flag those two areas and then hit uh create mask one more time foreign there we go and it looks like it's padding it more so than it used to so they may have fixed that we used to do have to do like expand mask region in order to uh get it to cover everything that it needs to cover so it looks like they by default are are adding a little bit of a buffer uh to which it needs um what we're going to do though in this area down here where it shows the mask it's hard to tell but it looks like it's masking out other areas that we don't want so let's increase this so that'll make it yet see how it's creating a mask around areas we don't want so this is where this area is where we come over here we select our masking tool and we don't want the hands masked so we're going to block that out make sure that's the same there we're gonna mask out this here and all we're doing is we're drawing the areas we do not want it to do anything with okay including up here it looks like okay once you get that selected then you just do trim mask by sketch and so with all the areas you just mask it'll just get rid of the uh the whites in that area okay okay so now we have the image and uh we can do a few different things with this if we're just going to be changing the clothes we'll leave it like this I'm going to expand the mask one more time on that so we have plenty of buffers covering all the different areas we want okay and then once we have this now we go uh Let's see we want the mask only okay and then we're gonna go get mask as Alpha of image see that's not what we want let's see get mask there it is that's what we wanted this is a new feature I'm not familiar with that yet that's Alpha of image so it'll keep that and then you can take that into other areas so we don't want that but let's see what it is so now we would just send this over to image image and paint okay so it sends the original image over sends the mask over now what we're going to do we need to come up with some different scenes okay so all we're going to be doing is changing the clothing here okay so I'm going to come back over to my stable diffusion prompt generator and what I want to do is I only want to modify the second one here or we could take this and say create five more prompts of this but what I want to do let's try this here using the second prompt only five prompts 30 words each five different specific styles of uh jackets and see jackets and short skirts because that's all we have masked out okay trending let's see what does with that okay perfect yeah so grab that list to read through these just make sure we get a digital illustration Showcases of fashionable one wearing a Sleek leather jacket paired with short plaid shirt uh exuding in edgy and rebellious five okay uh artwork cheek woman Don's a tailored Blazer short denim skirt okay shorter pleated skirt vibrant bomb or jacket this is gonna be cool so now we're just gonna copy that whole thing we got five prompts here that change the style of the clothing she's wearing okay again if you don't have this prompt under here this is this is a game changer for this anyway we're gonna come over here we're not gonna put it in here we're going to select our negative prompt um portraits again not that it should make too much of it you use a prime negative get that I got my style sheet out on the Google on my Google share I'll link that too so you have access to the Styles I'm using the negative prompts I'm using um we're going to do only only masked okay and then actually you know we'll do whole picture I think that's all we need on this okay we want to make sure we have the uh width and height correct I think this was uh 960 if I'm not mistaken let's go take a quick look at that and see what it was 960 by 768 okay just gonna make sure that's the same we're gonna switch this over to yeah we're gonna leave them Euler a it's fine okay leave that we are going to uh leave it on 0.75 because we're asking for some pretty significant changes actually no we're going to drop that down 0.57 that should give us the changes we want but leaving the overall aesthetic okay now we're going to come down here to the extension or script so we want um prompts from file or text box okay and we're going to paste all those prompts we just created in here just uh just exactly as they are okay and that's it now what we're going to do is generate five more images based around this mask our descriptions let's see how see what it does in that model well you know what we do need to do I apologize we need to switch this over to an in painting model or else it's just going to look funny so let's go to the v3m painting model for the same checkpoint uh this one does not have vae baked in so I I want to make sure the colors are correct so we're going to make sure we select the vae Ft MSE 84 or 840 000. I believe you can get these vaes uh either on the Civic AI page for that checkpoint or on hugging face okay now that we got the settings all set right we've got the model and the VA vae checkpoint or uh vae set in we'll go ahead and render this again and see what it does okay should be pretty quick because all we're doing is we're just changing what's there for clothing all right let's bring these up so there's the first one on the Plaid skirt leather jacket and we got the denim skirt with the tweed jacket and as you can see it goes through and we get we can have an amazing variety you can stick as many prompts as you want in that box down here um and have it render out an image for each one or you can put those five in there you can change the batch size or batch count and render out a whole ton more as well so very cool way of trying out different outfits I know there's probably been other people who do tutorials on this this was a real quick simple one now I know a lot of you are wondering about the um changing the entire scene so what we're going to do is go back to in paint anything and we're going to wipe out uh let's see no we're actually all we're going to do is we're going to grab and and include in The Mask um oh you know what we're going to change the entire thing so we're going to wipe this out all we're going to do is Select our face okay and let's shoot that down we're going to select our hands that's why I use a small man that's got a large screen okay and we'll select our legs too we'll keep that the same so all we're doing is keeping things that are the skin is exposed we're also we're going to let it change the hair too and then down here we're going to select invert mask create mask and we'll see what that gets us here okay so what it gave us is exactly what we're on we got our face her hands her legs everything else is going to be masked okay so we're going to go get mask over here we're going to send that over to in pain image and then what we want to do is see here we want to make sure that we're impeding the in painting Mast we're going to do the whole image leave it on the earlier a but we want to come up with some different problems let's put our ends we're going to be putting her in some different scenes okay so let's go back over to our prompt generator and we're going to do we're going to take that same one we did we're using the second prompt only five prompts 30 words each we're going to do five different specific fantasy scenes and sci-fi foreign prompts mixing fantasy and sci-fi or it might create five prompts of each I didn't say each so it'll probably just do five let's see what it does so we have uh reveals ethereal fantasy or a mythical force awaken Center shimmering Moonlight Conjuring sets of magic okay so it took out the person so what we're going to do is we're going to say uh include a beautiful woman in each front what this should do is just add a woman into each one while keeping the scene the same okay so we're going to copy that go back over to stable diffusion we're going to come down here we're going to get rid of all these paste those new ones in there and I don't think we really need oh you know we're gonna up this because we wanted to change the entire scene okay we're also going to increase the mask blur a little bit on this as we want it to incorporate as much of the well you know no because we're dealing with such slow resolution that we're gonna it might take away from her face on that uh we're on in painting got everything else set up the way it should be and let's go ahead and render it out and see what we get again this should be pretty quick we're dealing with low resolutions uh I know each person has different uh video cards but uh as you can see we're getting some pretty crazy looking scenes these are awesome oh my goodness that one's awesome I love that this allows you to try out just a ton of different styles all at the same time um giving you the ability to find inspiration honestly so there we got a beautiful nature scene a little more futuristic going on there that one's cool I don't know what's going on with that one but uh I love the angel wings which looks like she might be carrying some feathers or have a feather tail not sure again more futuristic dealing with starships that looks kind of like a movie poster a sci-fi movie poster uh there's more fantasy scene Princess magical then back to that one so very cool stuff I mean this you get a wide range of ideas you can do with this um let's try let's try one more thing here let's go back to this uh and let's say add a second person a man to each prompt so I want to have the the woman in there but I also want to have a man in there perfect okay so we got we got each prompt got modified perfectly we're gonna grab that head over here throw those in oops try this try that again grab that we're gonna paste that in there and let's re-render see if it even does it looks like you had a man back in the background there's there's a man there so what we're gonna do we want to change this significantly so we're going to bring this up right there on that a little bit and then come over and render again I'll see what it brings anything in that first one's not that one it's bringing the guy a little more in Focus which is good second person but not a man there we go yeah so some of them are doing it some aren't you have to play around with the settings on it it really depends on you know the width and everything of the scene so as part of this I want to actually take one of these images and we're going to use as my title image for my video when I put this out so let's see I like that one that one's really cool and what we're going to do we're gonna send it over to uh we're gonna do the uh the control net out painting technique okay so what we're going to do yeah we're gonna send this image that we generate I want to set this up in a 16x9 format so that um I can use it as a thumbnail on my on my video okay so we're gonna okay so I went back and looked some things up and what I want to do is we're gonna I want this image to be the image we're going to use as the thumbnail for the video as I put it up on YouTube so we're going to use the control net out painting technique for those of you who are not familiar there's a couple different ways you can out paint but this is a new thing that was introduced that allows you to do really quick out painting we're going to take this image we generated one that's one of the many we generated here recently uh and we're going to send it over to uh let's see where is it text to image control net see if this works there it is okay and then we're going to so this is the original image uh we have a description here we're going to grab that we created earlier when I was messing around with this and we just want to add detail to the surrounding image okay so we're going to enable Pixel Perfect we're going to use the in painting the xyle under control net under text to image okay and then the original image size was 768 by 960. I want to actually increase that but I want it to be same height but I want to be 16x9 so it's going to change that but all we do is come in here we're going to set this to 960. okay should automatically put that there and when you're changing when you're doing the outpainting method this way you only change one axis at a time so we're going to increase the width this time we're not going to do the height but you could increase the height on the second run so you leave this number the same and change that to out paint that way so we got that got that selected okay we're going to do resize and fill you always make sure you select resize and fill on this okay what that does it resizes the canvas and then fills in the blank area okay and I think all the other settings are fine we're going to change that we don't want multiple batches we just want one image okay and I think that's it so we're going to go ahead and hit generate on that oh we're gonna set this or I don't know if you need the end painting model or not let's try it with the end painting model and then we'll go from there give it a second kind of spool up here you can see the edge where it's buffering and pulling information from the type of scene that it's got going on now I am increasing the width pretty drastically to get that 16 by 9 ratio at that height uh some images you may want to gradually increase the width but I've worked with this enough I feel like I can trust it to do something really impressive what you might end up getting here is like a scene where it goes off into the forest Pat you know past this waterfall so I don't know we'll see what again what happens here it's actually not bad that's pretty good I like it got these kind of cool rock formations here definitely something I think I could run with but you can put in any description you want um it might be cool if I put like ancient ruins or something like that so I mean while we're waiting for that so we can take that and in pain with VA so yeah so we got the the Misti kind of whatever let's redo that again except we're going to change the prompt uh change the scenery to ancient ruins foreign there it is okay now let's grab that I know I'm not reading through it entirely but I actually uh I've used my prom generator and if I trust it to do something that I like let's go ahead and knit you know we're going to try it without the in painting model I want to actually do that I'm going to shut this off here I just want to see what it does I don't think we need the in painting model on this and let that spool up and start rendering I kind of want to look at some of the other options here I don't think there's many options with this oh that was the other thing I might do is is on the put the ending control step down let the AI really run wild with it but let's see what it does first here this is going to be very interesting as you can see it's putting a woman in there I totally forgot to ask it to take out the person in that one uh because I I let's stop that let's go back over here remove person from the scene all right grab that see how quick that is and let's come over here yeah hit it one more time okay while it's finishing this up I did switch it back over I didn't like the result I was getting under the normal checkpoint so I decided to try it one more time switching it back over to in painting and the results are much better so definitely use an in-painting model on this some of you may have already known that so here we got uh the ancient ruins kind of a mysterious view our focus is definitely on her really like this layout this is actually pretty cool you know when you do the out painting you do get what looks like a line right here as it tries to blend that so I'm going to show you real quick how to fix that okay so we're going to send this over to in painting bring in the image all right pull that out there let's send that over again oh we got to select the actual image here uh oh we had it okay try it one more time there we go okay so we have this line right here what we're going to do I'm going to show you a little technique to blend this in super easy this one's got a little bit I'm not so worried about that one it's this one over here that looks like it's having some issues and that's what you do okay and we are on the end painting model we are going to leave it at 0.74 we're going to disable the checkpoint here at this point we don't need it and uh I think we're gonna leave that we're going to do a whole picture no we're gonna do yeah whole picture because we got the whole thing there that's fine and uh leave everything else the same mask blur make sure it's up at about 12 for this okay and then hit generate what this should do is fill this in you're going to get a lot more breakup it's going to break that line up and fill in intuitively the details around that edge foreign probably was something that looks like hanging Vines a you know ancient ruins maybe a pillar or something but who knows so now let's grab this canvas shrink that back down pull this up much better you do get something that looks like hanging vines that blends so much better with the rest of it we could bring this over let's just for you know Giggles here let's bring this over we're going to shrink this mask down we're going to erase what's there we're just going to do the same thing here nature doesn't work in straight lines so let's do that leave all the settings the same generate real quick should break that line up or put in something that looks a little more artificial too and that's totally fine okay and you can keep working with this as much as you want to get the result you want that looks great that will be my image I might upscale it and adjust her face just a tiny bit but I love it I appreciate you guys following with me on this uh again I know this kind of feels like it's flying from the seat of my pants some of it is and uh some of it isn't but I hope you enjoy the content and learn something from this uh like And subscribe uh visit my store you see me using my prompt generator if you haven't gotten it yet you're missing out this it helps out so much and speeding up what you're doing in here and being able to come up with amazing content so we'll talk to you later thanks again thank you

Channel: AIchemy with Xerophayze

Views: 21,214

Keywords: AI art, Stable Diffusion, Inpaint Anything, Control Net, Image manipulation, AI creativity, AI-generated images, AI artistry, AI techniques, Artistic transformation, AI tools, Arcane Shadows, YouTube tutorial, AI art tips, AI art hacks

Id: 4b3eusLYNgk

Length: 32min 18sec (1938 seconds)

Published: Tue Jun 27 2023