Dual Characters One Scene: A Fun Stable Diffusion & Auto 1111 Tutorial with Inpainting & ControlNet

[Music] hey everyone welcome back to Alo zero phase this is Eric and I had a bit of a fun uh idea that I wanted to try earlier today went through it and L the result and thought it would be kind of a fun little quick tutorial we can show uh people how to uh how to get multiple specific characters into a scene and um we all know that working with AI image generation that sometimes well not sometimes most of the time when you say I would like to see this character and this character in the same scene that most of the time it doesn't give you that and I'm going to show you an example here so um going do Superman and spawn standing on a city streets okay uh let's get a couple things set up here and see okay we're doing kind of a portrait we're going to try to get these two this dynamic duo standing next to each other uh and see see how that turns out I'm going to change the model to a sdxl model here real quick and we're going to generate a couple of them CU there's a particular I want to see if we can get them standing next to each other and get the sizes right so let's just see what we get here I want something where they're kind of standing next to each other both their bodies uh well defined you know from each other separate from each other and it looks like that one right there should be good so let's see I think it's this one right here yeah so so this is what I'm talking about you know I don't know why it's interpreting that I mean obviously it's trying to interpret the darker tone of Spawn but this is kind of what I mean it's like you're getting um I don't know if it's bleed over through the process it just the AI has a hard time defining two specific separate individuals and so what we're going to do is we're going to just show you the process of replacing him in that same position and everything with spawn okay so we're going to go over and uh let's see got that so we're going to save this image and then let's grab that over here here okay so we're going to go take this image okay we're going to go over into normally I would upscale this image first but you know just for demonstration purposes you'll get what I mean you can you can figure it out from there so uh we're going to take this image and drop it into inpainting first okay see if it didn't I don't know if it dropped it or not let's try it again here there we go okay now uh we are going to switch this back over to uh a regular sdxl model and the reason why is you can do in painting to a degree uh pretty successfully with an sdx a regular sdxl model without using an in painting model um there's just certain things you got to have in place so uh the other thing we need we need a better prompt I mean this is great but I want to have a better prompt uh about spawn cuz this is going to be we're going to be replacing spawn right now we may even extend this tutorial to do a better version of Superman so we're going to take this we're going to go over here in my prompt gen we're just going to drop that in there uh generate a just a simple prompt from that oh not analy sorry submit get a prompt okay and spawn streen level I want to make sure it actually specifies spawn no we not getting try to stylish spawn mysterious spawn you know let's do this Todd mcfarling I'm not sure if I'm spelling his last name right but the AI knows so yeah Todd McFarland spawn so let's go ahead and copy that grab that over here we're going to go into the image damage we're going to just drop that in there I just wanted something a little more specific um you know what that the problem with that is I wanted it I wanted it um photo realistic let's just make turn that into a photograph ioto photograph okay that works okay now that we got that set in there let's come down here and here are some of the settings you're going to want to pay attention to to kind of help with uh in painting when using just a regular sdxl model first thing you want to make sure is Turn Up The Mask blur okay this will help eliminate that that border that seam that you get uh because it's having a hard time blending in that that edge um steps you want to increase as well probably do at least 35 we'll try that and see what see what happens we're going to leave the sampler the same um we want this to be a one to one uh but at the sdxl level so we're going to go all the way up to 1024 and we are on we're using the sahastra kai model and yeah oh we got to mask out what part we want to actually manipulate so we're going to just grab this and now see a spawn think about I think ahead of time what is going to happen because his cape is everywhere um so I want to give this mask area enough space to accommodate the more Wild look of the character okay um so what we're going to do he tends to have a high color C you know a cowl sometimes just going to do a nice big area like that okay it'll also help like when we using control net it'll help with bringing in the background and and blending those edges okay this this is actually a pretty simple image uh when it comes to the background not a big deal it shouldn't have too much problem with it okay now let's come down here we're going to leave the doo strength up high uh because I want it to change what's there now we may turn it down depending on how much it changes but I think this should be good next thing we're going to do is come down to control net we're going to enable it we're going to do Pixel Perfect we're going to upload an independent control image and then we're going to drop in that image we created okay and we're going to select canny now canny will grab every little detail including the symbol his belt and everything you kind of put them all right there so we want to make sure we give the AI a little bit of leeway in what it can do with that area but still maintain like the same kind of subject position pose whatever so we're going to drop the control weight down we'll start at 6 I think that should be fine I think that'll work okay uh we're not going to change anything else we're just going to try that we got blur setup we've got uh we're not going to worry about O mass padding yet unless we're not getting what we want and let's just hit generate and see see where we're at with this should give something pretty interesting here uh the control net is necessary to let's see what do we got going on here it's oh oh sorry okay so one thing I forgot to select was the only mask okay we only want to in paint this area here okay we may have oh you know what we are going to have to turn the uh control net weight down quite bit I forgot it's the canny is a pretty heavy hitter when it comes to maintaining what's there I that's a pretty cool look you're getting kind of a a very dark version of Superman Lobo is I don't know what that is so let's come down here we're going to drop that way down I think for the other uh test I did I think I was actually down at 02 so let's let's go ahead and run that and see and see what that looks like okay okay it's looking better not getting the chains I mean we're we're getting you know kind of the the the fill you know not bad let's uh let's drop that down even more you know another thing we can do so there's two different things here you can mess with to really kind of get it to do what you want it to do um we can drop this down down to 0.1 and the other thing is the ending control step we're going to render it one more time before we mess with this one though um what that does is it basically stops using control net a certain percentage through the generation and allows the AI total freedom with the with whatever it is it's doing all right that's looking much better not getting the hood I kind of wanted the hood you know but that's looking really good you know we're getting a a little bit of oddness right here where the uh the seam of the the mask is um but I think let's just bring that down just a little bit okay so what we're going to do we're going to drop the ending control step down to 30% about 30% through okay what that means is it's going to start the image off it's going to get us that character there but at 30% it's going to let go of control net and give AI full freedom to kind of work with it so let's do that I'm going I'm I normally I would pause these images but or you know these renders so that it speeds through but I kind of want you to see this so we're getting a little more of the hood which is cool one more Spike looking a lot like uh uh Venom that's interesting I wonder if what we can do is uh let's get rid of that spawn or spawn with chains and very epic I don't know I'm just gonna throw those in there uh to see if we can get I that's I mean I love that about spine I love the chains uh let's go ahead and drop the the ending control step down to 20 perish and redo that so we're getting a bit of a bigger character which is cool we're getting more like you getting some of the chains uh more detail in there yeah not bad still it's very I don't know it's okay it's maybe it's just because we're we're dealing with a fake character you know it's got that plastic look to it you know but uh that's okay so we're now starting to get a little bit of the color that's okay but and yeah everything else seems to be okay like there's no seam here um it's blending it in pretty well so let's go over here I want to say why a photography let's just get rid of this we're going to go um we're just going to get rid of that I want to see what it does okay that's pretty cool that's kind of what I'm thinking like that he's got love how he's got a watch or whatever that is on so that's pretty cool so I mean you can mess with this all you want but this is the trick I mean this is the key and you know by by blending in that edge using that uh um the setting uh maskable you can kind of blend that edge in a little bit better uh something else we can do oh that's maybe why it's not getting to be so wild let's do this let's turn that up and bring this up back up just a little bit so just so you know the dooy strength tells the AI just how much change you want to happen Okay when you're doing like an upscale you want it down really low so you're not losing the whole picture you're just adding some detail to it right so increasing this tells the AI hey you have more freedom to change what's there so let's let's do that okay so it did change it a little bit more we're still getting kind of that same feel to it we got some chains coming off here this one is not as good as the other one and here is the drawback to increasing the amount of change and this ties into the fact that we're using sdxl to do the the end painting is you now are starting to see a little bit more of that that Bard not it's not too bad like you could pass that honestly what you do is take this and I'll show you how to get rid of that that blur Al together um and actually blend the whole image together we're going to turn this up a little bit more and leave the other one down low and do that yeah so this is what I mean when I say uh you know when you give it too much Freedom you end up with something like that so which is cool if it was a normal looking character so if we come down here let's just bring this up here so that I want to maintain that 87 do strength but if we do that that means we got to strengthen this up a little bit more to tell it hey no you got to maintain this we'll give you you know I want you to change stuff but maintain that so a little too high on the uh control net drop that down maybe to 0 2 see what that does okay I actually like that it is a slight variant on the whole spawn thing but definitely got the bracers and the chain for the belt and everything I like that you got the chains kind of laying on the ground over here that's pretty cool although you know honestly I would love the chains to be up and flying around like they normally would be but this is pretty cool you got the the cowl down right now but there is one there which is cool I like that so this actually turned out the way I'd want and and we didn't get the seam um I mean it's it's almost non- discernable in this situation it's it's doable it's working so we can actually take that I love that we can work with that so we're going to send that over to in paint and now we've got that image there we're going to actually do a quick one with Superman here um this will be a pretty easy one because I just want to add more detail he just looks so like soft like Blurry almost you know not really blurry it's it's hard to describe almost 3D generated so we're going to do that um and over here we're just going to come and do change to to Superman on the prompts uh Superman flank no no no no let's see change to Superman standing on the street don't want it describing that he's flying uh okay yeah that's perfect just copy that one come over here we're going to do this paste that in there and then uh we don't I don't know if we need to change much of anything here we can probably leave all these settings the same it's going to give us a much different looking Superman hopefully a lot more sharp Vivid um look and so uh let's just let's just roll with it and see if there's anything we need to change after the after the fact all right let's have a look at this yeah it's much better you know kind of the the whole Henry caval look which is fine I think that's what the AI model tend to be a lot more trained on so what what's what's nice about this is it is um got got more detail to it uh obviously fa is a lot better and match it kind of matches the same kind of shiny looking style over here almost kind of like a some what do you call them the the little figurines you get you know or the the larger I don't know what you call them the dolls whatever anyway so uh but definitely looks great it Blended in nice there's no seam around the edge uh looks like they're standing there now here's the thing that you can do let's say you do have a seam okay you've done this I know a lot of people are used to using the in painting models so and because they blend everything in they tend to work well but they don't have the same capability capability as the sdxl model okay so you can do this use the sdxl model let's say you do get a seam okay now there's two things you can do to fix that you can go ahead and switch over to a an in painting model and in fact we can show maybe both methods here let's um this didn't really have a seam I mean it's got one kind of over here you can kind of see over here a little bit right there um maybe in here so those are some areas that we can kind of adjust I guess a little bit so what we would do let's send this over to in paint again so we get both those characters in there they look great now let's say we do have a seam over here I'm going to show you the first technique of getting rid of this and it really just involves masking out that area we're going to shrink that down and we're just going to grab that right there okay um we're going to grab this right here oops okay you don't have to be super precise on this um see is there anything else that looks like it could be could be fixed a little B like maybe that right there okay so what we're going to do we're going to come down here let's shrink that back down okay and we're still going to do only masked but this this time we are going to switch over to an in painting model and my favorite to use is the RPG artist tool V4 in painting okay this does require uh vae so once I gets done switching over here we're going to switch over to um the VA the standard VA the 8 840,000 vae so um I think it works pretty well with this now uh Mass Explorer we don't need it as high for this I don't think we're going to switch that down to eight um leave everything else the same uh we are going to change this to a super basic description of uh uh what do you call it uh city background slightly blurry uh so as long as you get somewhat close I mean that's all you need uh and and the reason being for that is the in painting model is designed to take into account what's around it um and that's how it pulls most of its style it's it's what we call context giving the AI context in fact we're going to bring this up a little higher the only paint only mask padding pixels gives the AI more context to work with Okay so we got that we got that uh we are going to switch this again to a 1: one but for this one we are going to switch down to 768 because we're working with a 1.5 model but this models uh is trained on some larger images so it does pretty good at 768 actually fact it does look really good and then we're going to change this down to fiveish we'll do five and a half .55 because we want to change it um not but we're not looking to get a whole ton of change we're just trying to blend that seam in and we can just disable the control net we don't need it anymore okay and that's it so what we're going to do uh sampling steps we're going to leave up high when you're doing any kind of in painting you want higher a little bit higher sampling steps I mean you don't have to go all the way up to 50 uh but you don't want to be down under 30 okay so I usually will do 35 40 depending on what I'm what image I'm working with uh I don't think we need to worry about a negative prompt next necessarily because we're just doing blurry backgrounds you know we're just trying to get rid of that seam so we're just going to generate on that so this doesn't take very long and it did a much I don't know did a little bit better job of getting that out didn't change much what it did do it looks like is it made the face a little more realistic here I'm actually going to turn up the dooy strength um because I think up to 04 you're not going to get a whole lot of changes I don't think um may even add some detail we may even uh I'll show you the upscale here maybe just real quick on this one let's goe and render this at 0 4 yeah so um we're losing that blur we fixed this edge here capes look fine uh overall yeah I definitely redid the whole thing which looks better now uh so that's one way of doing that so if you don't ever want to if you're working with sdxl and you don't want to use an in painting model this is one way to do it and it's super quick too if you're just trying to get rid of that seam go ahead and just render the whole scene if you're okay with it just changing slightly maybe adding some more details whatever okay so there you go um basically it's a combination of using control net uh in painting masking out the specific area so that you can change that character to what you want or the person to what you want it even works with you know if you want to just do a face maybe change the face out for something there's a lot of things you can do with it um it's just a good concept to understand that you can you can change individual things in the image and still get a very good realistic look or you know as if the IM the the character looks like it belongs there kind of thing okay um and as a little bit of a bonus uh if you guys want to know just real quick how to do a nice upscale on this uh we're in image image already so what we're going to do is we're going to turn this down to3 again we're going to do go ahead and I think switch this back to the five 4x5 and then what we're going to do is just slide that all the way up now okay I can slide this all the way up I got a 12 gig v vram card um you can I think uh you can for people with eight gigs you can probably do up to about I think it's like either, 1600 or 1536 fish on one side and then 1232 on the other okay but I'm just going to slide that all the way up and hit generate on that okay took a little bit longer to get that rendered out but you end up with a decent image that is fairly sizable and you get a lot more detail when you do that as the initial uh upscale okay actually fix some of the chains back here as well okay and even doing that upscale will help solve a lot of the you know you know those seams when you're doing uh in painting using just a regular sdxl model okay so um the other thing that we could show you real quick as part of the upscale uh workflow so we just did a a kind of an inplace upscale just basically increasing the sizes turning down the doo strength and getting a larger image with more detail while maintaining the structure and integrity of the image at this point now what you would do if you wanted to go up even further CU right now this is a 248 X6 40 image um what we would do is come down to control net again uh we don't care about that we're just going to enable it don't need the preview image or whatever all we're going to do is enable the tile tile blur it's going to add a bar down here and just a second right there okay down sampling rate we want to switch this to two don't ask me to explain why I think what it does it down reses it just slightly giving the AI some freedom to add detail and then we come down to to the scripts do ultimate SD upscale we're going to do we're going to change the the uh where it says Target size type we want this to be changed to scale from the original image size so scale from image size we're going to scale two times so we're going to go from 248 to 496 uh I use the 4X ultr sharp uh the upscaler works really well I love it maintains detail and structure adds detail um and uh just overall it's just an great overall upscaler okay that's it so we come up here I don't think we need to change anything else we're just going to go ahead and hit generate on that and I'll show you how big it is when we get back okay it's done took me about 57 seconds to do that and we now have a much Sharper Image that if we blow up why does it not look like it's bigger is it just scaling to the size of my window it might be see what it says here where's a size 16 64 by 20 48 but we just doubled it so let's go to image browser go over to image to image let's just check that and see what it actually is go over [Music] here huh interesting it did not upscale it let's give that another try what would we do wrong here we're already at 1640 enable this tile tile sample down scale down sampling sorry I don't mean to babble on here from original image from scale from image size four times yeah I mean that's that's it that's interesting that uh it did not upscale it see I must be missing something but you know what oh oh oh yeah switch this back up to one I'm guessing that might have something to do with it and then this all the way back up to here okay let's try that again see you can actually see what the size looks like okay I guess this is why we're here for because we're all learning there was a step in this I missed so there's a reason why it was didn't look like it was actually upscaling it actually was but it was upscaling the original 1024 image okay to 2048 so I needed to actually bring in the image that uh we had originally done either that or or uh we can even do one of these but uh let's go ahead and slip this one in here you can see there was a slight change there in the in the level of detail so now we are actually working with a 2048 image and we can now execute the upscale on this to get the 4,096 scaled image okay that took quite a bit longer I know it's going to be an upscaled image and uh we are getting some oddness here um this is probably a byproduct of having the uh dooy strength up too high typically when you're doing this upscale this particular upscale you're down I didn't think about this either before so um you'd want it down at 0 2 okay but we can take a look at the image anyway you can get an idea of just how big uh it's making it adding a lot of detail into it even gave him a nice little nipple um but you're typically going to get a much bigger image a lot more detail in various aspects of it uh um that's it's not updating the screen here there we go that was weird did A little weirdness with his face yeah I would definitely turn down the dooy strength so that it's not doing this it distorts stuff really bad uh when um that's turned up too high and you're doing that that final upscale getting that 4K image so let's do it one more time I want you to see a good quality image so you know it does actually work this is how I typically do it again I apologize with not with having that up too high there let's go and reender it one more time so you can get you can get an idea what I mean okay we may have to do another tutorial on this I realized that one of the bigger issues here yes it did upscale it and it looks okay okay but I'm going to I'm going to lay this out here the reason why I'm getting some some artifacting here that I don't like is I'm rendering these images as jpegs okay instead of PNG I you know to save hard drive space whatever you know typically I only do pgs when I want to do transparencies when I remove the background and don't get me wrong this is okay uh again it's all about about how you originally rendered it he still got kind of that plastic look and then that's fine whatever but what I'm seeing in the background here a lot of this this banding on the gradients here has a lot to do with the fact that I'm I'm using jpegs and so as it's upscaling it's kind of rendering that in kind of you know getting that banding in there so as you're doing upscales keep that in mind um if you're planning on upscaling an image okay render in jpeg is all you want when you find an image you want you know grab the seed grab the description so render it again but change it over to PNG in the settings okay and what I mean by that is is over here under uh save image grids uh under settings you want to change this over to PNG and then rerender the original image and then start doing the upscaling process okay or you know what we just talked about here replacing the character everything else and then upscaling you get a much much better image so maybe I'll revisit the whole upscale thing in another video and show from beginning to end the different techniques with that and the things you need to watch out for so that was just one of the things I had forgotten about not a big deal like I said it did turn out good the original point of the video is to show how to replace the character I hope everybody got something out of that hey like And subscribe to my Channel like the video and uh definitely leave comments let me know uh how crappy I'm doing or how awesome I'm doing whatever it is you want to do okay um love to hear back from people I love responding to people and we'll talk to you all later see you

Channel: AIchemy with Xerophayze

Views: 1,950

Keywords: Stable Diffusion, Automatic 1111, Inpainting Tutorial, ControlNet, AI Art Creation, Digital Art Tutorial, Character Design, Dual Characters, Creative Process, AI Generated Art, Art Techniques, Digital Illustration, Inpainting Techniques, ControlNet AI, Artistic Innovation, AI Creativity, Stable Diffusion Tips, Image Generation, Mixed Characters, Art Tutorial

Id: _7C0nO3_Ptg

Length: 31min 52sec (1912 seconds)

Published: Thu Feb 08 2024