Transform Your Images: Adding Exciting Characters with Stable Diffusion and Inpainting

foreign [Music] welcome to Alchemy of zero phase I'm Eric and I got a quick tutorial here I'm going to do on how to add people into a scene now somebody had asked about this in the comments and it is something that I played with and and and done before there's lots of different ways to go about doing it this is just going to be kind of a a quick and dirty way of doing it without involving drawing in anything or um involving control net in any way so it is going to be ex there's some experimentation and and trial and error hopefully it goes smoothly and uh won't be too long so let's dive into this I got a prompt we're gonna throw up my prompt generator here to generate a couple of Street Scenes um in Italy so I wanted something uh that was like a street market um professional photography wide view sunny and beautiful empty colorful Italian Street Market trending okay so we get our our prompts here standard prompts with camera information trending on National Geographic or turning in Magnum photos these are both websites have tons of photos and are very highly likely to have been trained into the image AI I'm going to copy that head over to stable diffusion for this particular generation we're going to be using these uh a zovia RPG artist tools with the vae baked in okay we're going to select our Prime prompt uh uh yeah I don't think I'm gonna I'm just gonna stick with that keep it there's not going to be much of a change on anything we're going to drop the sampling steps down to 20. we'll do two images per uh prompt and we're going to do this 16x9 I want kind of a wide angle I'm going to up this a little bit to about you know let's do 960. so we're keeping in the 16x9 we've locked it into that so it should automatically adjust the height gonna come down here and put in to our prompt text box we're going to drop those two prompts in there and then hit generate you know what we're going to go ahead and enable the a detailer I really have started getting used to this we're going to have a track and and because there will be people in here probably I want to make sure their faces are already set up uh and good to go so we don't have to mess with that so it looks decent but that start running through and generating some images here not bad the second one will probably we could probably use there's already some people there um there's definitely very few people up in here we could work with that one so it's currently going through the a detailers going through and correcting any faces that might find at least as best it can it's very low res but it sees and understands those and we'll just fix the general shape of them which is cool it is interesting how it'll pick various areas that it thinks there might be people and it actually added people into the market stand there that's interesting definitely some good detail though or maybe that was the second image I don't know we'll have to go to the room see pretty cool that it does that I mean it saves us so much time in correcting faces in in scenes and I did my last video that I kind of went through this and showed uh how it can benefit not sure what it's seeing there it thinks that there's faces or something there a lot of fruit maybe it thinks their eyes or something let that run through and finish up I think it's going to do one more set of two for the second prompt here a lot more people I didn't do any upscaling on this I mean it's finding these faces all by itself it's pretty crazy what it can do okay so I guess it did hit all the images that's cool all right so what do we got um we got some fruit stands got people off in the distance that's not bad you may want to add some people up front here um that one's not bad you could change that one up maybe add somebody sitting on a chair right here in front of the Fruit Stand [Music] I think I like these other ones better I'm going to start with this one here we're going to try that so here's the thing with AI and changing a scene that's already there you are going to be using end painting you are going to be changing the description and you are going to be doing things to give the AI enough leeway to really change the end paint section okay and I'll show you what I mean here well first thing we're going to do is we're going to send this over to in painting okay we're going to shrink this down a little so we have to change that so I'm getting used to this um oh that's right we're going to change this a little bit so we can see what's going on here what's up so this is a an extension that allows you to manipulate the in paint window and give you a better control over it you can you can zoom in on scenes or areas and in paint very specific things you can use f to drag the whole thing around Alt with the scroll wheel on your mouse to shrink it down it's it's pretty fantastic I'm just learning how to use it if you want to know the uh link to this one I want to say it is where is it here shot can't no you know I'm not entirely sure it might be um I'll have to look that up and if I find it I'll put it in the in the link I don't know if this was something that was just built into uh automatic 11 11 but it sure is useful okay so what we're going to do is uh first thing we got to switch this over to an in-pain model and the reason that's one of the reasons I'm using the RPG artist tool is because they got a pretty good in painting model it's on the same version just like the V3 but the one thing you want to make sure you switch over to because this one doesn't have the vae baked in got to make sure you select the vae uh ftms 140 000 ma pruned um I think in another video I said I would link to that well I'll make a note of that and go back and make sure that it's linked so okay so here's the thing we sent this over to in ping but it didn't transfer over any of the information um so that's just kind of one of the little side effects of using this down here is yes the information's in the metadata but I don't know why it doesn't bring this over here so we can either copy and paste it or we can come over here and select the image that we want to use let's see which one was it uh down there think nope not that one where is it here maybe it was this first one interesting got a bunch of other images here so let's see let's go back over here let's just copy and paste it I don't want to sit there and waste time trying to find it we're going to grab that prompt all the way down to here I'm not gonna worry about the negative prompt because we're going to go and add that in here in just a second paste that into all right we'll just leave that there actually you know I apologize we're not even going to be using this one since we're going to be modifying the characters in here all we need to do is add the prime and then what we're going to do is we're going to generate a series of prompts of simple prompts for people that could be on a this type of street okay and so I've got a prompt I've got generated here or not generated but ready to throw in here so we're going to Pro put this in here all I want is five prompts ten words each five different descriptions of five specific people in an Italian Market Street see if it kind of Pops it out the way I want so we got elegantly dressed women adorned with vibrant Floral Pattern dress browsing fresh produce on a bustling Italian art hey that's perfect like we could run with that one elderly gentleman well tailored suit selection of fragrant herbs okay that's not bad so I can pump Tomatoes um yeah let's run with this one right here let's grab that one okay now we're gonna put this in here so that's gonna be the focus of the uh for the AI on what we're going to be doing and what is she doing so we got adorned with vibrant Floral Pattern dress browsing fresh produce on bustling Italian Market Street okay so it's obviously not bustling we're going to try and make it bustling um so let's come down here let's change the size of our brush give it some space and let's just say she is oh produce fresh produce I don't know let's just put it over here so I know there's some people back in here maybe I don't want to Okay so one thing to think about when placing this is how many what do the pixels look like that you're covering up is this enough that the AI can work with to create whatever it is and there's a lot of a lot of different stuff going on here I think we got lots of stuff to work with and I think we'll be okay so think about where uh her feet are going to be and where she's going to be and how tall she's going to be okay and what we're going to do is just fill in it's kind of in a shade here but I want to just give it a little bit of pixel we want to give it enough context the AI this is how the uh diffusion stuff works is it has to have context okay and we're just going to put that there come down here we're going to do in paint only masked okay we're going to give the mask blur a little bit here's the thing we haven't done with this and I'm not sure if we need to uh we're going to run with this without upscaling it first okay but we are going to turn on a detailer here for this to make sure that if there is a face it will fix it we're also going to change this the sampler over to DPM plus sde cross okay when in painting it is always helpful to increase the number of steps okay uh it gives the AI enough room to figure out what the pixels are going to look like and I'm going to put this at 40 I think should be fine at least close to 40. we're also going to increase the batch size to two um just so we can get some variations and hopefully hit hit our mark now here's the other thing we're going to do most of you are familiar with the the denoise strength you know the lower it is the less changes it's going to make the less free in the AI has to really manipulate the pixels okay the higher it is the more change and typically you're going to be in the 70 to 80 range for most stuff for what we're doing because we're adding something that's not there we got to give the AI lots of Freedom with this so we're going to put it up into 95. I don't know it might be too high it might not be we might get something completely weird um but we're going to try that and sometimes I do turn this up allowing the AI to really do a literal interpretation of the prompt we're going to put it at eight on the config scale uh we got a detailer on I think that's it we're not messing with anything else um mask blur uh we're not dealing with high res so I think six pixels should be fine we are working with original because we are trying to give the AI enough pixels to work with if this proves difficult we can switch over to fill and it will fill in what it thinks it can with the space that we've given it okay uh you know the other thing we need to do is change this we want this to be square so and we're going to do 768 by 768. okay and that's it so uh just a quick review the in painting model make sure we got the VA for this particular one and then these very simple prompt describing just the person maybe what they're doing okay um and then the AI will interpret and and be able to adjust based on the surrounding area Okay and then Mass blur don't need to mess with that too much original on the in the mass content we're doing original and only masked okay we've changed the width and height to a square or one to one ratio you can do 512 by 512 I like to do 768 by 768. we are giving the config scale a little higher number get a little more variance a little more Randomness denoise strength we're giving the AI a lot of freedom to really change up what's under those uh what's under that mask okay and we've turned a detailer on in this particular instance you may want to experiment with this first before diving into this just to see uh what you can get worked out okay so we're going to generate on that let's bring this back over here actually grab that bring it up here up here all right I can already tell you right now I'm pretty happy with the result I you know sometimes you just don't get what you want but I've done this enough times I think it just kind of happens automatically so let's see what it see what it looks like here foreign Taylor now on each one you may see it jumping you may see the face kind of come up close we'll see what it does here if it's usually if it's one face you don't see that but we'll see what it does there it goes oh look at that oh party okay so yowzers look at that that's really cool so we got the person that in it looks like it kind of added somebody in behind her there but she is standing in such a way that um you can see the shadow kicking back there so she is in you know lit up looks like properly looks pretty good and uh we've got the nice floral dress on very very nice that worked out much better than I expected the first time so and that one looks great too look at the shadow on the feet and shadows in the right direction she's coming out around the corner um really great first try nobody behind her didn't really manipulate too much of the stuff behind uh sometimes you'll get like oddness in like like in this here you might see oddness in the shelves but I think this one kept that pretty normal so like if I wipe this out let's go wipe the mask out yeah you see how this one's lower right here so those lines are lined up the way they were before and it just added her in so that's awesome so now what we can do we have more prompts let's say we want to add in the distinguished elderly gentleman okay we're gonna grab that prompt while we're here throw that over here and then what we want to do now is send this image back over to in paint now that we've got that we've got a pretty happy outcome and we are going to let's see let's blow this up a little bit open this out here oh we lost it okay let's blow that out send back to impainting here okay and the elderly gentleman see if we can open this up there we go well let's see let's put the elderly gentleman over you know what no I'm gonna put them right here I want to be about this high maybe he's hunched over maybe not he's distinguished who knows we're gonna put him right on the edge okay mask that in okay and I think that's it let's add one more down here oh you know one thing we got to make sure of uh is that we don't have the original mask on there so I don't think we do because we clicked the X so when you send it in paint the original mask will look like it disappears but it's not actually there if you don't see the text in the center saying start drawing or whatever it says that means there's a mask there and you may not see it and so you can either hit the back button here which will get rid of them one at a time or you click the Eraser and it'll just wipe out any masks that are currently there and then you can start drawing again okay I think we're going to leave all the other settings the same the only thing we had to change is the prompt and The Mask you know obviously send the image over to the in paint window so we're going to go ahead and generate on that it's back over here this over here yeah see this one is a little different uh it's having a hard time it's not gonna get it on that first on that first try let's see so we're going to interrupt that again this is one of those things let's see we got elderly oh you know what we didn't paste in the prompt there we go distinguished elderly gentleman okay but it did not do the prompt the way I wanted it to but let's take a look at this for kicks we're going to bring this up to nine try it one more time here we are on the edge of the image and that may be what it's having a problem with too you can see how it's still not doing it so what we're going to do we're going to come over here and get rid of that mask and we are going to just mask out or we're going to stay away from the edge of the image oops I think that should be fine I'll just give it a try we'll put him right here okay give that a try oh we got one thank you oh we got both of them nice okay cool he looks a little small at least in comparison to her this guy looks pretty normal got the shadow cast on the thing there on the uh the um Market stand so give it a second it's going to go through and do the a detailer again so it may change her face if you have the noise turned up or the big scale turned up on a detailer enough let's come down here let's take a look and see what it said I haven't didn't change anything oops on it but let's see what it says come down here to the a detailer in painting and in paint you know I Strength is it 0.4 which will change it a little just enough and but not like overly change it and it looks pretty good with him he looks like he's looking at some of the food so that's down there okay so that one there let's uh take a look at that she's got a pretty blank face we might even paint her later but yeah look at that perfect awesome I love it it it uh dropped him in there he looks like he's looking at something inspecting something great let's go to the next one and see what we got so that was the elderly person we got an enthusiastic young Chef wearing a crisp white apron carefully selecting plunked paint plump tomatoes instead they got a woman in there let's see a local artist with paint splattered Smock captivated by the vibrant Ray of fruits ooh okay let's grab that guy now you can change those up however you could specify the type of people I just told my prompt generator just give me some random people that might be likely on this street so have you know that's that's the fun of the prompt generator uh you don't want to be specific tell it to be specific so let's go ahead and send this over to inpaint now we're going to go ahead and swap out swap out that prompt leave that all the same we're going to make sure we erase any current masks I want to put this guy over here I want him looking at this stuff right here so yes we are going to get rid of those people there his feet should be about right there and a lot of this is about just giving the AI enough um enough of the image to really play around to understand what it's going to do and you know it looks at the surrounding pixels and and understands hey look we're going to be adding this in I know there's a fruit stand here he's going to be looking at it it's really fascinating how the AI works and and looks at the image and understands that I don't think we need to change anything else um we're going to leave everything else the same and just see what it comes up with so again this ties into your workflow you don't have to change every settings it depends you may have to do some micro adjustments like I did with the old guy but let's see if it puts anything in there no it does not look like it's going to so let's come down here to this thing we may have to do some adjustment on them we are still only masked let's increase this just a few points hit generate again still not going to put them in there so let's do this just expand that mask a tiny bit and then we're going to come down here we're going to increase this sometimes you've got to just lay on the noise and whatever it comes up with may be distorted the whole point is to get something in there that looks like a person like if I didn't like the way this guy over here looked he's there I can actually take him and re-render him as anything else I want now that that context is there it's all about getting this to get something in there as context because right now all it's working with is you got these fruit stands you know vegetable stands whatever you want to call them and so it's trying to manipulate those pixels to get something that resembles a person out of it um okay again we're going to mess with the settings on this I'm gonna go all the way up here we're going to disable the a detailer for right now I don't want to have to worry about that just regenerate again here again A lot of times this is roll the dice yeah we're not getting him okay so we're going to add more pixels more context giving the AI enough variants to kind of work with it check our prompt two here's another thing yeah it's still not going to do okay so here's something we can do uh local artist um what we can do is take this phrase let's change it not change it what we're going to do is we're going to emphasize it okay so you can highlight the word and use it on your keyboard do a control up and what it does it starts adding emphasis to it we're gonna we're gonna blow this out we're gonna put it at 1.4 telling the AI really focus on this this is what we're trying to put in it does not want to cooperate I wonder if it thinks of this person at the back here is that huh yeah see you kind of seeing a shadow he looks like he could be something like that shadow figure in there interesting okay what do we got here uh we have somebody here something going on right there I don't like this this is not cooperating um let's wipe this mask out what do we got here yeah see it's taking those people there I think that's going to be kind of the problem I'm gonna go ahead and do this leaving those people out we're gonna give an interesting mask I'm gonna come down here um we're going to decrease the padding pixels too so the padding pixels oh you know what the other problem is we didn't change this back to 768 divided by 768 that does make things more difficult for it so padding pixels are the areas how many pixels out from this area this XY area that you're describing so the 768 by 768 area padding pixels are how far back it's pulling its view and it helps you kind of adjust the details by reducing it it's actually bringing that in paint area up closer versus almost like farther away consider it like a focus you're focusing the in painting area okay and so let's uh let's give that a try let's just see if we can get anything into that area this this painter that we're trying to get in there yeah it's putting people in the back stance there I'm not liking that and swipe that out what I'm going to do we're going to do this I know that's back in there too but we're going to give it context up here on the street oh what happened there let's try that again let's increase that mask size just so we can work a little quicker here try that and we got these settings just turn dialed way up to try and get anything in there what I might do is dial this I'll dial this down next to see if we can get anything to pop up it's having a real difficult time you know trying to come up with anything that looks there and we may end up going to one of the secondary uh purposes or secondary options of like drawing something in there to try and get it to do anything but I'm going to turn that up we're going to turn this down just a little bit foreign here's the thing because I've got this turned up the config scale to 15. it's going to blow some stuff out okay hyper color it may look okay kind of but now that we've got context there we're going to uh mask out part of that area and re-render it something with something a little less I don't know that's not bad got the colorful uh painters Smock on but it does look blown out you can see it's just super bright colors kind of doesn't fit the rest of the scene so now like I said what we can do is um actually before we go on just a quick review like I said make sure these are square width and height square and these are the ones you're going to be messing with turning these up all the way will get something image it basically scrambles the image and gives the AI a chance to kind of get something in there but then now what we want to do is turn them down we're going to go all the way back down to seven on this one this one since we got somebody there we can bring this down to uh the normal range like 6.75 okay on the denoi strength config scale down to seven we're still going to leave it at two uh render two images because we want to get some options we're going to send this one over to in paint we're going to wipe it out because it just the mask disappeared okay now all we have to do is just mask her out okay the rest of the image looks fine we're just going to do a quick mask what it's doing there again all we want to do is just normalize the colors a little bit okay all right so that's it we're just going to generate see what it changes on her and she disappears so what we don't want to do is have this one turned up I forgot we're not it's it's uh messing with a little too much local artist yeah okay and part of the problem is local artist is a very vague description we're really pushing it without instead of being very specific and oil painted Smock and so it thinks that these covers down here on the tables are the Smock you know the the with paint sprayer you saw some of the renders we did these looked painted and it's talking about the array of fruits and vegetables so it's having a really difficult time adding in that artist so now that we have the artist in we don't want it to change too much I totally forgot about that we don't want to change the whole image up here we're going to keep it down into the uh we'll say 0.55 okay that way it's it it's keeping the image generally the same but giving enough freedom to change up some of the details and normalize the colors as you can see it already looks a lot more normal oh yeah that looks a lot better okay at least that one does no she's not painted I want something with paint all over that looks great okay all right and so that's I'm just going to do those three individual examples um and we have the street here up right up the center we could add something in there but I think you get the idea um that you need to you mask out the area where you want the person give it enough context uh realize what you're masking or obviously I didn't pay much attention to the fact that this is still describing uh fruits and vegetables at a picturesque Italian market so because it had that in there the AI was really trying to stick to that instead of adding the artist in so if you get rid of these just put in something that describes a person just the person maybe they're clothing a little bit leave out anything talking about the scenery and I think you'll have a better uh a better you'll have better luck at getting that person in there initially now that that the person is there we could the AI has context to work with and you could actually change that person into anybody else you wanted okay I could take any one of these people and change them into something else because there's context there there's a person there that's what the AI looks for person female dress but it starts off person so you could write in uh mask her out and put in in young Debonair man or whatever and it would change that right there you give it enough uh freedom of the config know or the the denoi strength and it would change it into a man okay getting something there that doesn't exist that's that's a little more difficult part I hope that this tutorial shows you how you can do it um you could take a little bit of time beforehand to take like the image before you put anybody in here into like mini paint this is a an extension you can get and you take the image into here and just draw in like a stick figure and then you take that back out and go to image damage uh in painting and do the same thing mask out that stick figure and the AI has enough context to work with okay it's really interesting the way the AI works that way but very useful as you can see we got some decent results from here just take it in and do a late and upscale with a detailer so it goes in and fixes faces and makes everything look really nice so I hope you enjoyed this and I look forward to doing more videos uh keep the suggestions coming out this was a suggestion from somebody else they wanted to know how to add people into a scene and um I hope this was helpful for them uh so again appreciate it subscribe uh like the video and uh if you haven't joined our Discord go ahead and make the request uh we got got a ton of people on there now uh and uh we're we're really having fun exploring the uh use of our prompt generator or the prom generator I made so um if you have questions about it you know I'm I'm available you can ask me about it too so it's it's a great thing to have helps with the workflow okay talk to you later foreign

Channel: AIchemy with Xerophayze

Views: 14,154

Keywords: AI Art, Stable Diffusion, Inpainting Techniques, Image Manipulation, Artistic Transformation, Creative Process, Digital Art, Artistic Enhancement, Character Addition, Image Editing, Art Tutorial, Creative Techniques

Id: hM9afSE27AA

Length: 35min 21sec (2121 seconds)

Published: Thu Jun 29 2023