Advanced Canvas Inpainting Techniques with Invoke for Concept Art | Pro Diffusion Deep Dive

hey everybody I've been looking forward to doing this video for a while uh we're going to keep this focused on breaking down the process of using artwork in order to evolve a concept into a close to final asset we work with a lot of concept artists who are typically taking something that they have drawn on their tablet or in Photoshop passing in that rough sketch into invoke and working to get that closer to their final Vision obviously what I'm starting with here is a little bit more detailed since I generated this earlier before the video but the same idea is going to be pretty useful as you go through the process of transforming your work into something new I'll walk through it step by step and then we'll take the output to the canvas so we can get more details and finalize the rest of our image now if you're just getting started with evoke we're going to touch on some advanced concepts in this video I'm not going to break down all of the basics because uh there's a lot of other content out there that'll help you get started with the canvas so we're really going to focus on primarily the process that I'm using to add in more detail focus and tweak the ideas in the image and get to the final asset I'm looking for I'm going to start by taking the line art that I've created and pass that into a control nut one thing that you'll notice is that I'm on sdxl which means that I'm dealing with a higher resolution uh output you're going to want to make sure that whatever you're pre-processed image is that if you're using a processor that has the detect resolution and image resolution variables on the advanced settings those should be bumped up to match the new resolution you'll notice that you can kind of see how there's a blurry pre-processed image that's been generated here and with something like uh the HD processor and soft Edge we're going to want that to be a lot closer to our Target resolution which are a lot sharper and punchier and we're going to want that when we generate now I'm going to go ahead and update my prompt here um obviously going to take out all of the things that I used to generate the image initially and I think we've got a decent set of prompts here to move forward with now what I'm going to do is with the soft edge control adapter is I'm just going to decrease that to 90% for the instep so that I give it a little bit more freedom at that tail end and then we'll go ahead and generate a couple of iterations great I've got a couple of options to work with uh I do want to call out a couple of things that I'm seeing here because this is where I see a lot of concept artists struggling with kind of that first generation that comes out almost all of these have really lost all of the detail in that face uh originally when we had this kind of initial input is very clearly a human face there and that detail's been lost um it didn't really come through on the pre-processed image if we take a look there it definitely looks a little blank and so that's ultimately what we got out of the generation now if we wanted to we could go back in and try to fiddle with settings in order to get that face picked up additionally we might go in and draw a little bit of a thicker line in order to get that picked up by the pre-processor but I'm actually going to take a different approach and I think that this is a really viable way of moving through the generation process at the end of the day what you're looking for is to get that initial concept into the output and if we like some of the details that have been generated by the system we can kind of roll with it and add in those Core Concepts that we initially had in the canvas and that's kind of what I'll walk through today now of all of these I think the one that has the most interesting coloring is probably this one it's what speaks to me the most and so I'm going to go ahead and send this to the canvas uh and we'll go a and get started on generating with this now I'm going to go ahead and zoom in uh to decrease the size of my bounding box uh my scaled bounding box is already set to 1024x 1024 so it's going to generate this image at a higher resolution but what I want to do is I want to kind of come through and pack in as much detail as I possibly can as well as address some of the core misses in the initial generation now one thing that we're going to need to do is to come in and get some flesh tones on the face uh so I'll come in and pick uh a little bit of this color up uh and get that into this should let me do that with a little bit of transparency here now that I've painted that in I'm going to come in bring my D noising strength down a little bit and I'm just going to regenerate this uh to get some of that detail uh and one thing we're going to want to do is make sure that we disable our control most of the time I find it's better to just go ahead and delete so that we don't have any accidental mishaps and then we'll go and generate all right so we landed on one that I think is going to work for us uh relatively well uh just as a placeholder again this isn't final and we've got a lot more details that we want to pack in here uh so what I'm going to do now is I'm going to add a control net I'm going to go down to that soft Edge again just because that that's one of my favorites for dealing with uh concept art and I'm going to import the image from canvas again that's going to take this block that I've got here this founding box region and bring that in uh and I do want to bring that resolution up again to the 1024 by 1024 now in this case I'm going to go ahead and bring the uh InStep percentage down to about 75% um just going to give it a little bit more of that freedom because we're dealing with uh in painting here as well and I'll go ahead and select a pretty significant amount of this now one trick that I typically advise when using the unified canvas to regenerate sections is while there are a lot of blending tools that make the canvas integrate your in painting really nicely you'll generate the most consistent and coherent results if you treat your in painting like a jigsaw puzzle so mentally segmenting different areas of the image that should really fit together with the other pieces um this might be certain body parts certain sections of armor for example basically anything that you want to ensure is Unified and consistent in that generation so for this I'll go ahead and select the uh arms and pauldrons here everything from that mid Rift belt up and we'll go ahead and catch the head here on this side as well I'm going to erase the top of this collar uh and we will go ahead and regenerate this section uh I'm going to bump up the D noising strength since we're using control net uh and I'm going to go ahead and add human face to my prompt just to make sure that I keep as much of that as I can now I personally think this last one looks the best so I'm going to go ahead and accept that one and what I'm going to do now is I'm going to move to the lower body now one thing that you'll notice is that normally when you're doing in painting this type of region is really really hard to prompt around uh you've got this kind of weirdly cropped section of the image uh it's not going to really play nicely if you try to use just in painting without control nut but there are a couple of techniques that we can use to really refine how it's going to interpret and generate this section and I have found that this is a really interesting way of getting variations and packing in that additional detail without losing too much control so let's get at it uh we'll go ahead head and reimport the image from canvas so we're going to kind of take this new snapshot of our bounding box and that's done a really good job of kind of capturing this piece here I don't need to resize anything since I've already kind of updated that control adapter what I'll add next is an IP adapter and again because we can use all of these tools together uh this is a really interesting way of getting even more control than just control net so what I'm going to use now is the IP adapter plus sdxl model and similarly I'll import the image from the canvas and that'll just kind of take this founding box snapshot in and reference this image if you haven't caught up on IP adapter the release videos for IP adapter go a little bit more into detail about how these work but a quick summary is that the IP adapter models come in a couple of different flavors the base model which is really looking at Concepts while retaining variability in positioning the plus variant which focuses on concept and position and the plus face which is used for keeping a character's face consistent as you're going through different Generations in this case I'm using the plus sdxl to give me a lot more positional and concept control in the generation again one of the struggles here is how do I prompt for what I'm looking at in this bounding box it's kind of hard to articulate what all is in here but the plus sdxl model is able to visually capture that information and I can just pass in some of the other related Concepts that I need to in my prompt so we'll do futuristic machine Samurai cropped lower body and we give this a go now I'm going to want to go ahead and regenerate at the lower half of this section including this belt area and this Gauntlet and energy area I'll take out the knee here because like I said I like to keep the jigsaw or puzzle pieces distinct from one another and then we'll go ahead and invoke looking at my options here I'm noticing that most of these look almost exactly like my original image and that's because I left the weight of our IP adapter Plus at one I really want to give it a little bit more freedom here so I'm going to go ahead and decrease the weight down to6 and I'll also go ahead and reduce the EP percentage towards the end so that final piece of the generation is completely untethered from the controls we'll go ahead and generate a couple more so those initial Generations that I had kept this a little bit darker and in the lower weight IP adapter Generations we got some lighter variants here and we'll go and pick one and move on to the next section I'll go ahead and continue on with this technique finishing up some of the details on the image and then we'll come back to review a little bit more of how I might approach this with next [Music] steps [Music] [Applause] [Music] [Applause] [Music] sh now we've got a decent image but it wouldn't be Advanced canvas painting if we didn't try something at least a little out of the ordinary so what I'm going to go ahead and do is I'm going to import some images that I found in the public domain one of which is this Japanese flower pattern and the other which is this uh kind of Samurai plate with a dragon on the center we are venturing into risky territory but as they say no risk no reward so first I'll go ahead and drop this on my IP adapter uh I'm going to go ahead and turn off the plus version of this because I just want the concept from this uh and I'm going to bring that weight down as well to about 04 I'm going to add another IP adapter uh and we'll bring it back up to that and bring in my flowers uh I'll also bring the weight on that down uh we'll go ahead and take a new import for our soft edge control uh but because we are coloring a little outside of the lines as it were I'm going to reduce that InStep percentage just a little bit more uh and now we're going to go ahead SP some color on this chest plate and what I may do is I May zoom in a little bit more here take in a new import that's focused in on just this area and we'll make sure that we have Nate armor as a prompt we're going to select this region here I'm going to do uh two at 75 and two at maybe 6 now looking at all of these I definitely think we got a lot more of the flowers than that Dragon uh that's okay I'm probably just going to give up on trying to make the dragon work actually I'm really not over this Dragon I want to make it work I I'll figure it out can't give up right I'm going to take this one cuz I like the darker red on the armor and we're going to make this work so let's zoom in just on this Dragon region here brought in a new import of this closer in zoom up of the Dragon turn off our flowers update our control here and I'll make this plus about 6 I'm realizing I did not turn off my control adapter which is giv me some weird outputs always good to double check your uh control adapters when you're messing around on the canvas you can always see right here how many control adapters IP adapters you have active uh in this case I'm going to turn that one on off and we should hopefully see some better results I recognize that some people might think it asinine to focus on a 64x 64 pixel area section of the image but when we zoom out you can tell that somebody really focused lot of time and love getting that Dragon just right so we'll accept it and we'll move on uh what I want to do now is I want to take this dragon out going to turn our flowers back on maybe increase this uh weight a little bit and then I'm just going to touch up on our Samurai armor we'll go ahead and import our control net here and turn that back on and for this we can go ahead and keep our control high we don't want to change much of the shape here and and we'll go ahead and add in the term decorated and we'll give this a shot and I think it turned out pretty nicely so with that we now have our Samurai now as I look at this image there are a couple of details that I want to go in and fix and normally I might that someone could fix off the video uh because this is focusing on really that kind of like end to end workflow that a professional would go through I'm going to go ahead and dive in and finish out the rest of this so let's get this thing wrapped [Music] up [Music] [Applause] [Music] oh [Music] [Music] [Music] [Music] so as you can see there are a ton of cool tips and tricks that you can use in order to control the generation process use various inputs in order to guide Generations more towards your Creative Vision and rapidly accelerate workflows towards interesting Concepts and near final assets hope you enjoyed it feel free to share comments on what was useful what you'd like to see in the future and as always like And subscribe if it was helpful we've got more coming talk soon

Channel: Invoke

Views: 6,142

Keywords: stable diffusion, invokeai, ai art

Id: -xzvIpfGTXg

Length: 21min 5sec (1265 seconds)

Published: Thu Nov 09 2023