Creating Consistent Character Concepts with IP Adapter and Img2Img | Invoke Studio Sessions

[Music] right everybody turn this on hey oh can everybody hear me yeah we can great all right I'm going to share my screen here uh momentarily and we'll wait for a few folks to get in and then I'll cover a couple of announcements explain what we're doing today our focus with these events is really to kind of go through how we're seeing professionals start to use these tools uh what's compelling for professionals and you know people who have aspiring uh creativity and how they can really work with these as tools rather than just kind of Shifting the whole job of creating to an AI our goal as a company since uh you we started the company and before then when we were just kind of an aspiring open source project the very Foundation of what we're trying to do right now is equip creatives with the tools and education to succeed in the coming World we're kind of you know starting to understand uh how AI is influencing our work and how we create and you know overall I think we've seen a lot of feedback from the video tutorials that we've done that have shown how these tools can be used um and and how they are controllable an area that a lot of professionals uh site needing tools for is to control the AI to get consistency to have more influence over what's comes out the other side and to really have VIs visibility into what that process looks like and that's an area that we you very candidly work to find the balance for you know I think there are a lot of tools out there both in the open source space as well as you know some of the consumer grade tools that are on the internet um that have taken all of the technology and created a very thin layer where you put in a prompt and you get an awesome picture like 95% of the time but that awesome picture isn't always what you really wanted or needed for a project and you also get a lot of variability in between uh images that you generate one of the the conversation that we had uh just this week with a professional was you know they're they're they love the fact that mid Journey can create really cool pictures but they struggle with using it on projects because they really really need consistency and they need it to match certain specifications um they need it to like look a certain way be composed a certain way and so they are struggling with that uh the balance of control as well as the flexibility that these tools provide so this studio session is our first Studio St studio session and I I've got to remember not to like Dwight Sho the table because I am very emphatic and uh I like pounding the table a lot and I know it makes a lot of noise um each week we're going to spend an hour or so working on Creative problems that we hear uh both from the community as well as from the professionals we talk with we're going to show how uh I play around with and use these tools as I'm helping educate our customers um again inoke is an open source project we have an Enterprise tool that we're deploying to commercial creative teams and kind of helping bridge that Gap where a lot of companies are looking at these tools and saying we need you know to to Really integrate this into our pipeline in a way that the creative is controlling it rather than we're you know rolling our our dice on generations for 3 hours and hope hoping to get something um and what we want to do is is demonstrate how we can bridge that Gap to everybody and take the challenges that you all are experiencing and walk through and work through those live um I also recognize a lot of times on the YouTube videos um you're getting the kind of final result and not seeing the messy middle of me kind of figuring out how to show a certain idea or concept cep um you know I try to simplify as much as possible on our YouTube series so that it's accessible and I think a lot of people like that um but sometimes there's questions that people have that come up you know how did you know to do x what is what is inspiring uh why and I want to answer those uh live right so I want I want us to be able to to kind of walk through that together um I'll try to Showcase some of the way that um you know different workflows can accomplish objectives creative objectives um I'll go through a couple of the you know different techniques that you can use like control Nets and IP adapters and I'll show you that it's not it's not always perfect right there there are um these these kinds of conversations almost you have to have with the the tool to understand how is it looking at what I just gave it how can I help it understand what I'm looking for and how can we craft ultimately that that final asset that we want to create um so we are going to start with an idea of kind of the world that we're building in and just to kind of like you know uh I did I did improv theater for uh a couple of years so I'm ready for this you all give me what you want I want to show you that this isn't something I prepped for um you know we can kind of uh we're going to do it live and it'll be a lot of fun because because you'll see me kind of like struggle and work through how to get to that ultimate uh goal of what we're talking about from a um that kind of world and IP so you know we'll we'll focus on the idea of creating NPCs for some new world some new game um you know we were we're trying to aim for this process of ideating what might we be going for iterating on that concept and then really hitting a couple of key points on consistency how can we get that uh to to come through and generate some final assets that we can use as an NPC we are going to obviously record this and post it on YouTube um and we will also um kind of showcase how we get to those assets and share some of the process uh along the way if you see something that you're interested in like I'm going to use probably a a file that uh is is kind of a nice initial image template if you're interested in that I can throw it in the chat um but you all can can really just ask questions along the way um I will try to keep an eye on the chat as we go through and yeah we'll get started um so I know one thing that people are asking about is uh access to public models so for the sake of just being able to follow along I'll use Juggernaut uh juggernaut's pretty popular model right now you can download it from your favorite model Source uh wherever that happens to be uh and we'll just use uh that model I I might need to fight a little bit with the prompt to to get it ironed out um I've got some ideas on on things that we can use to get uh some good assets but uh follow along ask questions and um you know ask ask for what you need to see to help understand the tools a little better um So today we're focusing on NPCs uh I'm going to go ahead and open the floor for some suggestions from the audience I've got one already which is spaghetti demons but going to try to try to avoid the uh the spaghetti verse if I can uh despite that being a fascinating uh World um and we'll go through and kind of create a couple of things um so I'll give you guys just a second here uh to to to think through what world we're going to create if if I don't get one from you I'm going to spin the DIYs in chat chat GPT and see what it gives me um some some questions that somebody asked are these tips going to be generally applicable to other front ends or mostly specific to invoke I think it'll probably apply to most front ends um the the process of using the tools to get to the outcome should all not be exactly the same like workflow and the sense that like the UI is going to be different um but ultimately you'll still kind of get you'll get to the the same place if you use the same types of techniques um okay so I've got a couple of suggestions here we've got like old Fantasy Castle uh we've got medieval uh dragon born uh steampunk modernized Greek Gods uh I do think modernized Greek gods would be kind of cool um primarily because that's uh there's there's a a fun uh there's a fun campaign uh or world of like magic the Gathering Theos uh which funny enough is uh Peter morbach one of the artists that is on our Advisory Board actually created Theos so that one's actually a a fun uh fun idea maybe I'll take that one and run with it um yeah let's do it we'll do like kind of modernized Greek gods and figure figure out how we can do that um it'll be an interesting Journey because uh it'll also show I think this is a really good one it'll show how hard that's going to be uh and the reason why it's going to be hard is when you prompt for something like Zeus there's a there's this concept of Zeus that is multifaceted right there's the Zeus that we're all familiar with from you know Greek myth uh you've got statues of Zeus uh the kind of like old uh dude with uh you know white white beard kind of big Burly guy that's kind of the the mind that uh or the mental model that everyone has of Zeus we'll dive in and try to figure out how we can get um some of those gods in kind of like a modernized setting um maybe kind of reimagine what those Gods might look like um and this is also an area where maybe I'm not going to prompt for Zeus because my idea of Zeus is in this modernized world maybe it's different like maybe I don't want to do as many call backs to the old idea of Zeus and I'm thinking about you know kind of uh uh the modern Lightning God in some sense and who what would Mount Olympus look like in the modern world and all of that kind of stuff so that could be very very interesting um we'll focus today on more of an artistic concept art style like what you would expect to see go into something like a video game uh photo realism is something we can definitely do in the future I see a couple people asking about that we can talk about photography and inpainting you know photos and stuff like that which is a use case that a lot of people play around with um but we'll we'll do the NPC today and see see where we end up um so the first thing that I'm going to do uh now that we're kind of focusing on Greek gods is we need to get a couple of ideas down on paper and a couple of Concepts that we like I think when I'm playing around with an idea I very rarely like prompt and get the final thing and then I'm done with it because when you have an idea when you have a concept that you want to make real you want to use this as a tool rather than as just kind of like P Hawk giving it everything is very often you won't get what you really were thinking about in the first place if you do that so we want to kind of control this in some way um we'll talk about uh controlling for different specifications as well um typically when you're creating an NPC or concept art um you you have something that you are trying to map to your pipeline right your creative pipeline you need a certain specification you need the character to be in a certain pose or in a certain spot um and so we're going to try to control uh a lot of that generation process uh use the tools at our disposal in invoke and we'll play around with that and see what we get um so let's go through uh that process what I could do first and maybe what I want to do first is come up with a prompt and play around with getting consistency in some amount of that style through a prompt first before we start controlling it and getting the like specific characters that we're looking for um so we'll do um NPC concept art award winning uh and painted style uh typically very helpful when you're crafting a prompt to use more than just like basic stuff like this um a couple of things that I think about when I'm crafting a prompt is what are the stylistic elements and and uh the Aesthetics that that we're going for what what is going to create depth and what is going to create the type of asset that we need um I'm going to go for uh probably something that's a a little bit uh more on the finished side of an asset you can kind of do some flat color uh NPC Concepts and line arts stuff but I'm going to kind of go more towards the final asset just so that we can kind of play around with that um and so going back into art history there's like a lot of terms if you I bought an art history book uh from Half Half Price Books uh and just kind of went through and like picked up a lot of new terms and in the uh the lore of art uh and so things like uh Kos skuro uh lumine and pastos uh these terms kind of pull out and evoke different art styles that I've I've typically found uh pretty cool um medium uh we'll do digital painterly art uh I like to throw in a little bit of like oil painting maybe oil and acrylics uh and then we'll do Zeus we'll just see what we get Zeus as a modern what what what would Zeus be in the modern world he'd probably be some sort of like head honcho uh Mount Olympus maybe he's like a banker uh he owns like a big Bank not Olympus I don't know we'll see what we get uh negative prompts you know there's uh pretty decent set of um you know just use distorted blurry typically I like to use a a counter art style like Photo um just to kind of really separate in the latent space where are we focusing and we're focusing on like more more art um so we will give that a shot and see what we generate um I didn't have that model loaded up so it'll take a second to um uh pull in and then we'll we'll go yeah somebody somebody's calling out American Gods I'm a big American Gods fan um so that that is fun uh I hadn't I hadn't made the connection that we're effectively like recreating uh that that Vibe but um I think it was Thor in American Gods Not uh not Zeus but it's just kind of all all connected sad the show got canned okay uh so we've got Zeus uh as you can see this is this is where we get um if if we're pulling out something like Zeus we're going to do a lot of callbacks that we may not want to that concept of Zeus from mythology he's got these like coins on his chest he's got this big beard it's not really like modern in a traditional sense uh it feels like very historical in some sense he's got like this this money on the table so what we're going to do is we're going to change this to um uh Brony elderly man Whitebeard uh well-dressed banker and we'll even do like futuristic to give a little bit of like a uh we don't really want futuristic maybe like modern style dollar dollar bills CEO Zeus here we go so we've got we've got this uh this gentleman here and I think we're getting a decent style out of this uh I could I could buy that being Zeus he looks a little bit more elderly than I want so maybe um we'll just do a bronny man instead of elderly we let get rid of some of those wrinkles the thing that I want to do now though is you know this is helpful if we're doing character portraits and I probably would you know take this and uh we spin up another set of character portraits if we want but really what I want to do is I want to get this to be like a full body concept art so I can actually use this as an asset um so we'll do full body NPC concept art and then what I'm going to do to control the generation is go to our image to image tab um one asset that I pulled in um is this kind of white background with a darker Center and on the imageo image uh YouTube series where we talk through uh the basics of this this concept of like the image to image Den noising process the the thing to note here is we're we're kind of manipulating that initial noise that this process is going to run on to focus more on the center with this as our input so it's going to have uh kind of subject matter closer towards the center and white closer on the outside typically when you do this type of um process and you use something like uh white background we can get a little bit more control over where in space like where in the composition our generation's going to come in and so in this case um we'll take this over to our initial image and we'll go ahead and invoke and we'll see that shape kind of gravitate more towards the center now one thing that can happen is you might get uh ahead uh if you haven't really focused uh the The Prompt specifically on like a full body um but you can see we've got kind of like our our modern Zeus walking in an awkward way uh towards the camera so we might do standing pose uh and we will do Be A Plus on the white background just to make sure that that's clean and we'll generate another one all right so he's now standing he looks like this um he looks like a modern Zeus now I think the question that we have to answer now is how do we how do we incorporate uh the notion of like Zeus now this could be just any old um old guy here right and how do we like incorporate some of those electric elements or lightning elements um which we can start to do with a number of techniques um maybe we'll just try prompting for it um trying to figure out let's do like a lightning approach lightning bolt shaped maybe a belt bu fun do the Bel Buckle here see what we generate we might have to do this in in painting which is where it gets real fun I can see that he's got like this like fanny pack thing though on the side um I don't really like the fanny pack we're going to get something else here I think we might get that lightning bolt tie that somebody was asking for on this okay yeah I I uh I kind of like the the funny pose um we'll we'll kind of fix this up though because we're going to need to do some canvas stuff um what I first want to do is kind of go in and fix this face I think one thing that will happen when you using like base generation models is if you get the the more distant assets um like character concept art um anything where the face in pixel space is kind of in that smaller maybe like 64x 64 block you get kind of like the details look like a little artifactory um and in the canvas we're going to go in and really just do that core scaled in painting workflow um with the UniFi canvas does just as a reminder for folks who don't know about it we're going to kind of zoom in to a certain spot the area that we're focusing on is really within this bounding box and we are going to allow it to regenerate that area at a higher resolution and then take those details and bring them back in and that's because the AI models are able to generate really good details when you have a larger image but when you get more of these distant elements they can get a little bit um a little bit bit funky and you really want to focus in on the areas that you need to look good um so we'll take that in and we are going to decrease our D noising strength which is going to um allow us to keep more of our original structure here again that's kind of going through that image to image process uh and we'll just regenerate that face and while this is generating I will look at a couple of questions uh expanding the advanced options from the model uh we can look at the scheduler the steps the CFG scale I will note that I'm on a beta build of diffusers um which means that uh I've got a an update that is going to change how some of these schedulers behave um as of right now if you're using Excel I would use Oiler um and I'll type that out Oiler or Oiler a right now um DPM Plus+ 2m and all of the other kind of like Plus+ models or schedulers are um they're getting kind of like an enhancement and update in the next diffusers release which will be in the next invoke release as well um yeah I would I would not I would not over index on schedulers and kind of like models being optimized for schedulers I think most of the schedulers just have a different way of sampling noise and you'll be able to get pretty decent results out of many of them um in this case uh you know if you want to just see what will happen if we use Oiler ancestral um we've got a pretty decent face here I you know I like it um but we'll use Oiler uh for the rest of the generations just so you can see it's it's going to generate largely um similar results with maybe a little bit more um a little bit of a softer look to it so when I'm focusing in on this area what I really need to think about is what is the AI model looking at um what is it seeing and what is the prompt that I use need to describe so that it makes sense um right now because the prompt is focusing on the entire image it's not really helping the model understand what exactly are we focusing on in this box and what is it that I'm being asked to to regenerate um and and so in this case we want uh lightning bolt shaped uh roach and one thing that I also believe helps quite a bit is using an image reference here um we'll take out uh we'll do closeup of a tie or close up of the chest of a wellest banker we'll take take our IP adapter and we're going to focus in on this region and I'm going to take the uh image clip so that it is effectively I'm telling the IP adapter look at this region um so we'll take our uh sdxl IP adapter and it is now focusing on that clip that region and I'm going to decrease the weight of this to say 0.5 or so it should largely be the same and then I'm going to mask the area that want to regenerate I'll probably give two or three uh options here um and while it's generating we'll answer some questions um these are being recorded so we will post this on YouTube and answer those questions later um or we'll answer additional questions later but if you if you watch the recording and you have questions you can feel free to shoot uh out an update on Discord so I'm going to can cancel this because I think I probably need to just based on looking at this increase our D noising strength uh so we're going to increase that D noising strength up to 75 or so uh and this is where things get a little bit of little bit fun in the creative process we have to figure out how to get uh the shape that we're looking for to be generated by the system and this is where I think we're probably going to need control n um because it's just kind of going in and doing its own thing uh which is okay it's okay um we're going to do that we're going to take this uh cancel that out and we're going to focus on this region take a clip and we're going to just draw in the shape that we're looking for here uh what we want to do is kind of close this up and we'll take our yellow and we'll give it a nice lightning Bol shape and really what I'm what I'm trying to do right now is not necessarily get everything that I want out of this uh from a color perspective I'm trying to compose the color into regions so that when it's going through the D in process it's kind of picking that up but I what I also want to do is on the control net when I pick up this region I'm going to use a cany control nut I'm going to import this in and it's going to pick up that edge right because I've kind of gone through and made it really high contrast it's going to pick up that edge so when it's going through and regenerating this it's going to constrain to that shape that I've drawn now I just looking at this I probably need to to fix some of this here so it doesn't pick up that as an edge maybe blend in a little bit more some of these colors and then we'll take another clip yeah so now we've got a really clean lightning bolt there in the control nut and when we regenerate this we'll have a lot more confidence that it's going to regenerate it in this shape and with these colors um we still got our IP adapter up we still got our shape there and we'll pull our D noising strength down so that we're really focusing on this this concept now I will call out uh if you a professional artist and you're working on this level of detail you're likely popping this into Photoshop and just sketching it out yourself this is this part of the workflow is probably a little bit more of just showing the capabilities of the tools and how to control it um most professional artists what what they'll do is they will generate some of these assets um uh in individually and then composite those in Photoshop so that they can kind of pull those together into something coherent um if we zoom out now we've got kind of like our lightning shaped symbol he looks a little bit like a super superhero Banker um which probably fine that's I guess that's effectively what uh a modern Greek God would look like anyways uh it's a little bit like a superhero he one thing that I'm like noticing now though is he's got uh he's got a vest and he's got this like belt outside of the vest uh I'm being told that uh brooch is actually going to be spelled differently which may may help that generation process too um thank you Devon yeah or JP uh both of you guys calling out so I mean we we controlled A lot of it with our generation but I'm probably going to do a little bit more and I'll take down the strength of our um control nut so it's got a little bit of flexibility towards the end and we'll just do three more now we got the the puns are coming out bring bring the subject broaching the subject and we'll get rid of this like belt too I don't want to focus too much on Zeus alone because then we won't have any fun with any of the other GS uh but we'll fix up this like belt thing and then we'll uh say Zeus has been solidly composed all right I think I'm going to take that one it looks decent all right so we're we going to control this guy and get him fixed up uh we just go ahead and paste over everything that we don't like and I'm going to pull the prompt out of the original image that we created because that'll make my life easier um I just pulled the prompt out by right clicking on that and I think we can just I don't think we need the lightning bolt shaped belt bu anymore but we'll leave it in there and see if it honestly I don't think you should probably have a belt at this point um so we probably just want to take that out and regenerate and we need to take out our IP adapter too and our control knot those will cause some problems all right there we go I think we'll take that it's good enough for government business anyways so I'm going to save the whole picture out and then what I'm going to do is I'm going to uh the delete stuff that we didn't end up using I might keep this one around just for the prompt um and I may go ahead and create a board for stuff that I'm Keepers here we'll call them the keepers so I'm going to take out kind of like the final assets and we'll use we'll do our Generations in the NPC board and then on the keepers board we'll kind of use the the full set so what I may do now is I may take this image here as an IP adapter uh we'll pull it in do I want to use that as an IP adapter it'd be useful in controlling some of the style but it might pull in too many of the elements like the kind of business attire so maybe I won't do that I'll just use our standard um prompt here and I'll kind of drive that consistency with the prompt one thing that I'll call out um and this is you know we're we're dealing with prompts right now when you're a professional team that is doing this type of work more often than not the best way to drive consistency of styles with a custom model um we have not really touched on the training process yet um we've not touched on like how to create your own custom model but I think it's worth calling out um you know we've done some work with uh game studios where for example we'll take art from their concept art pipeline that has has a very specific style it also has a very specific structure um you you typically produce things like orthos where you've got um the character kind of looks like uh they're they're uh in like a rigid pose it's kind of like a mannequin pose because it can show how to to model that character you can have all of that inclusive of style infusive of pose controlled by a Laura model and then you're not really worrying about about prompt for style consistency you're just focusing on the things like what are the the specific elements that we want to see in this image and if there are ideas or concepts that you can train into a model that's typically going to help you better understand how to use the prompt language because you're tuning that language to what you're using in your workflow um but it also just helps from a consistency standpoint for now we're just going to use our prompts and we'll kind of go go about it that way um but we'll maybe go for maybe Hades now um what would a modern Hades look like maybe like a biker right um biker with we say like a spectral biker with a helmet shaped like a skull pretty cool right uh leather jacket black leather jacket modern style okay so we'll do again we're doing this kind of like initial image as our um template if you will to kind of keep everything focused on the center uh there's a couple of other templating uh techniques you can do you can obviously do something like a control net at a low weight where you've got a character in the middle in a pose if you're doing orthos you can like have the the pose that way and you can control it uh you just are going to want to make sure that the weight is low so that it has flexibility it's going to be relatively low like 2.3 on that weight um but this I typically find has uh a little bit more variability when we're trying to get these types of these types of poses out um so we'll and generate and see what we get uh oh modern Hades totally a politician that is that is true could be good um system specs that I'm working on I'm working on a 4090 uh so that is a question that comes up is how is it so quick it's I've got 24 gigabytes of vam um there are I would say other ways you can get speed um other than upgrading your car card uh you can use cloud services u and v has 24 GB cards on our cloud service um typically I think you'll find if you've got anywhere [Music] from like 8 to 12 gigabytes things will run pretty quickly for you locally um we're definitely getting more of like a skull like literally just a skull uh as a helmet which is probably like less the vibe I was going for than just kind of like a um biker helmet so we'll do a biker helmet and then I'm going to do skull themed and I'm going to use a minus on the end of that so that it's not creating just like massive skulls um it's definitely more Skeletor than Hades the and we still might be getting more more skull like skull there there's this like philosophical uh interesting ele element here with with skulls because skulls are so prevalent in art the word skull is like really really powerful we as humans are like fascinated with skulls and so you get just massive massive numbers of skulls um so we're going to try to like really downplay the idea of the skull and hopefully we can just get it to be a little bit of like a light theme on top of a biker helmet um I I still think this guy's got a skull coming through here uh I'm going to cancel and clear all one thing that um you know maybe maybe the easiest thing to do is let this be uh just the foundation that we go through and in paint over um that might that might be the easiest way to do this because we're going to get a lot of those um underlying Vibes just from the shape and color of this and I can probably just have that come through that way uh biker helmet so we'll turn this down like 7 or so got a good biker now I'm questioning myself on uh how how much this is going to look still like a skull uh somebody asked where my three iterations are going what I'm doing right now is I am generating uh I'm generating three in queue but if I see that my first generation isn't working out the way that I want it to go one of the benefits with you know being able to see the progress image is that you can kind of get an idea of what it's picking up with your prompt and you can cancel that out so you can save time uh so I'm I'm cancelling those uh with three the latest version the cancel current item button is a singular cancel but if you hold down the shift key that'll just clear out the entire queue and so that's that's what I'm doing right now um I think this one might need a little bit more uh Den noising strength to get away from the skull uh one one thing to to also call out with sdxl and in painting is you're really using quite a bit of the D noising strength to to kind of battle against the the way that the sdxl model was trained um this sdxl model had a refiner on the last 20 to 30% of the steps and so you're you're kind of like trying to do the mental math of what that denoising strength ought to be um this is definitely not the style that we were going for so I'm just going to kind of ramp this up and see if we can start getting this to go where we want it to go otherwise we're going to have to take this a completely different direction and start in painting on it and see that one's too strong this this 0.9 and8 Den noising strength threshold is one that you'll find with sdxl um and maybe this is not interesting but I I think I think it's like a useful tool to recognize um right around 08 to 0.9 you'll start to see that it goes from maintaining the shape and the structure to abandoning it completely and kind of regenerating the image so if you do this at like 0.9 or 0.95 similar similar to how we solve with the uh initial image where we used that kind of like Blurry Circle how it's able to create an entirely new image that doesn't look like a blurry Circle at around 0.9 it's because that's that's the uh point in which the sdxl models will start to kind of really do the composition piece so pulling that underneath can um help get a little bit more control there so we're going to pull that to 085 and we are probably going to do just a couple things here close up of a biker with a biker helmet uh we're going to try to see if we can get this to not see the skull it it wants to see the skull uh it wants to see the skull in this helmet there we go I think that's a little bit better uh looks kind of like a little bit like a skull yeah there's a good question um in the the chat of could you add a previous character as an IP adapter to get the vibe from that I think the answer is yes uh there there are some useful ways to pull that out and and maybe that's something that we should um kind of show right now um I might take this one for example uh here and say that we're maybe maybe okay with that I probably need to compare the Styles here I don't think this is too close to what we generated here before this one looks a little bit more like a rendering like an unreal render and this this has a little bit of a different vibe so let let's actually try that um I think I should just take skull themed out we're going to we're going to work on spectral and hopefully to give us some of our Hades Vibes that way I'll pull this into an IP adapter and we can kind of take a look at how that works so the the different IP adapters that exist you have your base level IP adapter which is going to pull out concept pull that style and try to like pull those together into the new image it's not strict and it's a little bit looser if you do IP adapter plus you are pulling out a lot of the composition the structure uh colors of the image it's it's effectively saying like that image is what I'm using as the prompt and I want to generate something like that image rather than be inspired by it so for this use case of we're trying to generate a biker but we're going to use an image of our like older uh Zeus Banker um you know we'll pull that down to 35 or so um might even need to go lower and we'll see how we can push this closer towards the spectral biker with a biker helmet even though we're kind of using that underlying um concept and I think this actually gets a better style um I need to move over to the image to image tab so we get the same structure uh but I do think this helped with like getting a closer style so that's um that's actually super helpful and generate this this way and I think that'll be useful as you can see one one element of this that you have to be mindful of and definitely got a little bit of that Zoo spirit in um when you're switching between these different workflows you're kind of bouncing between different jobs to be done um in the sense that you're using different tools to get to different outcomes um so it's important to kind of be mindful of the different options that you're switching between on the options panel um we've got some enhancements that we're anticipating will help with that type of work flow switching uh coming up in the near future um but in the meantime there's uh a lot of things that you can do to to control things you need to kind of keep an eye on now this this guy right here I I really like as our as our Hades uh biker this guy picked up the and you know this this kind of where I was calling out if you use IP adapter you can get the style but what you need to be mindful of is it may pick up certain elements like you know giving our biker a tie uh or a beard this one however is a really really good example of where you're able to pick up some of that the stylistic elements and uh pull that into a new generation and so we can take this we can kind of fiddle with it a little bit uh maybe we for example want to change the tint or color of this um and so we can take this maybe get some like spectral colors make it a little opaque um switching to the base layer and then I'm going to like map over this with the new color um maybe a little bit of green here as well and we'll get some highlights on the top then we'll regenerate this at a low D noising strength maybe something like 6 or so kind of recolor those those vibes um negative keywords could help get rid of the skull people did call that out that probably would would have been an easier thing to do um what one thing that is um definitely helpful is when you're doing a negative prompt it'll eliminate the concept what we were trying to do and where where this is you know kind of a a creative task is we were trying to get inspiration from that skull right we we wanted the skull we just didn't want it to become like a full skull we wanted it to to kind of Be Inspired ired by uh the skull and that's why you wouldn't want to necessarily put it in the negative prompt um so it's it's it would have gotten rid of the skull but it would have gotten rid rid of the skull completely um and that's maybe not something that we would have wanted um this one we've kind of gotten to a good concept here I think this is like a good one for Hades um we could probably play around with a skull belt buckle if we wanted to but I think you kind of get the concept here of of we can kind of create uh different ele ments already did that with the the Zeus character and I'm being mindful of time when I get to our last uh last god um so we'll save this to the gallery um one question that came up was are there limits on IP adapters um I think the the question there is maybe like what are the what's the extent you can use IP adapters you can use multiple IP adapters uh in the same generation so you can kind of like take two disparate Concepts and fuse them together um this is a technique that you know most people would call blending or concept blending and that would be um you know let's say I've done I've done a robot uh when I'm when I'm trying to create like some kind of like robot spider character I've taken a picture of a spider and a picture of a robot and used those to varying weights to kind of get that overall structure of the spider but the uh style of the RO and then you can kind of play around with it that way so a lot of a lot of playing with this technology and and creating with the technology is understanding how the prompts that you're giving it including your own sketches your own style uh those are all tools for you to control the output what most professionals are going to do in this workflow is they are going to establish and create a couple of useful workflows and patterns that fit fit to their process so it might be you know take uh a couple of Concepts that I have and create a character uh that is in a certain structure or in a certain pattern like an like NPC concept art and I'm going to use that workflow over and over again right I'm just going to kind of like uh regenerate for all the different jobs that I need to or the different characters that I'm creating I'm going to use that same pattern and that same workflow and and part of this is really nailing down you know what are the set of inputs that help you control that to get to that final asset that you're trying to deliver could be IP adapter images it could be um control Nets it could be an input image all of these are really just various inputs that you're putting through the process in order to get to that outcome now this is where again a trained Laura which understands a the prompts that you're putting in because you've tuned it on your language and the kind of the style you're going for your your purpose building a Laura for a job right is to do a specific style to focus on a specific character to you know really emphasize that concept and so that's like a helpful thing to remember is as a creative depending on what you're going for you have all of these tools at your disposal and you want to create that workflow that really fits with your pipeline and your process and then you can just use those um again and again so we'll use maybe the IP adapter from both of our Gods to get to our final uh image we'll do those uh again at like a low. three uh weight um we see we got our Zeus over here probably need something a little bit brighter um see what are we going to come up with for the other gods maybe Artemis Artemis is cool she's an Archer that be in the modern modern day that would be maybe a little bit brighter in a concept to distinguish maybe Artemis is like a cyber Punky H hacker with like a lot of like retro Vibes uh why why because the modern like Frontiers are the internet something something that I know um you you can push me for it uh the original challenge is four NPCs oh gosh okay we better rush then we got five minutes uh so we'll do a uh cyberpunk woman uh hacker Hood hooded uh right neon deel and green monor style we're using using these two to pull in that style and we'll do our image to image so now that we've got our style for these two we're using that as an IP adapter we're hoping that this pulls through the style and not as much of the color um we should have uh decent amount of our color com in from the prompt here but we may need to do bright this is a little bit further behind we need to go up and do our denoising so that low de noising strength still created the character but it didn't have as much flexibility as it needed oh yeah so yeah the fact that we've got two Styles or two examples of our style that we like now and we're using that as an IP adapter we're starting to really get uh a little bit more consistency in that style when we're generating these new assets um so we might actually be able to reach our goal of four NPCs uh pretty quickly since we've nailed that down uh well you know I think this will work we we'll use this one uh I'm going to do just some in painting because we want to make sure the face looks good uh and we'll turn these IP adapters off because we're not going to need it for this oh no uh we'll go down in paint and we'll turn this down to 055 someone better be coming up with some good concepts for the fourth God uh let's see got got Zeus and Hades my Greek gods are like failing me Mars Aries one of those is Roman okay that's it that's it that's what we want right there look at that that is sick oh we like that we like that okay cool we're going to take that and we're going to save it and we're going to get our last God here uh and it is going to be yeah Mars is the Roman version of Aries uh Poseidon Apollo okay there's some good ideas okay let's let's think here how do we get Poseidon sidon Mars which one do we like better trying to think what like a modern water God would be it's like danani the the god of danani uh maybe he's like a yeah he's like a marketing executive executive that sells bottled water uh okay Apollo a sailor okay that's obvious right it's a sailor um a modern modern sailor it's like a yacht Captain um all right we'll do that uh and what the hell we'll we'll go ahead and we'll add in our third character as just like double triple emphasizing our style uh and we'll do a an admiral yeah that's a good term right a stunning white suit black rated beard okay I'm just getting weird okay we're going to try it uh we'll go back to our image to image we'll increase our D noising strength and we'll go forward and have fun I don't know if it's going to have a braided beard I don't know how many I I don't know how many people are braided beards but I'm trying to get a little bit a little bit more out there with our uh our God this one this guy looks a little bit more realistic than I think oh you know what we don't have our IP adapters on aha so IP adapters they do help they do help Navy SEAL it's a good one I'm going to turn off the progress image so we can look at this one real quick see what we got oh yeah he got sunglasses on it's our favorite our favorite Admiral ever okay yeah we we get get a little bit of that we're we're pulling in a little bit too much our Banker though I think he's got like a tie and stuff um let's do the glasses are coming into let's pull this down a little bit pull this down a little bit that oh I'm like a minute over so I'm going to be be chastised for violating the the hour the hour length but I'm going to do it anyways we want to get to that final that final Admiral I think that style works out pretty well just really want to fight fight against that tie though I hate that golden tie on Poseidon needs to be some other color it's buttons are also though a little bit of work so we'll take one of these three and we'll fix it up in canvas real quick and then call it a day this guy is the best take down going to pull this out and when we're doing in painting the reason why we can you know do do it without the IP adapter as much is because it's got some of the context from the image when it's generating um so it's able to to solve for some of the style questions um so we're going to take that guy out um want blue like blue a dark blue tie and maybe we don't really want the sunglasses so we can do some of this I'm just going to rough it in and keep the denoising strength pretty high yeah all right do this at around 75 see what we get green Ascot to The Prompt probably do pretty well so we've gone in and we've kind of fixed up our Admiral got rid of that golden tie so some kind of like gold ornamentation but we could go in and fix that as well if we wanted to I think we're in a good spot though he looks pretty clean uh so we'll save him out uh and we are going to I realize I started generating in the keepers section so we're just going to move everything that we didn't want back out so we've got our four here we're going to keep go through our characters and talk about our consistency see if we like it we got our uh hacker Artemis we've got our Admiral Poseidon uh went to the wrong tab uh went to our Hades biker and then we've got our like Banker Zeus uh so we've got a really strong set of NPCs the Styles relatively consistent between them um again we touched on using things like control nut on the canvas to to guide some of the Generation pieces when we're trying to do very specific shapes structures things like that um you start to see a lot of how image color and structure can guide both the imageo image process if you're doing a massive image transformation you can use things like you know templates and stuff like that um there's a lot of ways that you can if you switch back and forth between invoke in something like Photoshop where you have some of those creative controls and you're you know like a wake on tablet you're just sketching out and adding all of these layers on top um that's a really really strong way of composing the control images that then come in and generate with invoke invoke pairs really well um with that that type of natural artistic process that you're already going through as a creative um so that is it for today but to talk a little bit kind of like uh close us out uh first thank you all for joining uh it's really fun to do this with an audience as large and as involved and engaged as you all it's really really cool um we are going to do more of these so we'll start to do this on a pretty regular basis um we are growing the community quite a bit um I think people have started to to sense that um if you are interested in in helping either on the coding side or on the kind of community aspects like um helping with education with uh teaching people how to use the tool things like that um make sure to to check with uh IMC he's our community Wizard and you know does a lot of that uh planning and stuff um also share what you're making uh both here in the the Discord we've got the Outdoor Channel but share it with the world right I think um we love to see every what everyone's doing on uh social media it helps get the word out it helps build the community um share share the ideas and feedback that you have on this session and how we can make the next studio session uh even better and if you learned anything today uh if there's like a cool tip trick I thing that you didn't realize but you had that like aha moment uh just share it in the tools and tips chat I learned XYZ I didn't know that before because it might teach someone else that concept that might help them understand it um so that's it for today we'll go ahead and conclude it but thank you all for joining and we're going to do this again next week see you then bye

Channel: Invoke

Views: 4,429

Keywords:

Id: 8p92pqHU9Ag

Length: 65min 36sec (3936 seconds)

Published: Sat Jan 27 2024