Real-Time Painting in Krita: Unleash Your Creativity with Stable Diffusion LCM Models

[Music] hello and welcome back to Alchemy with zero phase this is Eric and uh I got some I want to show you that's just really cool and exciting so I was watching some videos the other day again these are this is a fairly uh new thing that because of the lcms that came out that allow fairly fast or very fast uh renderings uh in stable diffusion uh they've updated the stable diffusion Crea plugin to use those now um if you're not familiar with this plugin I used it a while ago or at least I think it was a version of it and um so it's called Crea AI diffusion and maybe this is a new one that just came out I'm not entirely sure but uh I started watching this and I just like I got goosebumps because you can do real time painting on this and actually watch things change and if you want to see what I mean here let's I've got this image up right here and let's say we want to add something a little more starlike in the background here let's go like this I got it in scribble mode right now so we're just going to go all the way around and there you go I mean had some really cool stuff allows you to a modify things on the fly as you do this um you can go through and I use the just changing it to a white to erase things and uh you get an idea of what this kind of what this is going to do and and you can change the prompt on the Fly too uh surrounded by Magic and it adds a little more magical feel to it so um yeah this is very cool so I'm going we're going to walk through real quick how to get this how to install it some things to watch for and be aware of and just some Basics to get you started this this plugin has a ton of functionality uh because it's part of Crea it's a full-blown photo editing uh digital modification program that gives you a lot of capabilities now I've used cretive before I use I kind of go back and forth depending on what I'm doing gimp's got some nice plugins for doing batch processes on many different images and it's com in very handy when dealing with a lot of my AI rendered images so uh when you get to the GitHub again just do a search for Crea AI diffusion you'll find it looks like this kind of got a yellow icon you're going to come down here and they have everything kind of organized so you can go you know to the features download installation whatever and uh we're going to come down here show there's the features and we see the hardware sport we get down to installation okay first thing you want to do download the latest version of crto okay you can do this from directly from their website I use a program called ninite that allows you to install a lot of different software but uh then go to the download page download the plugin it'll be a zip file okay you're going to to extract the contents of that folder directly into the root of the py cre folder so there's under your users username app data roaming cre and then py Crea which is where the extensions go for cre you're going to extract the contents of this ZIP file directly into that folder okay um should be let's come back [Music] here you know I could go to I'm not going to because I don't want to show you my username or anything so anyway you get what I mean just follow these instructions here okay uh Linux Mac now um we're going to go into cre you go into cre and then you enable the plugin okay you're going to go to uh settings uh configure cre and then you go down to down to the bottom here python plug-in manager okay and once you have the plugin installed it should show up here as AI image diffusion okay you're going to put it check next to that click okay and you're going to want to restart cre okay just close it open it again okay once you do that then you got to add the docker super simple just go to settings go to Dockers and put a check next to AI image generation okay what that'll do is it'll put a a Docker over here on the side so let me let me kind of move this out of the way I've got mine separated I like mine floating okay it'll put it over here on the side with the rest of these tools so it might look something like let's see if I can put this over here I'm not sure if it'll let me I haven't I've expanded too much but it'll put over there you'll get these tools over here okay it'll start up um looking like this okay over here you can separate like I said I would recommend separating if you get because sometimes these tools over here get a little confusing cre is a little messy that way um and that way you have the ability to kind of move it around okay once you get this Docker you're going to want to go into the settings here there's a little gear box right here okay in here under connection you're going to want to select now I'm using local server managed by creat a plugin okay I wanted this to be as easy as possible so I'm letting this manage everything okay if you already have comfy UI installed you can go into the instructions on their GitHub and they they will show you what you need to get that up and running if you want to use your own local installation of of uh comfy UI uh it doesn't use automatic 1111 when you let it manage itself it downloads a version of comfy UI and gets it all configured for you like I said this was the easiest way to go for me I'm not a comfy UI user I use automatic 1111 I got a lot of people on my Discord server who do use comfy UI they love it they swear by it I've dived into it a little bit um maybe I'm hardcoded for automatic 1111 maybe I'll get used to it a a little later either way so you select I'm for the purpose of this video we're going to do local server manage by create a plug in and down here you're going to have a whole bunch of options okay um select whether you want to use a Cuda uh Nvidia GPU direct ml GPU or run on CPU okay most of us are using Cuda Nvidia GPU okay um just make sure you have the latest video drivers for your uh your video card installed should work okay core components will install that automatically so you don't need to worry about that under workloads you can select which versions of the uh stable diffusion model you want to install um I just selected both I like having the option because this is using control net some of the options for like scribbles uh soft Edge whatever they don't quite have that available for the XL model but you'll figure that out okay uh you can come down here and select uh um a couple of pre-selected uh models realistic vision is a great one dream shaper is a great one and Juggernaut are both all three of those are great checkpoints okay uh I did select the upscaler um again I just kind of went through and selected most of these the only ones I didn't really select were the depth and normal map I don't really use those pose I was going to try because this does have the ability to use the uh the pose models to uh create these images okay we're not going to get into that in this video segmentation and St and all that other stuff so um and then here's the XEL control net models okay once you get that installed okay you should see an install button it'll be up here where my launch button is you'll see the install button just hit install and sit back okay it takes I want to say on mine it was like 10 to 15 minutes okay because it's going through downloading everything installing and configuring everything including these models which some of these are about 6 gabt okay once that gets done you're going to want to uh close cre again reopen it go back into the plugin okay through uh again the docker which should already be there you're going to go back in here you're going to go back to connection and hit start or uh what does it say here it says launch okay um I think by default after it's been installed every time you start cre it it just automatically starts okay uh there's some other options up here that um we can get into like you can put in a pre prompt like this would be added to the beginning of every prompt that you do okay as well as a negative prompt you have some other options here the sampler you can change your sampler here uh the number of steps that it's doing I would say just leave all these as default for right now come in and mess with it later on if you want um I don't think it's really necessary to mess with it uh the LCM you could probably bring that down to four steps but I'm going to leave mine at eight or excuse me six uh there are uh some other options here I'm not super familiar with I'm not going to worry about it okay those are what you need to get going once you get that in you hit launch and it says server running connected you're good to go okay then you can come in here and start working on stuff now this was a little confusing when I initially got into it on how to get this going up in the upper left corner you have what looks like a paintbrush kind of a magic paintbrush and uh that's for um just generate okay I I don't mess with that one I like the Real Time stuff there's an upscale option here which uh is supposed to allow you to go and upscale certain areas add detail I'll figure that out later uh but what I like using is the live mode okay that's what gave me this here and uh we're going to go through I'm going to show you how to do this um on the strength bar that determines how strong the I don't know how to put this well let me put it this way if I by default it comes up at like 30 or 40% I don't remember exactly what okay and when it's at that level I'm going to hit play so when you switch to live there's a pause play button up here okay when it's on like this that means you got to hit play and it goes okay when this it's really strange I'm not exactly sure what the correlation is here with the strength bar when it's down to 40 you get this black screen okay not sure why I think there's some other stuff here like image we can go to image maybe it'll pop something up but um typically I'll go to scribble yeah there's the uh the image as it were uh came up but when you're under scribble or line art or soft Edge or cany Edge um this here I've learned to just turn it up all the way put it all the way to 100 okay and you're going to get some interesting results [Laughter] oh probably because I don't have a prompt in there and some of these models can be a little uh for what's the word I would use pervy so let's put a prompt in here we're going to do hell hound at the gates of hell and we're going to erase what I've got there so we're going to increase the size of the brush and just get rid of this okay going to start start fresh okay it's going to give you an image now I got the strength turned all the way up and I've got this on Scribble let's pull this over a little bit so we can see our full uh canvas we're going to switch this back over to Black switch our brush down to a moderate size okay and you can change the seed right here too um just hit random and it'll give you a different image like that's great that's a good one to start off with okay and what you're going to do is just start drawing so we got our prompt in there we're using scribble uh so you can just scribble start scribbling whatever you want okay let's uh let's start the head up here we're going to do an ear here's the head we're going to do this now Cerberus from my understanding has you know three heads but whatever do this we're going to do that I am not the best artist in the world doesn't really matter there we go okay there's the head head and we're going to come down this way do a tail come down here do a foot maybe that's not a tail I don't know uh let's add some shag to it okay see if it recognizes shag well it thinks that's fire okay just can of do this I am a master artist there we go there's the tail on fire um let's uh give it some teeth anyway you can kind of see where I'm going with this okay there we go see I mean you're getting this is GNA this is so fun to see this in real time I kid you not let's throw some rocks in here you know um just to kind of give it some ground effect there's your rocks I mean this is fun like the other night I spent as much as time as I possibly could I actually feel bad cuz I I feel like I was neglecting my my family uh just because of how fun this was now you can switch the model let's say we want to go to uh cinematic photo I'd like to see what that does here um because this ends up being realistic so uh let's switch this give it a second I want to see if it's going to refresh again here it's not giving me any air messages and if we move the strength down it's just going to go dark oh no there it goes okay so like I said you're going to have to mess with some of these settings not sure why it's doing that so let's that there we go oh maybe I'm not waiting long enough let's give it a second here change the seed maybe okay maybe the strength needs to be down just a little bit on this one yeah there we go so so we're getting something a little more realistic now you can switch over to the XL models um they will probably give you better body results you know body parts that kind of thing um dealing with hands or well maybe not hands cuz honestly 1.5 models and 2.1 models deal better with hands um and so there's so oh yeah so the the XL models are a little more stringent when it comes to the lines you're going to get the lines so uh what I found to fix that is by going down here on this that gives the AI a little more freedom to work with the uh model we're also going to increase this this is a digital image the strength back up to there to make it a little brighter there we go very cool okay so there is something I need to go into on this when you're first starting off um you're going to get a blank canvas obviously so let's go uh new we're just going to go create okay we get a blank canvas like this and you have your floating window okay to get the canvas over there what you're going to do is come up here you're going to select live that's interesting that it's putting that in there since we have a new uh thing up let's close this no we're not going to save that okay so we're going to play I'm going to change the model to uh the uh digital artwork XL we're going to turn this all the way up the strength all the way up to 100 we're going to put our prompt in okay um again let's do uh evil Easter Bunny with basket full of skulls not sure if it's going to actually give that to me but um this is just to show you the rest of so you get this in here okay you have to click this over here this is to add a layer to your image okay so you're going to click that it brings us down here initially it goes to image okay you're going to drop that down select one of the others here scribble line art soft Edge okay I'm just going to go with scribble again it gives us this and what we're going to do is click the little magic button here and that adds the layer over to the actual uh drawing uh uh canvas okay now it's blank so the AI is just going to throw a whatever image it wants to put up there okay and it's just set as the background so if you come over here and look at your layers you'll see you have background and control scribble when we added the background or excuse me the layer it did uh create this control scribble layer and that's the layer it actually works with even though this has background if you add another layer then you'll have the option of selecting the scribble one don't ask me why it does that not sure okay um and then you can again start doing uh doing your thing you know start drawing um if this is up to 100% with the XL we're probably let's see yeah we are running so let's see if it does any changes here says pause and play again let's see if we can kick this into gear here I wonder if I got something selected wrong anyway sometimes oh there it goes I just took it a second for it to come up you can see it's kind of made this little fatter rabbit here so this is scribble um I've messed with uh soft Edge a little bit soft Edge can be a little fun to work with CU um yeah see this one says not supported for sdxl okay so what we do is come up here and just select digital artwork which is the 1.5 model okay uh it actually put skulls in there yeah so you're you're getting an idea of how fun this could be um let's do that let's get rid of some of this other stuff here the soft Edge is nice because what it's doing is you're just giving it references to certain things you don't even have to scribble out the whole object it's really interesting like it just the AI just assumes things it's it's interesting how it works switch this back over here I don't know if I'm just making too many changes all at once or what but let's try just changing the seed here there we go okay do that you know I'm kind of wondering if I've got oh yeah yeah it's the layer the layers can be really weird on this so um as you can see I've got this extra layer up here so I'm going to cut this layer out okay so we're working on this actually let's uh let's just start a new one here let's go through it one more time okay so let's start with a prompt we got in there we got that we got that let's add that in we going to do soft Edge add that layer okay and Dr let's bring this down to 60% see if that gives us what we want want so as you can see it's putting the lines in there and again that has to do with this we're going to bring this down there we go it does you're kind of trying to find a balance because what this this is like how much strength control net has on it right and so and using soft Edge is a little different too but as you can see it's not too bad awesome so again this is this is fantastic if you have to go back and watch this again by all means uh go to nerdy rodent he's got another tutorial on this as well um he I think his was done a little bit sooner and so he tries going through a couple other things that I don't think is quite necessary everything else everything's handled pretty much automatically through the um uh the configuration for it so or I should say when you run the configuration here uh and you go to here and just select what you want to install use local server manager uh What uh nerdy rodent was doing uh was he was selecting connect to external server and using his comfy UI he loves comfy UI so shout out to him uh he helped me learn a lot of this so yeah uh this has been really fun too um you know in addition to learning how to put words and letters in to images this has been Next Level crazy for me uh definitely comment if you guys have questions join our Discord uh and uh like And subscribe the video and um subscribe to our channel for more updates we're going to start putting out uh some like AI news too AI image generation news there's lots of stuff going on right now uh one of the most exciting things that I put out a maybe you've seen it maybe you haven't a little news brief on uh stability or stable diffusion XL turbo they've got a model they're working on right now that can do uh singl step renders at full quality like it I I'm so excited for that to be able and you'll be able to do full High reses well I don't know if it's full High reses uh they give some statistics in there um but basically they can generate images like on a an a100 GPU in the cloud like I think it was uh what was it 512 x 512 they can generate single image in something like 207 milliseconds and it's just getting it's getting so insane on how fast we're able to generate these images okay we'll talk to youall later and hope you have a good day

Channel: AIchemy with Xerophayze

Views: 2,429

Keywords: Krita Extension, Real-Time Painting, Stable Diffusion LCM Models, Digital Art, Creative Process, Artistic Innovation, Artistic Enhancement, Krita, Digital Painting, Artistic Freedom, Real-Time Modification, Creative Tools, Stable Diffusion, LCM Models, Digital Artistry, Artistic Techniques, Game Changer, Artistic Creativity, Digital Art Experience, Artistic Vision, Digital Art Revolution

Id: 9UyNT_F18Do

Length: 23min 53sec (1433 seconds)

Published: Thu Nov 30 2023