Zero Training - “Instant LoRA" Tutorial for Stable Diffusion! | ComfyUI

instant Laura for stable diffusion Aces Styles objects poses whatever you like straight away no training required unpossible I hear you say well hello and welcome to More nerdy rodent geekery yes thanks to this comfy UI workflow from aloe vera aloe I can show you exactly how it's so easy to use with the most difficult thing being picking the images for your instant Laura and yes you can use this with control net or even slap in a bit of QR code monster so should you like thanks to the bonus workflows that you can grab for free from the very comfy nerd web page but more on those in a moment of course this isn't an actual Laura but you can use it in exactly the same way just without all the effort of actually making one while this video doesn't cover installing comfy UI I have already done one on exactly that so check out that video first if you're looking for help on installation links but all of these things are as usual down in the video description okay so how does all this work what can you do with it how far can you push it and what does it take to actually get it set up in the first place starting with the setup obviously you'll need comfy UI running and whatever stable diffusion 1.5 checkpoint you fancy if you've used stable diffusion before likely you will already have plenty of those if not there are loads of options available on civet Ai and hugging face I happen to be using this one here analog Madness from civit AI so should you wish to copy all these things exactly that is the model I'm using now as the exact requirements for these workflows will likely vary over time my suggestion is to use comfy UI manager to install whatever nodes are missing for whichever workflow you load seeing red well just click install nodes we can close that open up comfy UI manager and then just install missing custom nodes very very easy absolutely love it do it all the time come for UI manager can also help you keep updated they've got an update all button there also very handy two models you may not already have are the ones for the IP adapter just like they indicate on this civit AI page but the IP adapter plus underscore sd15 bin into the custom nodes IP adapter models directory and the model dot safe sensors into models clip Vision so here you you can see it in my directory structure we've got the IP adapter models and that's the IP adapter plus bin and all the way down here in models clip Vision there is the IP adapter Vision you can see here that I renamed mine because generic name confuse tiny brain but that's it no magical incantations or Feats of bodily strength required just super simple downloading excellent as mentioned all the workflows I'm showing here are available from a very comfy nerd though they are based on that aloe vera workflow they're mostly for stable diffusion 1.5 at the moment but there is that experimental sdxl version 2. okay so I'm giving you three different versions available for download we've got the basic instant Laura just like that aloe vera civet AI page but I've also got one for instant Laura with control net and also with two control Nets so you've got the normal control net so to speak and the QR code monster for doing Spirals and hidden text and all sorts of things like that as well they're all really basic examples so if you have used comfy UI before it should be very easy to see exactly how everything works and therefore to integrate it into any other workflows you may use right let's start with the this basic instant Laura you've got the usual purple boxes over here to select the models you like to use for your creation the clip Vision model and the IP adapter models are the ones you know you just downloaded now so those should always stay the same but feel free to use any stable diffusion 1.5 checkpoint or vae you like next up we've got the image width and height along with where to load your Laura data set from what's a data set who knows well it's just a simple directory with whatever images you fancy so here are a selection of images I've got a directory there pixabay woman because it's some images I downloaded from pixabay there's eight items in total there and they're all slightly different but it is of the same person there we go so we've got lots of different images in there and we're going to use that as our Laura now you don't really want to put too many images in there anything from 1 to 20 is absolutely fine also if you want to put a prompt in then you can but let's just see what it generates by itself without any prompt to start with and there you go almost something like her not bad for zero training and very little effort all right let's see what happens if we generate a whole bunch more images so we've got a new one there I think perhaps we're almost starting to see what's going on here there's a little bit of a trend so they are fairly consistent it's got the same sort of face but the rest of the contents are also fairly similar what you're getting here is like a fairly consistent mashup of every image that you provide so that's very useful especially if you want consistency on each image generation or you know say perhaps you always want a window in there so you you can just drop in an image of a window we're getting leaves here because there was a leaf image there you go for example we've got that leaf image so holding the leaf it's it's picking out sort of all the different aspects from the various images and then creating a new one for you excellent all right so we can have quite a lot of fun just by creating mashups which I think is perhaps what this was sort of originally designed for put lots of different images in there you know a person and a style and all sorts of things and that's what you'll get out but wait there's ever so much more given this is an instant Laura we can also prompt our way into doing loads of different things and we've got a couple of extra boxes for that here we've got the prompt strength and the Laura strength as well but bear in mind this is a very strong Laura let me give you an example so we've got a photo of a hipster man in there as the prompt and if we cue that up and do a little generation what we can see is it's definitely not a hipster man at all that's because this Laura strength is very very strong indeed if we drop that down however to something like 0.5 and then regenerate what we will get instead is something that looks a lot more like a hipster man but still with those elements of our instant Laura as well so he's got some very lovely lipstick on there and there's the leaves and all those other things we were seeing with the images before typically The Sweet Spot is between 0.4 and 0.65 on that Laura strength for a good mix though do experiment a way for other interesting blends to change even more you can lower the lower weight even further so that your prompt is more powerful here we've got the Laura strength down to 0.4 to make an anime style cartoon woman illustration and there she is we can keep generating those and we will get fairly consistent characters it looks to be the same person each time but in an anime style and we've done this without any training let's pick another style here as well so this time I am going for watercolor and once again you get a fairly consistent woman from your Laura's whatever it is that you're putting in but this time in an entirely different style so I think that's really fantastic for absolutely no training whatsoever just putting some images into a directory doing a little bit of prompting and some balancing of Weights very nice indeed and of course if you want to change your instant Laura just change which directory you're using so here I've got bruty mcnut face which is some images I just generated using Darley alright let's pop him in instead and we'll copy that there pop that into that directory there we go now I've got a new Laura that's pretty instant isn't it and if we generate again using exactly the same settings other than the Laura we'll get fruity mcnut face as a watercolor painting I think that's rather cool look at that it's nice okay so what if I wanted to make a really nerdy rodent but in that sort of style well it's okay but as you can see each time I'm generating here even with a fairly low Laura strength I'm getting a consistent output with this roundness and I've got a you know I've got a nerdy rodent but it's it's always the same sort of pose and stuff how am I going to change this if only there was a way to have just that little bit more control oh just I'd control it all right okay why not so here we've got two images this time we've got the IP adapter for your insta Laura I'm just using one image this time because yes it works fine even with a single image I've left the load images from directory box over there if you do want to use that but I found it more convenient just to drop a new image in there each time the prompting isn't really changing in this example I'm just putting you know stunningly iconic Masterpiece in there not particularly changing the style or subject in any meaningful way and what we've got here is got this face Okay so we've got a sort of Elf woman and a weight lifter and if we generate this what do we get well we get that sort of elfish based woman lifting weights brilliant all right and if we drop a different picture in there instead so we've got a face there with rather wild and crazy red hair going everywhere well that gives us just exactly that so it's got both the style and the subject there so you get a very similar face and the same sort of thing going on in the image that you get generated excellent all right let's do a more cartoony one there so slightly Stern side eye going on does this come out as an illustration yes it does there we go so that's her doing those weight lifting bits excellent but of course you know if it hasn't got a face in there it's just going to copy the style so if I put a black and white image in there what do we get out well you've probably guessed it already Yes we get a black and white image or how about if I put a painting of a person in there what do we get are we going to get both his face and the painting style well yes we are there he is that looks absolutely crazy okay so yes there he is lifting weights all right how about if we drop a statue in there are we going to get a statue as the output lifting weights I think you've already got the hang of this yes we are there he is a statue lifting weights and of course once again if you put something without a face in there it's not going to transfer the face but it is going to transfer the style of the image so we get that rather cool Van Gogh style output all right that's you know that's fantastic but what what if you wanted it to be a beaver and you don't have a picture of a beaver it's okay because we can use prompts remember so we've got a furry Beaver there lifting weights and this time when I cue that prompt up and generate what we get is a rather fantastic Beaver lifting those weights instead how cool is that but talking of Weights let's throw this image into the control net adapter there so we've got a weird sort of creature and we've got a face in the IP adapter if I put the control net strength up to one and I've got this Laura strength of about 0.65 then oh oh oh dear oh dear we really have entered the valley of The Uncanny with the horse-faced person or is it a person-faced horse I don't know but basically you can do all sorts of things with this just just by adjusting those different weights and having some really interesting pictures going on down in that load image area all perfectly normal almost as normal in fact as this next workflow where I've thrown in the QR code monster control net as well because everything is better with more control Nets this workflow takes three images the same two as before with the third one being the image for the QR code monster I'm sure you remember from my last video if everyone does watch all of my videos don't they don't they anyway as you can see here now all you have to do put the images in there click your cue prompt and at the click of a button you get a magical image that copies the style the subjects the pose and adds your hidden message in there as well if you can't see the hidden message let's make this really really big there there we go we can zoom in on that it says control net on it you might have to square no but basically there you go so another control net on top of the other controller put your words in here adjust the strength so if you want that more hidden for example you could drop that down to 0.9 and if we wanted a completely different person there we can put that in and if you wanted a completely different pose you could put that in two mix them all together adjust whatever weights you want use whichever workflow and create some really strange things though if you need an imagination boost what you could do is check out this next nerdy rodent video

Channel: Nerdy Rodent

Views: 25,719

Keywords: nerdy rodent, ai, ai art, best ai tutorials, Instant LoRA, Instant Mini LoRA, easy LoRA, zero training LoRA, stable diffusion, generative AI, generative art, simple LoRA, ComfyUI Instant LoRA, mini LoRA, ComfyUI, lora without training, AloeVera instant LoRA, MiniLora, no training LoRA, Stable diffusion LoRA, Ipadapter ComfyUI, ComfyUI IPAdapter

Id: HtmIC6fqsMQ

Length: 15min 22sec (922 seconds)

Published: Fri Sep 29 2023