Lightning Strikes the Art World: Mastering SDXL-Lightning with Stable Diffusion Auto 1111 Forge

[Music] hey everyone welcome back to Alchemy with zase this is Eric and uh in this video I want to dive into a new model that's kind of Hit the scene it's called uh well it's it's an sdxl model and called lightning and this model is absolutely phenomenal uh in the level of detail the realism uh on a lot of different levels so this model is based off of a uh so there's a base model from bite dance just called sdxl lightning this is an Progressive adversarial diffusion distillation I know I'm reading just this right here I don't understand exactly what that means um it's a new way of doing uh the sdxl models that allows you to do extremely low steps uh while producing amazing results we're going to dive into that into this video uh I would highly recommend going to this page reading through this and becoming familiar with this they do offer uh there is a uh a demo where you can go and try it out if you don't have like uh stable diffusion uh automatic 1111 or comfy UI or whatever on your local system you just come over here there's a um uh a group called ap23 where they have an sdxl lightning uh test interface where you can put in a prompt um it just locks it at four steps and just hit run uh It Takes a Minute sometimes to grab a GPU but then renders it and come and gives you the results so it's pretty good what we're going to do though I'm going to go and we're going to do some examples using different models there are several different models now that are using this technology and we're going to just go through and review those real quick um in fact let's dive into showing you the different models first before we uh uh dive into this okay so I'm pulling this up in my stability Matrix software I'm actually using um stable diffusion automatic 1111 uh uh Forge which unfortunately uh stability Matrix does not have a package for that one yet uh I'm I'm expecting them to get one in but in the meantime I've manually installed the forge Edition but I've got it set up to utilize the model base uh model folder that stability Matrix uses a central folder so I just come in here type in lightning you can go to civit Ai and type in lightning there as well and do a search for just checkpoints and kind of pull the same results I will warn you that it doesn't show all of the lightning based models which I think is very interesting um there's one in particular that I love using it's the Juggernaut lightning model uh and that's the one I'll start doing the demonstrations with but if you come in here um there are looks like eight different models right now uh at least that it's showing here that utilize the lightning technology you can see up here um if they'll either put in the title it'll be tagged that uh whatever so um I'm going to walk through the settings that you'll need to have set up on this to get the best results uh what to look for and um really kind of kind of how to get the results you want so let's let's dive into this okay in the interface here um I've got certain things already set up and I actually have a config preset for this because it's just easier to have that set up okay when you're using a lightning model I don't I'm not used a vae with it and it seems to turn out great so this is the Juggernaut XL lightning model uh it's a photorealistic model but the results you'll see like that image I showed at the beginning was from this model it seems like most of the models are are uh based a little more on realism I think that's what the base model is on so I'm kind of expecting some more models to come out that will give more digitally Illustrated or some other results but uh let's go through this and tell you what the settings I use to get just absolutely Speedy results and great results so first of all uh if you're using the forge Edition you do get uh some different Samplers uh I typically will stick with these three right here the Ula turbo DPM Plus+ 2m turbo and the DPM Plus+ 2m SD turbo uh they're just designed these are Samplers that are designed for Turbo models and or the fast models we'll say sampling steps I'm just going to put it at four for this particular demonstration uh you can go down pretty low on this and you can actually go increase it okay uh the more steps you uh put it up to it actually adds a little more detail uh and I'll show you what I mean by that when dealing with like higher res fix okay the config scale you want at one uh no higher than 1.5 uh the reason for this is you get above this and you start getting artifacts it starts kind of looking blown out um anyway you'll see what I mean by that too here we'll we'll show some examples uh and that's you know that's really it now um right now I've got it set to a 16 by9 uh aspect ratio I like doing wide format uh if we do some characters I'll flip that and go to 9 by 16 or by 4x5 so I kind of switch between those so first of all let's get uh let's get some prompts and let's just show some examples on this I'm going to jump over to my prompt gener I want to do some uh sci-fi scenes I love sci-fi you know I know a lot of you like fantasy too that seemed to be like the higher rated one on my on my um uh the survey I put out so let's do that let's do intricate fantasy uh Landscapes with characters let's just po pop some prompts out like that and just see what we get here um adding you know with these models I've tried doing like digital illustration um and other mediums it kind of sticks to realism I we'll we'll we'll try some things here so let's head back over here drop these in to the from fileer text box and we are going to do um the 9 by 16 I want it kind of a long version so and I'm not doing high res fix on this yet I want you to see how fast this is and I have it set to show this kind of in real time as it generates so you get to see how fast this goes so this is five different images okay and as you can see these are a little faded see make sure I got everything else set right let's try 1.5 there we go that's a little better yeah I mean these are going to these have great color there's tons of detail now this is again just the first render um typically you're want going to want to upscale these uh but the the speed at which it pushes this out is amazing so let's now let's move over to using um I don't use a negative prompt with these I don't feel it's necessary they've got these models pretty well refined but we are going to turn on high high reses fix uh I'm going to turn oh you know what that's I apologize so there is we're going to start over so what I've been doing like to get the images you saw before the initial render I do at like 8 to 12 steps um it push it gets the image oops and the initial image done that's why these are looking a little B so I'm going to kind of go back on that um and then when I do the high res fix I'm doing the high res fix at four steps okay but I'm not going to turn that on let's re redo these at the 12 Steps it's still going to be incredibly fast I know they say you can render these at two steps one step two step four steps and you can uh but I'm going to tell you right now the the quality and the how Vivid the colors are will change based on the number of steps you get and the number of details that you that you're putting into it so uh as you can see those look a ton better okay um what we're going to do we can even put that down at one uh on conf uh one on the config level and it'll look a little less painty now again I have a 12 gig uh vram card uh it's a Nvidia 3080 TI um so it tends to render pretty quick uh but these models are designed to run fast uh you know each person is going to get a slightly different results on a lot of this but okay now what we're going to do is turn on the highrisk fix and I am using a different upscaler I usually use the 4X Ultra sharp right it works really well but the 4X NM KD s SII 200k I think is is better designed for this it seems to give me better results when you when generating pictures from a lightning model so we're going to select that one and again High steps at four dooy strength I want it down depending on what you're you're doing when you're doing Landscapes like this I know I said to have characters in it and it doesn't really look like it put characters so I'm not going to worry too much about the doo strength if you have people characters typically you want a little bit less on the dooy strength because when you have it up above five it seems to mutate the characters I don't know why something somebody said about Laten space I'm not sure so with these ones here we're going to we're going to do F 0.55 and kind of show you what what it does here so it starts surrender goes through the 12 Steps shifts over to the upscaler model and and goes through four steps on that adds in the detail and then moves on to the next one okay I'm going to pause the video we'll come back okay those are done rendering and let's just blow these up so you can get an idea of what this is uh I mean just the level of detail you get in this how how solid everything looks and it's just I think it's amazing this is just AB absolutely amazing I mean these are my new favorite models and you got some characters down here which is awesome I mean don't get me wrong I love the other sdxl turbo models and everything but this is like a whole another level and being able to do it that quickly with this level of detail and um just how how intricate I mean it's amazingly intricate the reflections the water everything like this one this one just looks phenomenal wonder how they would use those stairs comes down here to the top of that but yeah and then you can take these and run these through another upscaler and even pull pull out even more detail but these are just beautiful images so now you've seen kind of some Landscapes Let's uh let's do some characters let's come over here and let's do some fantasy characters and we're going to do a specific color set I love the dark purples lavenders let's add in some dark green that might add some vegetation and maybe even I love orange you know I love Orange okay bring that over here let's do this okay let's make sure we got sh Wings C golden sunlight be want to make sure it actually gets the color sometimes it doesn't actually put the colors in so let's redo that I made some modifications the other day because it wasn't putting them in at all and so dealing with AI sometimes it decides just not to do it there we go lavender purple deep forest green orange red Hues okay I like it when it integrates it within the prompt um we'll go with these anyway these are good because it still gets it we're going to leave highes fix on and let that render out here okay those are done let's go ahead and bring these up here I mean look at the intricacy and the details around the branches I love The Styling in this one it gives it kind of that I don't know Grim almost seia tone it's doing better with hands I you know most of the images I'm I'm generating the hands are done better like this hand is great this one has the standard you know six fingers or I should say five normal fingers and then the uh thumb kind of in there but the level of detail on this is is really good uh it does great with faces I'm not using a detailer on this at all uh which I actually find um kind of messes up the faces and doesn't blend well with these with these renderings um it was interesting as I was using a detailer it would actually you'd see the seam around the in painting area and I tried doing things like increasing the uh Edge blur things like that and I just couldn't quite get rid of it so I just kind of left it and I actually like this better because with Adil you tend to lose a lot of the the uh the face the skin speculation uh speculation specularity I don't know just the blemishes the the details in the skin the dimpling that kind of thing and yeah it's just that's amazing um come over here let's go to this one right here it does great with fantas see I mean these are beautiful green eyes and yeah I did bring in the the colors I want got the Green in there it's got some of the the lavender and purples in there uh I don't know what other color did I say orange so yeah it's got you know introducing the kind of the orange up here and the gold dragons it does phenomenal with I love I love being able to I'm sorry I I do love being able to pick my colors themes over here and incorporate that into the prompt because then you get beautiful stuff like this where you have this kind of this uh orange scaling but with with like the lavender highlights you know so it's kind of theming the whole thing with that fairies that are not facing camera you know it's a hit and missing with the wings but again what we're looking at here is the detail um that is incorporated into this and what looks like a painting uh I think this one was water watercolor painting yeah and it doesn't it looks kind of like a watercolor that's the thing with these sdxl modes I think it's just everything kind of looks either like digitally Illustrated or to look real so um one I I do want to do one more demonstration we're going to do some photographs uh photography so um um let's just do different models and we're going to leave the color theming on there because I really love how it incorporates that so make sure it actually gets the colors in there deep purple gown orange red lipstick yeah looks like enough let's grab that bring that over here and I love being able to do multiple prompts at the same time okay we're going to let that run and be back in a second all right so let's pull these up and show you the detail that it incorporates in these I mean I it does eyes so well detail like you can see the dimpling in the skin with this this is Ed so the high highrise fix I'm doing it takes it up to uh 2K so we're you know we're going up to uh 2048 by 1152 and let's pull this up here and so you get to see the detail there so if you were to take this into uh do an up Res on a highres fix on see you even get the uh the skin blemishes in there it's adding a lot more detail I think you can even slightly see the hairs on the arm it's a little hard because it's still not super high resz but definitely definitely better hands are getting a little better this one's a lot closer up and and you can see the intricacy in the irises and the pupils the reflections get more detail on the skin color theming is coming through which is nice with the uh colors on the lips and the eyes you get Textures in the in the material and it's just a solid model I you I should say just lightning this lightning new lightning model model is really solid uh I'm I'm honestly in love with it and I'm going to be playing around with this I've tried a lot of different images Landscapes and various aspects like that to uh get different results and honestly I don't know I mean there other models have their ups and downs and I you know I'll use them for what they are but uh this is just I'm sorry I'm giddy because it's like anytime they release new stuff like this it's like Christmas you know you get to you get to play with something new you know and kind of push its limit see what it does see how well it performs that kind of thing and uh yeah we're going to do we're going to do one more one more here it's going to get rid of the uh the colors we're going to close this I don't want to do anything with that what we're going to do is intricate industrial Factory gritty dirty and complex Lex large Machinery let's get some prompts on that now uh this model of my understanding also works well uh without using the emphasis but it does take the emphasis formatting so you know with parenthesis and the emphasis number here uh but it should work well even without that in fact I've had people tell me that's how they do it they they just use the uh the no emphasis generator oh you know what let's uh let's set that to landscape this time so we're going to do 16 by9 give it a second to cancel that out now we're going to generate on that and let that generate okay let's pull these up so for those who are curious um total render time on these five images with the upscaling is about a minute and 23 seconds um I know typically when you're doing highres upscaling or the highres fix it can take a lot longer but with these particular models it does seem to speed it up and again I am working on a 12 gig C I know not everybody is I do take that into account um so what I would expect though is that you are going to see a speed Improvement okay yeah the level of detail on this is just insane that's a cool one they look like great big dynamos for like a power station or something like that so be that what it is um definitely check out those links uh I did put up a Blog article that has links to each of those models I'll link that in the description and uh have a look these are these are kind of game changers when it comes to the speed and the level of detail uh on how they are getting that detail in there um definitely like And subscribe to my channel like the video and uh you know if you haven't checked out uh my prompt generator zero gen I love it you know the people who use it love it gives them something that uh inspires them allows them to check out lots of different things uh dealing with whether it's the artists um being able to look through different artist Styles being able to add colors in and then getting uh I think it's something like 35 Plus drop down menus with all sorts of you know hundreds of different um uh I don't know selections options uh that you know for things that maybe you wouldn't have even thought of uh when it comes to what you want in the image so if you haven't it's a free three-day trial give it a try if you don't like it cancel it not a big deal um but uh you're going to you're going to love it especially the image analysis dropping an existing image and getting a prompt out of it so yeah I'm going to plug my my uh uh prompt generator here the I call it my online prompt Forge uh because of the level of um the ability to manipulate things a lot you know you can there's a lot of things you can do in uh modifying your prompts um getting access to these artists we're just going to do a quick demonstration of one aspect of this I I've kind of shown you this the the uh artist and reference screen but I want to show you the image analysis so what we're going to do is we're going to pick an image and bring it up so here's the original image we're going to try to duplicate I mean it's not going to be exact but we should get something that's very close or at least has the same characteristics okay so we're going to drop that right there and hit analyze once it finishes it comes back with um basically the prompts based off the description it was able to determine through analyzing visually analyzing that image we're going to copy those come over here let's change this over to a 5 by 4X five more of a uh portrait we're going to turn off highr fix I don't really need that right now and put these down here oops and let's just generate that and see what it comes out to each model reacts differently there we [Laughter] go so that one turned out pretty good right there I mean it's like I said it's not exact but the coloring the hair the eyes the the clown makeup clown outfit uh I did find that um there was a different model model that seems to react uh a little bit differently to these let's see I think it was um turbo Vision switch over to that and see what that looks like these are going to look a little bled out just because uh I left the config scale down at one this model works better at like 2 to 2.5 but um as you go through these you do get the same kind of thing you red hair you get kind of the the same eyes the same framing so that's the whole point of the image analysis is the ability to get a prompt that'll get you very close to what the original was okay very cool stuff uh if you ever have any questions for IT join our Discord and uh I'll answer whatever you need on it uh again you get a free three-day trial with it so there's no risk full fully functional uh I don't restrict anything in the free trial so give it a try uh like And subscribe and we'll talk to youall later

Channel: AIchemy with Xerophayze

Views: 2,795

Keywords: SDXL-Lightning, Stable Diffusion Auto 1111 Forge, AI art generation, image generation technology, AI technology, AI creativity, prompt engineering, artificial intelligence, digital art, tech innovations, advanced image generation, AI for artists, creative AI tools, high-quality AI images, fast AI rendering, SDXL model demo, Stable Diffusion tutorials, AI model comparison, technology breakthroughs, AI art revolution, Automatic 1111, stable diffusion, ai art

Id: saZJuPtvdZ4

Length: 25min 38sec (1538 seconds)

Published: Wed Feb 28 2024