Stable Diffusion - Prompt 101 #ai

all right welcome back to part two of the stable diffusion series last time we installed automatic 1111 and got stable diffusion running on our desktop today we're going to be focusing on the prompt to better organize writing out your prompt you can break it up into sections first there's the subject then there's the medium then you can look at Style if you're interested you could add a particular artistic Flair to it you can look at what type of resolution or scaling you want and last you could look at color and Lighting each of these have a different impact on your prompt to start let's look at the subject you may want to generate a picture of a woman go ahead and type in a woman and type generate stable diffusion using dreamshaper 8 and the settings we provided decided on this seed this is a woman now if we want more detail we have to add that in to our prompt so maybe we want a woman with silver hair now as you see we have a different image it didn't just take the old image and change her hair color we actually have a different image to similar looking woman her hair is now silver but she's wearing different clothes and has a different background suggest tweaking say your hair color can fundamentally change your picture given this is our subject we may want to have her doing something so instead of just putting a woman with silver hair we may want to say something like a woman with silver hair walking we get a picture of her of a woman with silver hair that happens to be walking you can continue to play with your prompts here but if you're not getting what you want if you have something specific in mind you've got to go with detail right we'll try one more prompt here a woman with silver hair walking through fire and as you see we keep progressing through putting this woman into a different situation her face is slightly changing we're getting different lengths hair different types of hairstyles different clothing so every time we tweak our prompt we're getting something different so if you have something in mind or you're trying to steer towards something maybe you guys can guess what we're trying to look at here you want to be as specific as possible so really what I'm trying to get is I want Daenerys Targaryen walking through fire and now we have a picture of Daenerys Targaryen walking through fire now I'll be the first to admit this isn't the greatest picture but we're keeping our width and height low so we can generate the images fairly quickly given that we're going to be constantly tweaking it and another thing we can do after the fact is we can add in a high res res fix which will help upscale this image and make it a little you know less weird looking I'll go ahead and put in some settings here we'll just upscale it up a little and see what we end up getting so we didn't change any of our settings but we are upscaling the image so this way as you're going through and creating images you don't have to think this is what my final product is going to look like keep it at a lower resolution generate to get close to what you want and then start doing the fine tuning so as you see the image that we had that looked absolutely atrocious well may not be what we're trying to get to once we've upscaled it it looks a lot better so don't be discouraged as you're testing your prompts that's not necessarily what your final image is going to to look like you just think of it as fine tuning and getting close to what you want and then you can get the rest of the way by changing some of your other settings we may want to tweak this a little by adjusting some of the weights so something that you could do throughout your prompt not just to the subject but to any of the attributes you put into your prompt you can do something called weight adjustment so I'm happy with the how much he's walking I don't need her to be walking any more or less than what she's doing there but I kind of would like a little more fire so if I highlight fire I can do a couple things I could type in a couple different prompts to change the weight but honestly the way I do it and the easiest way to do it is to highlight the attributes you want to change hold Ctrl and just press up or down so as you see it automatically puts the format which is a parenthesis followed by the attribute colon and then a weight with a closed parenthesis so what we see here is fire is actually weighted at 130 percent we also could have held control and pressed down and even after you've modified it you still can you can bring the weight back down to one and you can drop the weight even lower so like I said I'm happy with the picture but I kind of wish there was a little more fire in it so I'm going to turn off the high res fix for now we'll do that when we actually want to generate images we want to see and we will quickly generate a uh another example just with the weight different on fire the picture didn't completely change but we are seeing more fire with the weight up higher one of the things we can do actually is we can compare what it would look like at different weight levels so we can come down to this script and do an XYZ plot and we can change the seed to prompt search and replace s slash r the other two set to nothing it's fine we want to see what values we want to change so what I'd like to see is how the impact of the different weight of fire has on my picture what I can do is on that search and replace line I can identify the first item I want to find and what I want to replace it with and then have a string of variables that will be replaced in subsequent trials so if I place here um we need the first string to match what we have in our text so we want to replace fire with fire weighted low higher higher and all the way up so we can take a look at how the different weights of Fire have on our image now you'll notice Fire by itself is the first thing we're generating and in our string here we have fire with a weight of one when we take a look at our images we would expect those to actually be identical images because fire with no weight and a fire with a weight of one should be identical so with these settings we'll go ahead and hit generate and what we end up getting is a grid of images with the one variable we identified replaced and then the prompt run through the generator so our first image here is what it looks like just having fire unweighted and then the next image shows a fire with a weight of 0.7 weight of 0.8 a weight of 0.9 so we see that slowly we're getting more and more fire sort of appearing around her and then a weight of fire one actually takes the background away and adds some fire into the background fire 1.1 1.2 and then 1.3 we're getting more and more fire in those fire colors in our image the fire with the weight of one is the same as fire without any waiting on it at all it's the same image it's just a different way of writing the same prompt now you may have wondered why did I stop at 1.3 well part of it's because I I don't want the entire image to be fire focused I just wanted to make sure I have enough in the background to make it interesting but also I have found that once you take fire or not just fire once you take any weight to 1.4 or above it starts to overtake the image itself and become less interesting we'll go ahead and take off our scripts here and we'll just pump up one image of fire really high and take a look I have a feeling we won't like it I'll probably use the 1.3 going forward so yeah not a bad image um but we are kind of losing some detail uh the trees and things in the background are also just kind of becoming smoke and wavy the whole image is really just starting to become fire itself so not as interesting once you get the weights above 1.3 1.4 you've got to be really careful there another thing you've got to be careful of is what items you're waiting so we've taken fire up to 1.3 and I think that's what we'll stick at for the rest of the tutorial but you know what if we want to up the Daenerys or excuse me the Daenerys Targaryen wait maybe we want to have more of her features so what we can go ahead and do is take that and we can bump that up so now we're waiting her at 1.3 as well as the fire at 1.3 and what you see is it actually starts to get a little bit ridiculous you don't necessarily need to wait human elements higher it's more stylistic effects that you want to wait higher you know we start to get some weird detail on the clothing I think the face is actually starting to look worse and overall you're not really going to get more of what a person looks like than what a person actually looks like so if we take that back down we're back to our fire with 1.3 and then you know what if we wait up walking are we going to get an image where she's walking faster is it going to be more walking focused on the image and see as uh it's coming into view here we're getting more focused on what she's walking on and the fact that she is walking and actually the fire details are kind of dying down so what I found is if you start waiting too many things they actually kind of cancel each other out so the image starts focusing more on something you've waited High which probably means something else you waited High originally is now competing and ultimately gets weighted Less in the overall image so you know I was happy with the fire 1.3 I think that's a good start that we can take through for the rest of the video here the next thing we want to take a look at is the medium the medium just like in the real world is how the AI interprets the image how it was processed so is it supposed to look like a photograph is it supposed to be digital art maybe an oil painting or hand-drawn you know rendered out of a 3D engine there's all sorts of different directions we could take this I I went online and found a few examples of different mediums that people use let's go ahead and copy this over and we will do the same format we did with the weighting so I'll open up a script an XY plot instead though we will take a look at some different mediums here and we'll go ahead and take a copy of the first medium and add it to the end of our prompt so that way it has something to queue off of so now we have the same image we're starting with here but we're adding portrait and we'll substitute out various other mediums so let's go ahead and generate this image out and there we go take a look at the first one we have what it looks like in a portrait medium we have a digital painting we have concept art it's a realistic illustration sort of a hand-drawn look underwater and even underwater steampunk so it's kind of funny looking at the underwater here where obviously beneath the surface and we still have fire and flames shooting up so you know kind of a funny interpretation there hand drawn is getting a little funny I'm not sure that's supposed to be the smoke or if that's supposed to be burning trees in the background uh portrait actually looks really good even without upscaling I'm not necessarily certain that's the direction I was trying to go but we do have digital painting concept art and ultra realistic all kind of looked the same textures are a little different but it doesn't have a huge impact on the image itself so really you could go anyway I'd say portrait kind of takes you off on its own path most of these middle ones here are all about the same with a slightly different look to them I think I might pick a couple here and we can use that going forward along with the medium you may want to take a look at Style now this isn't a hard boundary a lot of things that might be considered a medium could be a style and vice versa so really you can kind of Clump these two together but with style we're going to take a look at things like hyper realism art the modern impressionist type Styles textures details fantasies that type of thing leaving our prompt here I kind of like to Portrait I kind of want to see the direction that's going we'll go and leave digital painting let's take a look at the image again we will go ahead and continue to generate with portrait digital painting and the ultra realistic illustration so I kind of like those three and I kind of want to see what we can do to tweak those to make them better but let's take a look at some of the different styles we can add on to this so saved here I have some Styles and we will go ahead and make this another prompt search and replace I think we lost our h on our hyper realistic there so what we'll do is we'll take the first term add it up oh we lost portrait up there so we'll go ahead and add portrait up there for the first replace and we'll go ahead and add hyper realistic up there for our second one and we'll hit generate Well that took a bit let's take a look at what we got so zooming in here we have portrait digital painting and the realistic illustration across the top and then we have our various Styles here across the left hand side so zooming in we can take a better look at how the prompts have influenced our images here as you see going down actually can have quite an impact the portrait seems to create different images almost every time the digital painting seems less impacted by some of the prompts than others and the ultra realistic illustration seems to have the least amount of impact by the various Styles we can add on they're all interesting but we really should only choose one going forward if we start hanging on to too many variables we're going to get into a three-dimensional Matrix here portrait I think maybe looks the best at this stage but it's not really the direction I was hoping to go maybe the pop art and ultra realistic illustration seems the most interesting mostly because While most of them are holding weapons they're either longer swords which doesn't really jive with our character and some of these actually get into dual wielding and I'm not even sure what they are dual wielding I'm not particularly thrilled with this patch of fire here behind it there seems to be a few of them that do it the digital painting under hyper realistic is creating that obviously the pop art and the ultra realistic illustration is creating that I see it in a few other ones so we'll just take a take a look at that see if that starts to change as we add on additional prompts as I said most of these are interesting and I'd like to see how we could take most of them into uh you know further revisions but we've got to pick something and I kind of want to get away from this weird dual wielding thing that's happening coming back to our prompt we'll go ahead and add in the two that we like I'm going to go ahead and disable the scripts here and we will generate the image with the ultra realistic illustration and pop art features so checking where we are now again I mentioned I don't like this fire going on in the background but maybe that'll get cleaned up the hand doesn't look amazing here but it's okay we can go ahead and test the waters by running a high res fix on that to see what we look like if we were to upscale it well we had a bit of an issue there the render kind of froze mid processing so we had to restart but I was able to bring everything back the way it was and render out the image in high resolution so let's take a look uh you know I'm happy with this the fire is okay I don't particularly love the fire splotch in the background but as we continue to process this that might just kind of go away on its own um the sword actually looks really cool it kind of looks like in a rock which you know a Dothraki would be using so it's kind of appropriate for her I doubt the sword Will Survive the continuous processing that we do but as a kind of Midway Point here this is a pretty cool image next up we're going to look at artistic style so I'm not particularly a big fan of using the Artistic Styles um really what you're doing is you are creating works that represent and kind of look like how a particular artist would be creating their own work so I feel it's closer to kind of stealing someone's intellectual property versus you know this model that may have been trained on Game of Thrones and you know other things out there but you're not specifically copying like a frame from the show or something like that you are sort of creating your own work here not to say that it's not yeah you shouldn't use it I'm not trying to shame it or anything it's just not my personal taste I don't particularly like using other Artistic Styles but for the video we'll go ahead and explore that so we'll open up our prompt and replace here we will go ahead and add on an artist so I pulled out a list of artists off the internet that people tend to use for some of these works so we'll get those pasted in here and take off high resolution because that'll take forever all right we're happy with that let's go and generate and see what we get I messed up a bit here I just put the name of the artist in the prompt uh typically you don't just put the name but you'll put Buy so that it indicates that it's art created by that person if for some reason you happen to choose an artist and just put their name in and they're a well recognizable face you may end up adding the actual image of that person to your artwork as opposed to their Style I'll go ahead and re-render this with buy but I'll keep this and we can compare maybe they'll look exactly the same or maybe we'll see a few differences but let's go ahead and generate it with the uh the word Buy all right so comparing the two there's not a whole lot of difference I'll go ahead and overlay the two on the screen here but for the most part they're they're pretty identical there's there's a few subtle differences um again really neat I think each one really takes it in a different direction and of course if any of these are your particular taste then go for it um honestly I kind of like each one of these but for the sake of the tutorial I'm going to go ahead and clear off the artist and we will keep it with us creating some of our own works here next up let's take a look at how different resolutions can affect our image now we've looked at the high-res fix that will produce some upscaled images that's not what I'm talking about I'm actually talking about putting resolution markers in our prompt so we're talking maybe like 4K or 8K Sharp you can use Unreal Engine which I think we'll go ahead and do so looking at it as if it was rendered out through unreal again some of these kind of blur together with style or medium there's no hard definition of what goes into what we're really just kind of clumping things together and how they impact the the image overall your medium tends to have a huge impact whereas things like resolution and maybe even style can have a smaller impact on the image so taking a look unreal we'll go ahead and start up a script here where we will substitute unreal for Sharp or 4K 8K Etc so with high-res fix off let's go ahead and generate and see how alternate resolutions affect our image was there a huge difference not really you know we're not changing the image overall I'd say the subject is still pretty much in the same pose doing the same things the background sometimes has a few extra trees we're still walking mostly in the same way we've lost some of the lava under us under unreal but yeah there is a little bit of an artistic Twist on each of them so again you need to go through and choose what type of look you're looking for at any point along the way you could take a snapshot of the image as you're building it maybe run it through some high-res upscalers to see you know what it really looks like it's kind of hard to tell when they're low resolution like this I think they all look pretty neat we did lose our fiery a rock we still have somewhat of a fiery sword here under unreal I'm not certain what we have here for Sharp we've got a couple swords another odd thing kind of in both hands so I feel with the extremely intricate and detailed it's actually searching for a little more than what we have to offer and I think maybe it's taking it a step too far I'm kind of a fan of this for unreal um maybe the background could use a bit more cleaning up but I mean they're all they're all pretty good I went online and took another look at some of the resolution options people are doing and I've seen depth of field used so I went ahead and added it to the prompt here and rendered it out I did do a high-res pass on it and I'm actually pretty pleased with this I feel we have some of the fire on the ground that we lost with the unreal but the other one still had however we'd have an a real object in the hand that is you know a flaming weapon which is pretty cool we have some fire in the background the trees in the background are somewhat realistic I mean they're artistic but I think with some of those other renders the background just kind of turned into round Rocky smoke hybrids so pretty pleased with this I think we'll use this for the last bit of the tutorial this depth of field last up uh color and lighting and other effects that you may want to tack on so there are many different options here a lot of people go with things like cinematic lighting or dark Edge lit things that you'd hear about in a photograph tutorial where you're learning where to place your lights and how to properly light your subject there's a lot of control and there's a lot of options you can do I selected a few common options here go ahead and play with this and take some time yourself to figure out what type of look you like like with the rest of them we'll open up a plot here throw in our depth of field and just so you can see what other options there are we'll have it replaced out with some of these other lighting blurs glows Silhouettes other types of options so let's go ahead and turn off high-res and generate that out taking a look depth of field is what we looked at in our high resolution we have cinematic lighting here we have some motion blur glow lighting we have a backlit Fuji color and silhouette again all of them look pretty neat in their own right I'm not certain what's going on with this Seed where they really want her to have two things in her hands but that that is what's happening it's interesting with cinematic lighting we lost our sword and this looks more like a jet of Fire coming up behind again I'm still a fan of the depth of field I think the Cinematic lighting here looks pretty good I mean they all look good but I think without having to try to figure out how to handle these flame bursts and things in the hand I believe these two are our best option now I really wanted to keep this to about 15 minutes and I see we're pushing over a half an hour so I'm going to go ahead and call it there on screen here is the final image that I came up with I'm going to go ahead and make a companion video on showing how I took the prompt which I'll be displaying here on screen now how I took that prompt ran it through a few more filters that are non-prompt related to generate the image we see I hope you found this video informative I really had a lot of fun making it if you've made it all the way to the end I appreciate you watching if you have any ideas for the next video please let me know

Channel: Not That Complicated

Views: 18,008

Keywords:

Id: uZlSKdkQD3A

Length: 30min 5sec (1805 seconds)

Published: Sat Aug 19 2023