Prompt Engineering - Part1 : Prompt Tricks You Probably Missed for Stable Diffusion

hey everybody Scott Weller here no I'm not dead sorry the channel took a bit of a break there so many of you know I took a job with stability so I've been uh doing the quality assurance thing uh since November and kind of went a little AWOL on making videos so I plan to get right back on that soon uh so we're gonna start with this one today what we're going to do is we're going to kind of explore a bit of prompt craft or prompt engineering so working with the prompts that are available in automatic 1111 and some things that I think we have been kind of remiss and zoomed over because we see all these really great new things coming out and we kind of forgot about some of the basics so not so much that they're Basics but the things I think you skipped that you might find really interesting so the one today is going to talk about uh switching between different prompts as we inference and then I have two more advanced videos I'm gonna be coming out with very quickly now next week I'm actually going to be in North Carolina speaking there uh so I'll be gone for another week but I'll be back so you won't see too much of a gap now with the video so we're gonna get the show right back on plus I'm thinking about doing a podcast in the show there's a lot of information that I receive uh that's good for reading but I don't want to do a video where I'm just like talking to the camera I think you'd make a better podcast so there's not a lot to show in some of this is more like AI news that I'm privy to that I kind of want to pass on so let me know what you think in the comments below so let's get started okay so I have this fixation with airships and the whole steampunk thing so I'm going to do is we're going to use this as an example so I have a very basic prompt here a note that I'm mixing artists so I'm not a person who like to choose one artist because I do feel that that is stepping on their toes in this case you can say that I'm inspired by both of these artists and I've kind of created my own style and by the way they're not photographers so this actually is its own thing and so this is a unique creation and I kind of like this one as an example so we're going to start with it so we're going to start with a very simple thing and we're going to complicate it pretty quickly so so bear with me so what I want to do is I want to take an Airship and I kind of want to mix it with a train and by the way this works with all models so we can do is we can put braces around the word Airship but what I want to do is I want to put a pipe character that is above the enter key in your keyboard the vertical line I'm going to put what I want to turn it into so I'll say a train now I want to put a closed brace on there so I have an Airship and a train what this is going to do is choose one or the other each time it does each step so we're going to do is we're going to get 20 airships inner spiced with 20 trains so if we do this we'll end up with a Airship train thing and that's pretty much what we get so this allows you to kind of take something and add to it now what if we wanted more Airship than train what you could do and this is not the way we're going to do it but I give you an example you can do Airship Airship Airship train so it's going to do three airships and then a train and this will work but there's a much better way to do it and that is to use the from and to Method so we do see here we do an Airship we do a colon we would do a train and then we would put the either the step number when we wanted it to switch to the train or the percentage so let's say we want to do Airship for just 10 steps and then the remaining 30 steps we want to do with the train and you notice that it started out similar and then it got pretty Twisted immediately afterwards now another thing we can do with this which is pretty neat is if we remove the second part of this and we leave the two colons what this means is it's going to take the Airship and replace it with nothing after a certain number of steps so basically it's going to remove Airship from The Prompt after only 10 steps this might be handy if you're doing like a dramatic sky or something like that so you see we have our Airship and then it's going to obviously continue working with what it was given but it's no longer going to use the word Airship as part of the prompt so it removes that word after this number of steps now we can also do the other way by adding just one colon this means it's going to add the word Airship to The Prompt for the first part of this there is nothing there it's going to do whatever it's going to do and then it's going to attempt to add the airship in near the end of The Prompt and obviously we're getting something that's not a lot of Airship but a whole lot of something else so let's go back and knock this number back a bit say to 10. so after 10 steps it will then add in the Airship this is probably more realistic as to how you would do this so the first parts of building again and then it adds in the word Airship and takes whatever we're doing so we have our building now turned into an Airship after 10 steps so this before and after capability or this switching capability is built in to automatic 11 11 and you can use this in all the different models okay we're going to do one more you see here I have a diesel Punk race car ice clouds highly detailed muted colors winter cinematic dramatic lighting 8K and then a photograph by Dennis Lofton man I'm gonna do two is actually Zoom my monitor up a bit so you can see this because I was doing some super um superimposing of words earlier I think it's just easier for you to see it in place so let's do this with the decimal method here so we have a race car so we don't want a race car all the time we want to take our race car and we want to combine it with something let's say we want to combine it with let's use the pirate ship again okay and now we're going to use the decimal so in this case if we do you know zero point 5 would be the same as putting a 20 here because it's half of 40 right so if we do path pirate ship half race car that's what this will do you could see it as it was drawing there that it was the race car first and then it became the Pirates so this is a really nice way to kind of hone in what you're looking for so this is a 0.2 which is the same thing as 10 there's 10 of these so it's a 20 percent you could do it either way you'd like with the decimal or with the steps steps it's probably easier to look at but in this case what it is doing is using the pirate ship 20 of the way in it is adding in the pirate ship instead of 80 of the way in which would be a lot more race car it'd be the other direction this will add in 20 pirate ship instead of the other way around there we go that's pretty interesting so it's combining those things together and you could do whatever you'd like here spaceships you know what have you but remember it's how many steps in before it's going to switch to using the second prompt so the lower this number the sooner it's going to switch to the second part of the prompt so again 50 50 if you're using 50 50 you want to mix that would be half of this so that's easy to think if you want more pirate ship this number would be lower so it switches to pirate ship sooner and if you want more race car then you would switch it to be higher so this kicks in with only the remaining 10 steps left should be noted as well that these can be quite lengthy you don't have to just put race car you can put race car on fire in the sky I mean you can really add anything you'd like to add in here if you want a background for example that's special you can do that and then have it go away after a certain number of counts like we talked about before you can use the double colon which is basically replacing it with nothing after a certain count let's say for example you wanted there to be a race car on fire and then a lot of smoke the punk race car that's right on fire or smoke so we could say that we want it to be 50 50 mix this and there we go so we can mix the amount of smoke and fire that we wanted to depending on how we put these numbers remember this will last until the 30th frame of 40 or the 30th step of 40 and then the smoke will kick in so that's why we got a lot more fire than we do smoke so our ability to adjust the scene is pretty amazing all right everybody thanks for watching as I say we're going to get everything right back going let me know what you thought of today and if you thought this was useful because I think a lot of these tips and tricks kind of get skipped over by the fact that again this flood of information is coming out so fast it's like drinking from a fire hose we want to try and get back to the basics so we can make fantastic work and move forward and kind of let's say master all the tools have been given instead of just kind of handling all of them briefly and then moving on to the next there's a lot of great things that have stuck around in withstood the test of time and that's what we kind of explore so everybody uh see you again soon everybody take care I'll see you next time

Channel: Scott Detweiler

Views: 21,030

Keywords: ai art, ai art tutorial, ai generated art, ai art generator, ai, ai created art, stable diffusion, stable diffusion tutorial, midjourney alternative, AI Artists, how to use stable diffusion, AI art inspiration, automatic1111, prompt engineering

Id: MgLpmksylOc

Length: 9min 5sec (545 seconds)

Published: Wed Jun 14 2023