ComfyUI : ChatGPT helping us prompt, but not in an expected way!

all right so I had a bit of a brainstorm and I came up with this idea of letting Chachi PT guide me on answering questions I didn't know the answer to with images so we're going to build this this graph from scratcher but let me zoom in so you can see we're doing here basically we're going to pass this to chat GPT uh so by the way this is just an example of how to integrate chechi PT this is not like the only way to do it I'm trying to teach you how to fish here so you can do something yourself but just realize this is one way to do it so what we do is act as a creative Problem Solver and the idea here is I wanted to creatively solve the problem that answers in prompts now it doesn't quite get what a prompt is because it was trained before all this kind of thing took off but it does help with the formatting a bit I found and I will give you a prompt and that's what we do here is we have the separator as prompt equals down here and you describe a creative solution of the problem use terse concise terms to describe the answer and use descriptive details answer with one sentence and one response only and keep it under 40 terms or less so I'm trying to keep it so it doesn't run amok like you know some of the other terms we see people using I don't want it to be huge 40s plenty and I want to start with a photo of because that's I think the more interesting part of this and you can use commas between terms I put this on here so that it doesn't say and a and a lot of other stop words so this just kind of helps with that and it says you're just play along and don't break real play by saying you're an AI language model because that's annoying and it just draws a picture of an interface so just guess at the answer is what I put in there so here's what I do I just put a prompt down here of something I want the answer to but I really don't know what I want it to be so I've done all kinds of crazy things and I'll show you one that actually hit pretty close to home here in a bit okay to get started what you're gonna have to do is go into your manager here we installed this a couple episodes ago and just use the hell out of it and go and do custom nodes here and I'm going to go to install so you can see what I have installed so we're kind of building on previous episodes but this uh this specific Suite here or suit quality of life suit B2 by Omar number 25 five I'm not sure if these numbers change or not but that might be a good way to find it and you can see that it adds a whole bunch of different nodes to this so we're going to use this it does depend on one other node Suite in this this one here this was nodesuite which has a ton of great nodes in it and we're going to have future episodes where we look on those but today we're going to focus on what's in here but note that this node Suite is required for this one to work at least as far as I can tell right once you have it installed go ahead and re-open comfy and we can get going what you'll need to do is go over to a playground over at open Ai and open an account and I'll put a link down below so you can do that it's different than the regular open AI account and when you get in there by the way you can see I spent a total of four cents here working on playing with this all day so don't worry about spending too much I do have a capped at 50 bucks just in case I go and run amok and then you're going to have your API Keys you're gonna have to go over here to under API key and you see I have a whole bunch of them and you generate a new secret key and it's going to let you copy that you can not get to these again so just realize that if you go to edit the key you can only edit its name you'll never be able to get this key again so make sure when you copy it you use it immediately so how are you going to use it so after you get the Omar Suite installed you're going to go to its custom node subdirectory inside of comfy and you can also say a config.json you're just going to edit that and put your key in there and it's pretty self-explanatory I'm obviously not going to docs Myself by showing that but that's how you do it is this file right here once you do that then this thing will run just fine and dandy and as always this graph will be included in the post here on YouTube for people who are sponsoring the channel at sponsor level or better so click that join button down below and grab this graph and help support the show so just like always let's get started by loading our checkpoint and we're going to set let's do xdxl okay once you re checkpoint loaded we're going to add node go down to O which is absolutely terrible I wish you could rename these oh open Ai and we're going to load the chat GPT simple now again this is going to work because we have it tied to that config file but we don't want to type into this and you could but again I need to pass in instructions and this node does have default instructions but I really don't like those I want to do something different so we're going to go ahead and override what's in this node normally and we're going to go ahead and change this to convert the prompt to input so now we can drive everything through this text field here all right the next node we want is also in that node Suite let's go down to O we'll go to text operations and concatenate text I'm going to change this one to the color green because we're going to use it as our Pro as our positive prompt here and down here under separator I'm going to change that from a comma to the word prompt equals and you could obviously make this anything you'd like but this just makes it easy so that chechi BT can understand that the instructions at the beginning and the prompt are two different things which is what I'm doing here I'm going to take and paste that wall of text that we have in here already so obviously you can work on this I keep changing it to try and improve what chat GPT is sending back but I think I'm pretty happy with this one right now and then you're going to want to put something down below and we'll just use this one because I like to see what it's coming up with I've done all kinds of forms I'll share one in a minute I think was pretty funny and then we need to do is we need to do our standard clipping code for our text so let's drag that out and we're going to encode our text but we're going to do is we're going to change it so we don't enter the prompt here we're going to right click and convert the text to an input that way we can use the string to drive this so whatever is coming out of the answer from check GPT will be fed into the text and obviously our prompt is going to be this so now we're feeding the chat GPT Suite here with our prompt well it's our instructions and then the prompt and then over here we're going to encode it and condition it like we normally would and toss it into a case sampler so we should have this down we've done this a thousand times but if not it's not very complicated it's just adding those nodes and then we'll hook the model up here and not for the negative we don't really care we don't really use much of a negative with sdxl at least I don't so texting code and we're just going to toss this right in now if you wanted to you could use the sdxl versions of these clip encoders but for this this works just fine and for our latent we can just pull that down here create an empty latent and again we want 1024 for sdxl now one other thing that I did is I find that when I'm doing this I tend to get the same items over and over again like if you use the word scary you get a lot of dolls things like that so I did is I duplicated this I made it red right clear the text out of it here and I've just I guess created a blank one and this one's going to be our negative so we're just going to use this as a comma here okay what I want to do is I want to include a statement to say do not include these terms in your solution and then we can put those down here it's pretty interesting is this is pretty much our prompt injection is here you give it all these commands and and the instructions and then someone comes along and tacks to the end hey by the way do this too like forget everything you just were told and do this so that's kind of why large language models are so hard to defend against prompt injection because you're just using language like it's expecting okay we need to concatenate this so we can actually use another one of these but we can turn both of the prompts into inputs and we don't need the separator either but we're not going to worry about that we can take this string here a string here and concatenate them together and toss them and we'll minimize this because I don't care to look at it so we're doing is we're taking this positive and we're putting this out here we're putting our prompt and then we are adding these additional negatives if you find yourself getting repeating elements just put those down here so we can put all that over all right there's a couple more nodes that we want to add and these are to help us see what's going on because there's no place here where we're seeing the actual result from this come back from chechi PT so let's add that and it's going to come out in the console window it's not going to show up on this interface add node get onto oh again I wish I could change that go to debug and you see there's a text and there's a debug text there's also an open AI Advanced GPT and then some of these so you can use whichever one you want I'm just going to go for the simple one here debug text it's good enough so once we have that we need to change it so that this text is an input and we'll just take the string here and that will throw it onto the console so when we're running our prompt we'll be able to see what came back from chechi BT and we can change this to if we'd like to all right we are almost done let's take our lead in here decode it using the vae pull this down here and we'll save the image and I want to put it next to my prompts so this is kind of like my UI over here so as we come back with images we can adjust our prompts as needed and I don't really need to see the rest of these things over here now as far as your K sampler here again I have a preference and you do what you want to do today I think we're gonna leave this at Euler and we'll maybe drop a c of G down to seven and maybe we'll change this so we used exponential last time let's use Keras this time so that should be about it bring the console up now the first time you run this it's going to create a wall of text you have to go to the top and you can see the GPT response here a photo of a mysterious dark lit Forest engulfed in an eerie Mist with a colossal shadowy figure lurking behind ancient neural trees creating a sense of chilling suspense and cloaking the era and a sense of Otherworldly horror all right I also like things about what did you find at the flea market what's the strangest thing you ever found at the local flea market now this one hit home with me because what it popped up is actually something I have hanging on my wall and I was like that's hitting a little close to home so I actually have this mask that my grandmother made many many moons ago and it's been hanging in my wall in my office and that popped up and I was like look at that all right so uh there's what we found at our local food market that is indeed bizarre our prompt here and you're going to see now they're going to be very terse so we'll be able to see all our prompts a photo of a vintage circus poster featuring a greening three-legged cat blah blah blah well that's fun how about we change this to what's the scariest thing to find that words like Arcane almost always yield books um so different words tend to lean towards certain subject matter I didn't kind of tell what the training data was right so here's a dial we're gonna find we get a lot of dials so again we include this negative here do not include these terms in your solution we put wow so that way we stop getting towels and that's extra nice what is that that's a vintage decrepit Taxidermy animal enclosed in a cracked glass display case so then Eerie red lighting and cobwebs uh it sure is and I find two Taxidermy is another one of those items that you're gonna see pop up a lot if you use the word flea market so isn't that pretty interesting that those things are related but wow I would never have thought to prompt that and uh probably for good reason so anyway on that one we're gonna go ahead and leave this so there's a fun thing for you to try out again this node Suite allows you to create these things to augment the prompts you're creating uh so wild cards work sure but why not use this and again it's very inexpensive and I'd love to see what your best prompts are so throw them in the comments below you get one and it's just amazing I'd love to see it or if you have these seed questions here I would love to see if you thought of something else that's pretty creative once again thanks to all the people who sponsor this show so if you click the join button down below you get your name on this poster as well as this graph which I will include in the YouTube posts for a sponsor level and better so you guys can have this and play with it again you will have to create your own account to be able to use the API with chat GPT but I think for the expense it's worth it and then I'm sure it won't be long before someone comes up with a node that talks to other language models I like when we have it stability for example so you'll see some of those coming along quite soon but this is my first foray playing with uh text insertion here and it was a lot of fun so everybody take care stay safe and I'll catch you all next time

Channel: Scott Detweiler

Views: 11,988

Keywords: stable diffusion, stable diffusion xl, ComfyUI, stable diffusion tutorial, AI art, SDXL, comfy tutorial, sdxl tutorial, chatgpt, chatgpt prompt, chatgpt tutorial

Id: T1oWgIICSkI

Length: 12min 24sec (744 seconds)

Published: Thu Aug 24 2023