Using LLMs in ComfyUI with Ollama

D way welcome to another AI fuzz video and today we're going to be exploring an interesting topic the world of llms now llms are large language models and they are m models that are training on a lot of information usually a lot of the text info or specific information llms you may be familiar with would be Google's gof or mistl or a clud or I gbt or there's a bunch of them um so how is this fitting into your AI work with t fi well something happened AMA was born which is very very good AMA is a way for us users to actually use these llms and communicate and ask them questions so I'm going to put the link in the description if the llama.com you would actually go to download your OS of choice Mac OS Lins or Windows very simple download it uh and then you will uh uh un unzi it and then double click the axe or whatever you download and then you'll get a llama installed so you have a llama which is a what is the program that you'll need to run these models and the models a Lama a lot of great models have been converted and make can use with the Llama if you go to the list of models there's the 65 plus there's Gemma llama 2 mistra mraw lava which is a gray multim moral model which will we will be using Ki in the second part of this video you know there's neural chat they have the dolphin um they have coating llamas there's Hermes too fine which is really good and popular dolphin the straw and a ton of other lot modals that you can download so to download a model say you want to download download zher right so click on zypher well first let's do this let's bring up T I want you install llama so You' bring up a command prompt or whatever you use oops what what what will try from D of be there uh you bring up com command promp and start llama oh llama you would need to type in a command serve and then boom it'll bring up the directions and Al that it's currently running on your system let say you want to download a model say you want to download Cipher right so it'll show in the description a key uh 2inch th Cipher basically it's going to be a command line like run Lama zypper or something like that oh let's see here let's I'm want to choose something else um how about Med llama 2 let's see Med llama 2 might be good so you open up your command prompt and you would type this R Med Lama chip so I want a girl Hadad go back to my command prompt and I paste that in and it's going to pull that model in this case Med llama 2 so it's going to download everything you need it's pulling the Manifest and it's going to download that model to your system so you'll have access it once you start running AMA two ways you can do this you can do a text appr prompt where you can get assistance from any of these models or there's an image The Prompt where You' can use the image and ask a model to kind of decipher what's in the image and come up with a prompt based on that now in order to do this you'll need a set of custom nodes and looks like this is done so just real quick I'll W up and high this is how you talked to the model uh I made a medical question um what causes a sore throat IIT that you know it's Brad up and it's gonna obviously answer me uh and it's very uh you know very pool I said I love you and then it's going to say thank you and so that's cool that's how you get that's if you just want to talk to again all day in this way you can do that you have it set up and easy but I wanted to take this for the next level how we can use this information and this knowledge for prompting now the node you need installed is this it's the F AI tool by impact frames we will put a Lo a link to this so you can just go right to it and it's essentially fairly easy to do all you do installations are right here so you would open up a terminal and then you would copy this command and run that in your terminal then you would go into custom nodes like you normally would copy this and get clone this uh if Ari tools did all right guys when you have everything set up we are going to start on the workflow so we will build this workflow from scratch and we will add note it is under impact frames it's right here so you can meet an if prompt to prompt car and then you're also going to need and if save text okay so these this their handles the actual uh prompt the one you would put in and this would save the text and push it out to Le conditioning and stuff so we will need a low checkpoint low PA he uh K sampler uh we would need two clip and text nodes one for positive one for negative all right stuff over uh with a clip and text node you would actually convert the text to us to an input so we're goingon to do that now and for both okay and I'm gonna might be smaller because is super huge okay so model go into the model clip go into the clip and text nodes right here C conditioning go you know let's color be so we know which is positive this is positive this is negative all right so the conditioning is is positive to positive negative to negative give mean an empty latent that'll go into the latent right here let's just set this up as 1024 by 1024 okay all right let me explain these to you there the question is what you would be inputting inputting into the text okay so let's implict something very very vague let's do um birthday party for feing now okay that is going to be our question or our basically our input text okay now in here we have different bottles that we've downloaded you have your Baka there's gamma there's llama lava Med Lama mistrals uh and so on so in this uh megative you're going to have your uh birthday party for female text M we are going to choose the lava embellish prompt is something that you can add to it it'll add more uh cworks to it you know Epic full body shot let's do a simple style prompt you can add this as well and it'll just add additional keywords it can be analog anime cinematic horror whatever and I'll let's do photographic and the negative prompt as well I usually like to do simple negative okay so that's good now the question goes to the question input response goes the response input and the negative goes to the negative input all right now here's pretty important if this is your first time using these prop using this um these notes you want to have save mode to create that way it creates a brand new file and next time if you use it again you can always you would have to switch it to a pend and c I put mine in create because I cleared all out now these knows okay okay now the question stays blank because it already asked you already have the question then though the response is going to go to the positive because that's basically going to be the positive prompt that it creates so you going go into the text of the positive the negative will go into the megative text of the positive and the turn which basically ties it all together we are going to do a show text so that you can actually see what it says so this is going to have everything in it it's and add the question the response and the prompt um let's see that's all set up let's go over here let's do a checkpoint um I will do my favorite checkpoint ever everything and it's the um gr wasm natural vae we will use the 84,000 million wherever I will'll keep the lad image the same K sampler we will do randomize seed and I'll do 100 steps which you don't have to do but here there F we kind of just kind of stop to giving a 100 steps and we will do uh M psde with carass the noise at one all right then we're going to go need a bcode here and an image to save image and we have to load the V from here and then I going to do a save image right here iing this yeah it's a small workflow it's cute and Compact and it works it do gives you what you need to get done but it also um you know really harnesses the power of these llms because there's a lot of good info they've been training on so much data that they can embellish your prompts really really in a great way that you've never really even thought of so I'll show you so we have everything connect everything settings are set up we have birthday party for females so when you hit two prompt the three a for the model this is the um this is the the the the node that you want kind of zero in on so this is actually going to show you what the prompt will be if you see here right here it says the question birthday party birthday part for female and the response going by lava it is giving us this best quality Masterpiece High detail birthday party celebration fumal candles cupcakes festive decorations all this is added between your Stella prompt your embellishment and the model that you chose and the megative it gives us that as well which we have no negative so it really you know took care of that for us as well for me and the situation and herei heart here is the final image birthday party if his haki B up here is uh what does that say ready um but yeah it it creates a nice picture and again from one simple prompt and running it through Lava it added and embellished The Prompt even more now that is text to image using the um amaing these these these um models these lolm models you can also do image image and I'm going to start that workflow right now and essentially an image to image workflow you're replacing this with another note so let's just do that here so I'm going to get rid of this okay then we're going to go back to impact trains and it is going to be if image The Prompt and it's this guy right here so it's the same thing question to question response to response negative to negative okay now these settings will be similar you're they'll just you'll you're just going to be able to add a photo and we will click a photo to add let's see what we can add here uh all we have is Blanka um let's add this okay this is a generated a girl and Ice um thing so what he can do is ask it to analyze the image saying what is LIF this image right and then you can question mark and then you want to put ple uh prompt cre cre8 sorry SD prompt from image okay so that's what you're telling it to do that's going to be technically the question it's going to be your ask of the model um we're going to select llama lava again because it's a nice multimodal llama all right ever G to do Epic do cinematic on Native front Blitz do simple temperature seed seed randomize top pay is fine and the context and the repeat penalty is fine as well so what you're going to see it's going to analyze this looto in PR so it's BR everything over and everything is set already from before so I hit Q prompt here you will better go to this show text it's going to see we're going to see what it's done or what it says all right so this is it does the question what is in this image create SD prompt Thun image this is what the response was here is the prompt that's qu like a masterpiece epic individual which create a cool there's ice formations so we got what was in the in the image and it created this prompt for us right and here it is the image it's a it's a it's a man then a ice thingy like that kind of so Guys these two What flows are awesome you know you can be playing with this all day that's what she said but you can be really using this and get coming up with great ideas um you can be adding on to this workflow uh I will probably add on the superior workflow to this because I like to upscale you can check out our video on that it's the last video we did and make sure you like and subscribe because we're you know we're all invested in this and we want to be able to provide good content with Ki and everything Ai and present it to you guys in like the simple manner not a lot of fluff and we just want to show you the workflows and show you what you can do because it's just a lot of fun and we we love it so we'll catch you next time or AI fuz zido I'll see you bye

Channel: AIFuzz

Views: 1,296

Keywords:

Id: nZx5g3TGsNc

Length: 16min 24sec (984 seconds)

Published: Fri Mar 22 2024