How To Create ComfyUI API Endpoint For AI Image Generation (Tutorial Guide)

hey guys in this tutorial we are going through the process of create API serve with comy UI and you're able to communicate using the API interface with other applications or other AI models that you can generate image or videos through other applications it is uh well uh cannot be say easy but it is kind of simple process to enable these features but then it require a little programming basic skills that you need in order to do something with this so by starting from here we have to go to the comfy UI manager now you must install comfy UI manager in order to do this and you click this gear icon on the top here you will see this setting menu now you have to enable this enable developer mode options so once you enable this developer mode options close this button you will see this save API format now whenever you create something on this workflow diagrams it will allow you to save as API format so allow the comy UI to run this workflow from the back end of the comy UI server now as everyone should know about this when you start the comy UI you load up this command prompts bat files that is running all the things starting from your computer loading up all the custom notes that you install and other files that it has in here now and the last thing in here is set starting server so basically comy UI is act as a web server for you to use everything on here that's why the comy UI are opening the web interface through this Local Host link and you are able to access the comfy UI interface through your web browsers so what we have right now is that we will connect this and we are going to use well by default they have this comy UI script example here now everyone should have these two files when you download the comy UI project in GitHub and you can access these subfolders comy UI script examples you get these basic API examples and the websocket API examples now what I'm going to do is that I will use the websocket API examples as the demonstrations here so this is the one that without any editing but right now we have to do some editing of course to fit into what we need to run this API clients act as a clients to communicate with our comfy UI web server so although this is running in one machine but you can bring this python code to other client apps or client machines that you can communicate through the server address or have something to other applications from remote to connect with the comfy UI server here so by starting that I will be doing a very basic text to image prompting here first so by loading the default you have a text to image so what I'm going to do I will be choosing my Juggernaut SD 1.5 models and here we have the negative and positive text prompt and what I will do is that I will leave this empty but then one thing we have to remember we have to remember is all these custom notes you have to set the unique titles for all the notes here basically that will be easier for you to manage later on in the coding areas in here than without guessing all these numbers and data in the Json files so what I'm going to do in here because for example like this one the clip text in code of positive and negative prom is the same title so I have to change that as well so later on I don't have to be guessing all the stuff so clip text encode I would just say clip text prom positive okay I would just do a short form say Poe here and then right in the negative prom I would just do clip text prom negative here so nug represent negative and then positive and if you have 2K sampler then you will be having a label for different sampler as well for example you have one groups for the first Group K sampler for Generate image and then another groups of K sampler to enhance more detail of the image then you will have to do for example in here I would say k sampler one right and here we have the K sampler 2 then you will be knowing that it is different sampler that is running in the same workflow okay but for Just Right Now demo purpose we don't need too much sampler right now we just need checkpoint models loading the text to generate image that's it and later on we might do our videos to videos or image to image or image to videos or even Stable videos diffusions run everything as a workflow Pipeline and automate the whole process of running the vide Generations in there as well so let's save this one so remember when you save the workflow for API call and you have to click on this button instead of the save button now the save button they are both Sav Json files as well well but the format for API are different so what I'm going to do I will save this workflow uncore API Json files and I'm going to save in our project folders for this tutorial at this moment okay so this part is done but leave the comfy UI on do not close that then we are going to the vs code now you can use other text editor if you prefer but for me I just like to use Visual Studio because I use that for a long long time and right now the vs code is just more flexible for code editing so right here we have to check first of all the server address now we have to see if we got the same server address in here so in here they said now the comy server set the 127 point and then the port number 8188 so check back again in here we see okay we got the same values on here so no problems for that now in here they are start to creating clients IDs and then requesting the server address that's fine again this template this coding you don't have to create from scratch you have the example in here already so you can copy one of these files and bring it to other folders location and start editing your own copy of API clients for your project so in here we can see get image and then prompts ID blah blah blah you can go to this part as well now in here we have to change this part for our workflow this is a predefined simple workflow Json format that is hard coding in here as you see it's a prom text and then it hardcode the whole Json files actually is writing into here but we don't need this because we have our Json files for workflow already right so we delete that and also we don't need that because we are going to load the Json files I have to do a little coding on this part okay and then as you can see in here we can locate the notes numbers and input and text and then we can do the text prompt like this so in our examples we have positive and negative prompts so in here later on we can set our prompts this is for positive and maybe another one is for negative so say low quality extra hands extra legs something like that okay this is for the negative prompt then the seat numbers we can set that as well just any values in our workflow in comfy UI we can set this as well but you have to set the correct one because it doesn't do any error Corrections or checking on the interface like comfy y here so we have to make sure you have set the right values in order to run this otherwise you will get an error message so let me do the coding in here first so in here I have add the one little features to open up the workflow API Json files which we just saved from the comfy UI this one so let's bring it to here let's see if we can do that yeah so we got this workflow API Json files then we have another line for reading this Json files so we defined this Json workflow Json data and then equals to f. read so what is the fs is here we have defined these values for open up the files of this Json text files here then lastly we have defined another IO values in here I call this the Json WF so just for Json workflow shortened names and then I put the Json load so using the default python Json functions to load this workflow data into these variables then after that we don't need this one because this is for the by default we just delete that text prompt the hardcoding all the Json format in there so starting in here we can edit our values in each custom notes so for example we got this positive text prompt and negative text prompt so let's find this one where can we find in in our text prompts here that we can minimize our error so score it down as you can see now it's way easier to see the custom notes locations and all the save data in here when you have the title you need imagine if you did not change the titles you need you got everything is clip text then you will have to guessing which one by whom just by seeing this Json files so in here we can see this is the text prompt for positive then this is the 86 right so as we are using the positive prompt six and then we do the input so where is the input is that is inherit the 86 custom notes the input and then the text is inherit under the input space here so we got the text correct and everything is correct for this one so this is a pass okay so next one we got to do negative prompt so where's the location for negative prompt let's scroll down we'll see this negative prompt right here and it belongs to id7 so in id7 we have to change this Json WF id7 and then again the input text it is the same as the positive text prompt we do this in here so let's say we can also include in this one say that it's no text Watermark and Nar so we are all good to go for positive and negative prompts Now set the seat numbers for case sampler nodes again we can go find where's the case sampler so for the fast way I can do a search on here justrl F and you can search the case sampler names and there you go you have the case Samplers in here that is inherited in ID3 right so ID3 input so seat numbers right here so I like to use not a hard code values in here instead I will put seat numbers let's say a seat nump okay I call this the seat nump and I will do like uh 777 this is for my default seat numbers and then okay this is not a c so we don't need this semicolon that's all I need I got a bad habit not bad I mean the old habit of getting all this semicolon or like this in just like the C Styles but in Python we don't need that so that's all we need for here and then lastly we add the seat number variables into here so the Json workflow in case Samplers note the seat numbers are equals to this seat numbers so there you go and lastly we need to go through all these processing and generate the result right we need to return the result in order to let us to have something to see in front of us whether you are doing an image or videos so let's change the last part in here so we got this template for output and showing the image but then I want to save the image to somewhere else maybe in the files folders maybe we can show it on the web browsers anything so we got to do that okay so lastly we got to do is save our image the generate image into our app or this software in the same folder here so that will be here in this line the image save and then I put the files name I would do like the format is output the seat numbers and also including the notes ID which will be very unique files name for this PNG files because we will get this notes ID in each four Loops okay so here we got the for loops and we will have the image data in this notes ID so that will be the if you have one image so it will go through this for Loop for one time if you have generate four image in here then you will have for Loop in here to save four times of the image individually so in here we got from P import image so this is the image files type and then we have to import IO so IO is like commonly in many programming language this is for files in input and output library then we are going to have the image loading up the IO Data from the image data here so put that data into this image variable and lastly we will save this image into the same folder in here but then we don't need image show in here because we have the files already so this is by default we have in this template in confi again you can download this or not download I mean you will have this eventually if you download The Comfy UI so it's right here so let's go back to our code here so save this Python and we can test this that is a very easy simple workflow that we do let's check one more time again and see if there's anything now the next thing is we have to install and enable the websocket in our comy UI project so by doing that we will go to Windows command prompt and starting into here we will go to our comy UI pendence embed area so right here let's go to CD my path in comy UI mostly we are all installing the comy UI portable versions so it will have these python embedded files beside your comy UI so it will be something looks like this so it will be something looks like this and you have the comy UI Portable Files and then inside this one you got the confy UI and then you got the python embed here so what we got to do is we got to install the web socket in using the python embed dependence files here we got to use this python XE to execute everything so going back to here we have this one so we have the python embed location already so we got to do is using that python XE files and do executions for installations we get the pipe and install and the package is websocket client so this is the thing that we need to install in here so press enter it will be installing very fast this is just a very little scripts to install so once we got that then we can move on into the next step right now we comes to the code project folders that we just added in versus code so in here I have the locations of the web shocks and the workflow API Json so in here what I'm going to do I will type the CMD enable command prompts in this path then I'm going to to use the python xit to execute because here I have just installed a websocket client library in my comfy UI python embedded Independence Library here so just copy this line and just paste it on here so after the XE we have this websocket API examples Pi so this is this file's name so let's run this and see okay so after finishing running you see here we got our image so if you want to have something showing up popping up in front of you without a notice anything is running finish running then you can do something in here which show you the image I know some people want that and some people don't like it that is up to you but for me I can just skip that part and just have my files saved in here but let's run one more time because I just enabled the show the show function and it will show up the functions of our image here right and one thing you can do is okay so in my example here what I can do is I can put a random seed numbers in here so every time I execute this command prompts it will generate other image for me instead of having the same image here so let me do this part and I will generate a random seat numbers here so I have did random numbers for here that is not going to be a hard code same seat number every time so you will have other image generates when you execute it in our prompts here so let's try it again and you will see what I talking about and also I will show up this one and this one it should be really clear for you to understand what happens right here so on the top here this command prompt is the server of comfy UI as usual well we always run this when we are using the web interface in comfy and that is the same one and the second one in lower part here is the one that we doing the executions of our scripts of this files the websocket API scripts so this act as a client app you can put this in other locations or other device maybe you have other laptops that you want to remotely run stuff in your main computer server you can do that as well so what we got to do is we executing this again the same path okay comfy API tutorial 1 and then the code is the folders in here and I'm going to execute this one more time and again I'm using the python the set of python that installed within the comfy UI folders to run this so just click just press enter and you're able to run this oh actually I have Frozen this one oh yeah forgot about this the seat number random integer needs two set of arguments so A and B like between 1 to 100 or 1 to 2,000 like a set of a range for that and we have to go back to here and set it up so let's say zero say 1 to 9 and then there we go let's do that again yes it works so there you go it show up on the client side it will pops up this image and we will have also the loading log in our server side command prompt Windows as well and and also it won't show anything on the client side on here because it's just a command prompt to execute this python script so what you will have is in your folder you will have this so in here it is also following our file name structures that I did the seat numbers so the seat numbers always random right now and then the nine is our note ID so the note ID for safe image is or preview image is id9 um let's go search that we can do that in here yes we can see the nine is right here that is the save image notes so in theory we will also have this image save that in the serfer side which we will have that in the comy UI output folder as well yes we got this in the output folder these two files is saved by default in comfy UI as well and also we have saved one copy in the client sript or the client side application as well so there you have it this is kind of advanced stuff us using comy UI and you need some programming basic Foundation skills in order to play around with this so yeah feel free to ask if you don't understand anything in here we can go step by step if you think this is too complex then just go back use the confused that is comfortable for you then that's fine but then I want to build a workflow that automate everything without me to open this web UI of comfy and I can just execute one command prompts in Windows and and then I can automate the whole entire videos animations creation process in here so this is my automate goal of creating this so step by step first we will do this websocket API call and then next one we will try to integrate with more valuables more values or the input image in here or input videos to videos in here then eventually we will set up the whole process in here and once it's done we can use a large language model to act as an AI agent to control this script as well to execute this API script to automate everything for us so I hope this inspire you to run comfy UI and AI application not just using normally in here instead we can automate things like really automate this whole uh software as our workflow pipeline so hope this inspire you and I will see you guys in the next videos have a nice day bye

Channel: Future Thinker @Benji

Views: 1,987

Keywords: ComfyUI API, communicate with other applications, AI models, generate images, generate videos, Comfy UI manager, developer mode options, save API format, websocket API examples, text-to-image prompt, Juggernaut SD 1.5 models, JSON files, Python code, Visual Studio Code, server address, client IDs, request server, JSON workflow data, ComfyUI, stable diffusion workflow, ComfyUI API Endpoint

Id: 7ODVE7So7j0

Length: 21min 26sec (1286 seconds)

Published: Mon Feb 26 2024