Create Your Own Custom Nodes in ComfyUI

hey guys how are we doing today in this video I will show you how to write your own custom NES inside of comy UI I will explain about the example Noe inside of the custom n folder back comes default by onui then I will show you how you can write your own using that example node as a base we are going to take these two clip and cod one for the positive P one for the negative p and combine it into a single prom where we can have a positive text a negative text and then connect it to a Cas sample let's begin by understanding how config UI loads a custom node so let's go into file explorer navigate to where confy UI is on your local machine and uh you should see three folders confy UI the python embedding and update folder go into confy UI and you should see H Custom new folder enter that folder and if you are using a newly freshly installed config then you will have only an example uncore n.p. example file if you have already install a few custom nudes then you will have some folders in here which will look something like this so all of these folders they are a custom node which I'm installed using git or the confy UI manager going back to the vanilla installation of confy UI where we only have the example _ n.p. examples let's start up this version of compi so open a terminal navigate to the confy installation and then you will do your run Nvidia GPU dobat file if we take a look at the terminal you will see that confy UI is start up app and open up the browser interface right away but if you already have custom noes you will see that cofi will first load up some of the custom noes and then continue on loading down below as well as install any of the packages that require before starting the webu this may take some time depending on how many custom noes you have installed let's go back to file explorer and let's rename the example node. py. example to Simply example.py so remove the do example at the end press enter you may get a confirmation box click on yes and now you have a python file open the python file using your favorite code editor I will be using visual studio code this file gives you an example on how to write your own custom node it is just one file and everything is inside it so at the top we have a class it's called example we have an initialization P it's also called a Constructor in some other programming languages then we have a class method in this case it is input types we go down we have a return types a function category and then we have the actual function now at the bottom we have two important piece of information we have a nude class mappings as well as a nude display name mappings so let's minimize this SP let's go back to the terminal and now we are going to run Nvidia GPU be so starting compy y so that was quick let's go back to the terminal and this time you will see import time for custom node 0.0 seconds and it found our example in Disco node. py so the way that custom nudes work inside of confi if a python file which is inside the custom nde order as these values then the confy UI when loading up will load this file but you will load it as a custom nde so if I go inside the web browser right click click on ADD nde at the bottom I have example and then I have my example node which just says hello world here let's try to see how in node Works inside of company wife so I'm going to add a simple node here let's do an image node and we are going to do a load image now in this case we have display name at the top so it says load name when I rightly click on ADD node go into image I also have this name here load image this is going to come in handy when we are trying to name our custom node next we can see then we have inputs so we have a name here we also have a file and this node outputs an image as well as in mask if we take a look at the key sample note we can see that it has the typical inputs that we've seen previously so seed steps CFG these are values that we can change but there's also inputs in the form of another nde output so in L checkpoint outputs in model and this model can be passed in as an input to the case humler similarly we have positive negative and latent image and this sampler this key sampler is outputting a latent image let's try to visualize this example custom node inside the code itself so at the top we have a class called example this name here has to be unique so you cannot have another custom node name as example if you go down let's forget about the init for a second this input types which accept a parameter of s returns a dictionary and this dictionary defines what will be the input types of the new so in here we can see it is accepting an image and we have this image here then we can see that we have fields we have an integer field we have a floating point field we have something called print t screened as well as a string field and all of these are reflected in our custom node so the image at the top the integer field is here floating point field is here print two screen togg button is here as well as the string field hello world here so this function or I should say class method of input types is what is defining these input Fields inside the custom node now similarly the outputs if we go down it's defined by the return types here in this case we have image as the return type and we can see that we have an image as our return type in the example node now this node is also doing something behind the scene when we connect an image to a node and we output something else The nude is doing something and one defines what that nude will do is this function here so this function is telling us what is the name of the function that is going to process what is happening behind the scene and in this case we are saying function is equal to test and test is defined here as a python method or function because it is inside a class we call it method but if you're new to programming just think of function and Method as the same thing now in this case this function is just checking if we are printing to the screen and it will print something on the screen or if the print G screen is disable it will skip this part and just return the image which can actually see this by adding an image and a preview image note so let's disable it first click on Q and we have an output here but if we go into the terminal let me put the terminal down in this section here we don't see anything we just see God promp prop executed in few seconds now if I toggle this button to enable and click on PW now I can see all values that would pass to the function so integer value was Zero load was 1.0 and we have it here as well as the text hello world if I change this value let's say hello YouTube change the integer value and change the floating Value Point cck on Q now I have these updated values here so basically we've seen how we can take the input do some kind of processing with it and output something else afterwards now let's take a look at this section in the code so I'm going to close out the browser and I will comment out these two lines the ner class mapping line as well as the new display name mapping in Python the way that we comment a block of code or a code line is by adding a hash or pound sign in front of that line this means that when the code is running these lines will be ignored I've stopped the previous confi interface and I'm now running a new run Nvidia doore GPB file okay so this is what we got this time we have import field and it cannot import the file because we disable these tube lines so whenever you are creating or writing your own python file make sure that you have these two lines enabled and give it a name now next let's try to see what this example nde does inside the nde display name mapping I'm going to add modify to it C on say make sure to enable these two lines by uncommenting them remove the hashtag from it restart the confy UI now at this time it was successful and we can see the change right away so in our example node it says example node modify there by right click click on ADD node go to example I can see the name change as well now you may be asking yourself what is this example here then let's take a look so if I'm modified and I say example uncore test and I still have my example nude modify here go into the terminal exit out and uh rerun config why I will add the nudde back so go into add nde example and uh right away I see example here instead of example node modifier as we've suggested here so what is happening right now and this is because we change this name here but let's see what happens when we add this node I'm going to click on example I still have the Noe the name says example at top I still have the input image if I try it by adding an image try to get an output from it on Cube I have an image so the Noe is clearly working but the name is not so maybe you already know about this if we go up we can see that we have the nude class mappings and this one is defining what is the name that we are going to use in in order to map the example class that we have so this here this example is referring to the example class that we have at the top so for example let's say I change the name to example and then I will do change click on Save go all the way down leave this as it is change this one back to how it was originally C on save and run confi again now you can see we are getting an error and the import failed again so this example that we have here has to match the name that we have at the top so if I copy example underscore change go all the way down replace this example to example underscore change and then run conf UI again and you see it works I can generate the image as well so now we know what these two lines are doing I'm going to split my screen into two just for this example it's the same file so at the top I have class example this change as the same thing so what is happening here is we are just putting a name here this name can be anything then we are linking this example _ change class and putting it here we are telling confy that whenever we refer to example inside of nude dis play name mappings and there are other places as well that we can use it but for now let's consider what we have on this ring so new display name mappings if we have this example which is the same thing as at the top so these two are the same then we are referring to this class which is here and finally this here this is just a name that will be displayed to the user when they are selecting that Noe so so I'm going to go back to file explorer I will right click click on new file and I will choose a text document bar now if you know how to do it inside of visuals YouTube or if you are using Sublime Text or any other quid editor you can do it from there I will name this example 2py and remove the txt extension accept the change I will go to into Visual Studio code and load that new file I'm going to to use the split screen functionality inside of Visual Studio code by clicking on this button here so on the left I have the example F let me change it back to how it was so example on the top class example go down we have example then referring to the class the name is example node now let's say you want to make a custom node first thing that you want to do is to have an idea of what that custom node will be be doing so let's go into confi and see a couple of things that we can modify I'm going to clear the canvas and load the default view now in the default view we can see right away that we have the positive front and then we have a negative form they all separated so how about we make a custom node that will contain the positive from as well as the negative from in a sing single node so we are going to have something like this these two will be in one node we are going to have positive and negative at the bottom like this of course you can add more but I'm just going to show you as an example how you are going to do it so let's isolate this part now we can see that we will require the clip as input we are going to have a text in the code it's called as string for the positive front as well a second one for the negative one so we're going to have two string fields we don't need to have two clips One Clip build is okay inside the code we going to pass it both positive as well as that negative encoder then we have to Output two conditioning one would be for the positive prompt one would be for the negative okay let's try to implement it let's make as if we don't know all of these by hard so the way that you are going to do it is open the example file always keep it hand it the most important part is the bottom one that is what will tell confy UI that your file is actually a custom node so copy it and paste it in your file in my case it is example 2p I'll go all the way to top and make some space now I'm going to call mine uh as conditioning Ops now remember my class name will have to be this exact name so I will go at the top to class and paste that exact name and just so that I don't get an error inside my ID I will do pass here now this name here I can leave it as it is but I'm going to change it just so that we can differentiate between the two files we change it to example two that is what my file name is at the top and because I change it here here I have to change it here as well then what will be the name that will be displayed to the user when they rightly click on ADD node and then find it from the list I'm going to call it conditioning prompts and this name here you can have spaces but at the top where you are creating the class you cannot have spaces now there are a couple of things as well that you'll need inside the class for it to work if we go into the example node on the left sides we you can see we have an in it we can actually leave this one we will require the class method inputs right now if you remember we need clip string so we're going to Define this inside the input types here there's a good documentation here that we can use basically what you need to do is to return something you need to return in this case going to be a dictionary and we can have optional as well as required Fields but the first thing would be required this also is a dictionary and if you go in here we need clip now I already know that in confy one the clip is actually called clip lies in all caps but if you don't know how these names are whether it's called conditioning as upper Kee or lower casee you can go into file explorer if you are inside of confy I go back once you can press the arrow key here and in here there will be a file called nes. py and right click this open it inside of your editor I will double click on it to open it and this file is quite long it has all the default nodes that come with config y so we have our clip text encoder here and uh if we go into input types we'll see that it's accepting the string clip as well as this capful letterer click this is how confy UI will know what this is I will leave this node as a tab here we can refer to it later let's go back to example two now this has to be a tble or Tule however you want to pronounce it in order to make this into a topple we'll have to put a comma at the end okay now next let's go back into model here and we also need the text or positive as well as an negative FR so again if you're confused as to how to write the code go back into the example and we've already seen text in the example node I do it really quickly and this field here is string Fields it is is allowing us to pass in text right can type in text here so all you have to do is to find where this is written in the example file if we go down we can see string field and then string then we have multi-line false we have a default value of lello world now this is really good because we can copy this entire thing after the click we'll put a comma go down one line and paste everything that we have here I'm going to click on save you can do it by file save or you can just press control s to save I don't need this pass anymore now I'll also need a return type so I'm going copy the return return type from there into here I will need a function to do some kind of processing so in this case the function is called test I will leave the name test and Define the function test here actually a method that's why we are passing self here and when I right click click on ADD node you see I have an example here this example is being defined here under category so we'll copy category and paste it in our file now I'm going to call mine example two just so that we can differentiate between the two so this should be enough Let's test by exiting the server and running a new confy okay so we did not get any errors if I go into the terminal again in this case I can see import times and I have my example node which is the default one as well as example 2 py Got Loaded correct I'm going to clear the canvas right click click on ADD nude and the example with without any number is the default example node the example two has conditioning prompts and this one as you can see here it's accepting clip which we Define and it has a string field right now it's saying hello world and this one is outputting an image which is not correct so we have to correct this plus this field here we want to make it multiple lines because if I go into conditioning the text encoder and write multiple lines here so want something similar to this also this image has to be a conditioning note so let me go back to the example toy and fix those errors so the first thing that we see here is string field is string multi-line is false let's change it to true I'm going to remove the comments that we have at the end you can leave it there if you want it for a future reference and default value I'm going to change to positive text and then I'm going to copy this paste it into negative text at the bottom now let's see this change it's good practice whenever you are writing your own custom node keep testing if okay it's loaded right click add nde example two conditioning props and right away way we can see something is wrong here so if we go into the code we Define The Click at the top that one is showing here we Define a negative Tex and one is showing here but where is the positive the reason as to why the positive PR is missing is because the code gets run from top to bottom so when U is reading the code it is reading required okay we have clip it put the clip here and then it sees a string field positive text it will put positive Tex herei and because we've used the same name here is string field this time is changing it into negative Tex so all we have to do is to change this to I say positive X and this one to negative Tex load come fori once again and this time we can see the nde go got updated correctly so if I delete it and add a new node we can see positive and negative text now lastly we need to modify the image and change it into conditioning so this will done inside the return types and we can just change this to conditioning like so I knows what conditioning means again where I'm getting click a string conditioning that's coming from nodes and you can see here under the clip text and code the return types is conditioning like so if you ever need a particular type that you need to return out then you can find it under the nodes. pyi save restart right click add node example to conditioning props and now we have what we needed so we have a name at the top we have the clip we have a positive Frump a negative Frump and it is outputting a conditioning let's see if we can wire in clip to this if it will accept the clip and if we can take a conditioning and wire it to a key sample positive or negative so I'm going to add a key sampler doesn't matter whether it's the advance or the simple one now we'll add a load checkpoint just so that I have the clip in here so if I take the clip from the load checkpoint drag it on top of my conditioning prompts custom n can see can get connected right away so conf you why is accepting this clip into here now similarly if I take the positive prom from the key sample and drag it on top of my conditioning prom get connected to conditioning here but now we have a problem because this conditioning prompts needs to Output a positive and a negative so we can do it like the casy amplo is doing having two different fields for the conditioning let's see how we can implement this so we already have a field for the conditioning here we can simply add a second field okay so it got updated right away we have conditioning on the top and then we have a second one for netive now if I were to wire everything correctly like model goes into model and add an empty latent do a vcode send this over to VA and then output an image this will not work the coupon so loading the model I'm using the SD 1.5 base model you can use any model for your example okay so it goes into the conditioning prop and then gave us an error it's telling us unexpected keyword argument pause underscore text this is our positive text now everything that we've defined at the top as in input types which is required will have to be passed to our function and the B here so after Sol you'll put a comma we have we have a positive text and we have a negative text these are all the values that we are passing over now in this example case it's not doing anything it's just basically inverting the image by doing one minus the image that's what inversion an image means this in the coding language and it's returning this new image the inverted this is not part we want want what we want is to house this text file include it and get back a conditioning so we'll have to go into noes. py and we can see how it's doing the encoding here it is taking the clip it is taking the text and then the encoding basically Clips tokenize it encode from tokens and then it is returning the condition here so for now let's copy everything from here and paste it inside a test function that we have I will invent the code better okay so now we have this we need to change the text name here to our positive text now we need something similar for the negative prop thankfully the negative prop is also using the clip text en code so if I go into the config Y and uh load the default we can see that positive promp is using the clip text incode the negative promp is also using a clip text incode so it's the same toiz that is happening same encoding that is happening can move the return down and we can duplicate these two lines let just see this toen now is going to be for the negative let's do negative the top one is going to be posi want to pass this here so this top EMP pass it here this top and negative you want to pass it here negative text will go into the tokenizer here and then again we have differentiate between these two conditions that we are receiving so I'm just going to do to or positive and the bottom one is going to be negative now when returning we need to return two things now this first one as well as a second one I'm going to put it on different lines so that it's easier to see so so this is going to be the positive and the second one is going to be for the negative we'll need to change the name condition positive is going to be here pull positive is going to be here and then the same thing for the negative okay so let's close out of compy UI and restart the web interface I'm going to clear and load the default once again then just so that we can compare things I'm going to right click on key sample convert the seed into an input and then track seed down to add a primitive that's going to control the seed I will set the control of the generator fix so that we can have the same seed both this as well as our new example to conditioning prop field going to to copy the text at the top get the clip from here pass it here and then I will leave it as is and generate an image in the meantime I'm going to construct the second one with the key sampl I got the first one I'm going to queue for the second one now and uh we did not get any errors it accepted everything from our custom nude and pass it to the key sampler and as you can see it is actually generating here by this green line I go into the confi terminal I can see in got the pump requested and it is going and generating the image okay as you can see both images this one comes from a default um separated clip text and code Noe and this one come from our custom conditioning promps here and it is the exact same image so we know that our conditioning pump here which has a positive and a negative is working now with this knowledge you can create any customize node that you want so let's say you want to combine the V code with a preview node or a save node so that you don't have two of these every time you want to see the image you can do the exact same thing and decod the latent space image as well as output image within the same function so you can have a function here that is doing both things at the same time you can also have other NES so you can do something like a l low Noe for example it takes a model takes a clip but it also Returns the model and the clip in this case it is patching the model so so we are adding information to the model and returning that new model here so as an example you can see that we have the impact back in this case it is the 2D tailer pipe so what it's doing it's taking all of these input and outputting is single Pipeline and we can take this Pipeline connect it to a different node and it will have all of these information inside this one line so I will stop here for today with this video we've seen how we were able to create our own custom node we're able to change this part here combine it into a single Noe and make it work inside of comy UI now there are a lot of things to explore as custom nodes let me know if you would like to see more videos on writing your own custom nodes thank you for watching this video until the very end have a nice day ahead I will see you next time

Channel: Code Crafters Corner

Views: 1,709

Keywords: ComfyUI, ComfyUI tutorial, comfyui custom node, python, custom node, text encoding, positive prompt, negative prompt, ai image generation

Id: RSp9_fh3JoI

Length: 35min 58sec (2158 seconds)

Published: Sat Dec 16 2023