Advanced Custom Nodes in ComfyUI with Python for Stable Diffusion Image Generation

hello everyone how are you doing today happy New Year in today's video I will be talking about some Advanced concept when it comes to making your own custom node so we can call it as advanced custom nodes however do know that most of the concepts are mainly for python as long as you know python you will be able to create any custom nodes in the back end when it comes to the front end side of things you will require canvas HTML API a little bit of CSS JavaScript mainly nodejs and if you do have those knowledge you will be able to make a good front end in today's video I will be covering the backend side of things so the python custom node backend I want to take a moment to say a big thank you to Sebastian Anthony who is my first Patron on patreon and join the Discord TI big thank you to you Sebastian your support means a lot and welcome to the community I am also using the patreon as a blog post I do not have a personal website and I found it easier to manage on patreon so if you'd like to see the different experimentation testing and what is happening in the community you can go into the patreon at the top I have a join for free button and this will give you access to the free blog post with that let's get started with with today's video and it does come with a few requirements so the first requirement for today's video is to watch this video here create your own custom nde in comy one this video acts as an introductory video to how custom nodes work I also explain how the example node which comes built in with confy one and break it down into its individual components I also show you how you can join the positive and negative promp and make a single custom node with it if you find any difficulty when following along today's video go back and watch this video I will also be talking about some Advanced Concept in Python and I will be using the path Li library for today's project in case you have any difficulty navigate to the python documentation and you will find extensive documentation on how to use the half Library I will not be able to cover everything I will only cover the essentials which I'll be using in today's video and like I mentioned before I will only cover the python back end and not the front end with HTML the canvas CSS and JavaScript let's begin by going into the file explorer navigate to your confy UI installation where you have your run Nvidia scho GP file in there you will see that there is a compi folder go into that folder inside it there will be a custom nodes folder enter the folder and if you install additional custom nodes you will see folders like this if you have not install any custom node then you will only have one file which says example node. py. example now in case you do not have the example file or if you've deleted the file or maybe you've used it and done some modification let me show you how you can get the file back I'm going to delete everything in my custom newes folder then I will go back once click on the back button again to reach the folder where we have the Run Nvidia GP in here we have an update folder open it and we are going to have two files the update comy y.b file as well as the python dependencies one double click and open the update uncore C.B file this will update your confi to the latest version as well as download any missing files once it's done you will see press any key to continue you can press enter and then navigate back to the custom noes folder you should see example ny. example now when we are talking about Advanced custom nde we are meaning to focus on having IM modual and all of our custom noes will be within that mol so we need to tell confy UI to load all of these custom nodes to do that we'll have to structure our project a little bit differently compared to when we only had one file I'm going to click on the new button here if you are using a different file explorer then find the new button or right click and select from the context menu I will select folder and for today's video I'm going to name it as tutorial if you want to follow along to have best experience I recommend using tutorial and once you practice a little bit then you can go ahead and start creating your own folders select the example node. py. example press contrl C to copy you can right click and select this icon here to copy or you can select the file and click on the copy button at the top go into the tutorial folder and then click on the pace icon or you can press contrl + V or right click and select Pace from the menu once you have the file in you can rename the file by pressing F2 on your keyboard or you can click on this button here you can also right click and select rename from the menu remove the dot example at the M accept the confirmation and you will get a python file now in my case I can double click on the file to Open visualist studio code but if you have a different text editor if you're using notepad notepad++ sublime or any of the text all code editor open the folder inside your editor now in Visual Studio code I will have to click on this icon here which stands for Explorer and then click on the open folder option because I'm double click on the file Visual Studio code has navigated me dur directly into the tutorial folder if that is not the case for you select the tutorial folder then press on select folder this will load the folder here on the left side next I'm going to click on this icon here to create a new file and I will name the file as double uncore init double. py you need to make sure that your file is this exact name this file will ser as the main entry point for our project so when configu y will load up it will first check for this file and whatever we put in here will get loaded as custom node so as an example we are going to load this example node using the init part go into example node scroll all the way down where you have the mappings select all of it press control X you can also write right click and select cut or you can go into edit and cut then go into the init fun and paste everything there now at this point you may get an error saying that example is not defined or does not exist which is true in this case because example is a glass which is defined within the example uncore node. pyi however we are inside the init file right now and we are referring to it in order to make this work we'll have to go all the way to the top make some spaces and then do from dot example node import example the dot in front of example is very important because we are telling confy UI to look into the current directory find any file with the name of example _ nude look into the file and then find the class example and give a reference to it in the init file here which we are then using here into the mapping okay at this point you can press contrl s to save go into file and sa so you can go back into your confy root folder double click on the Run Nvidia GPB file or you can open a terminal and run the Nvidia GPU from there now going back into the terminal we can see that our tutorial project was loaded successfully if you see around here if it says import failed it means that you made a mistake somewhere in the code possible that this top import statement is not correct or there are missing brackets it's also possible that you did not put in the example F instead you've copied in that case just remove the mapping section in here so your init file should be like this and then the example file from the top should be like this we have the class example the method inside it return time function and then at the bottom we do not have any of the mappings so I hope you are able to follow along until this point let's continue on let's start with the first project where we are going to have a folder this folder will contain some models this case I'm just going to create some dummy files to represent the models but you can take it as IP adapters IP adapters custom nde have their own custom models which needs to be loaded inside of comy Y and they usually reside within the custom node folder itself so we're going to do something similar to this and these models will need to be loaded inside of comy UI so that we have access to it when we are creating the workflows let's see how we can load some files from the operating system into confy UI so for that I will be using the path lip Library you can also use the OS library in Python the West Library gives a functional approach while the path liap gives a class based approach object oriented approach and since we are working with a class here I've chose to go with the path lib Library so inside the example n. py5 mixing spaces at the top du from have lip import half with a capital letter P this comes built in within python you do not have to install any additional ad B this now to see how the path Works make some spaces again before the class example online and I'm going to do a print statement where I will do path do CWD and then round brackets so this path here refers to the class that we've imported at the top and then within this class there is a method called CWD which stands for current working directory and because it's a method we need to add these parentheses inside and then all I am doing is printing on the console the output that we get from here this will tell us what is the current directory that we have access to in this particular file so all this file is within our tutorial folder and this tutorial folder is inside the custom noes folder which in turns is inside the config y folder and then this config Y is inside the root folder we are invoking pan. current workking directory here at this point and let's see the output so make sure to save the file contrl s to save if you have an existing confi server running you can close out of it or you can press controll C type in y press enter and then do run Nvidia GPU dob file again now once again we can see that the tutorial folder was loaded correctly but at the same time we have a new line here which is pointing toward my root comy y directory so although this line is inside the example node inside the tutorial folder confy UI thinks that we are within the root folder and that is because confy y also needs an entry point and the entry point is within the root folder here this means that everything that we do will have to be base of this root folder we cannot directly call out that we want to go into tutorial folder and then into models we have to go into the root folder confy y custom nodes tutorial and then from here we can go into models if you have any other custom noes which you want to load any text file images videos all of that will follow the same pattern now of course because this is python there are multiple ways of doing the same thing I'm just showing you one way on how you can approach this problem let's go into the example node we can remove the print statement and then at the top I will say base directory as a variable I will point this towards the p. current working directory which we we now know is the root folder then because this example is about loading models I'm going to call another variable as models and I will use this base directory which is the root folder and this path library at the top has a method called join path which allows us to join multiple path together to form One path so you can see the example here and you can read more on it on the documentation side I will have this reference here so that you can visualize what is happening I'm taking the root folder here which is the base D I'm joining it with comy UI so inside here I will type in comy UI this name has to be exactly like in the file explorer if the C is capital you write in C capital u and I are capital you and I will be Capital here otherwise we are not going to find that order it is case sensitive now do a comma the next folder that we need to go into is custom nodes I need to do custom nodes then the next folder is tutorial and then in here I can point to any folder within my tutorial folder I'm just going to make a folder here called models so on the left side here in order to create this models folder we'll go and click on this new folder option and I will name it exactly like what I type in here okay now I have this models folder inside the models folder I would right click click on new file and I would just name it model _ one. save hens this is just to simulate that we have a model save tenses file within this folder of course this is just a text file right now it's not a real model file so I've created four of these dummy files and just name it model one model two model 3 and model four for do save tenses one this is just for the example in a real case we are going to have a real model file in there the next I'm just going to create a Dy variable here for my models and I'll need to get access to individual models that we have here so we can do this using a list comprehension let me explain it how it's going to work so right now we have the base dot which is a path object when we do the join path and join all of these folders we get back a path object we can take this path object there is a method on it called glob which is basically going to go in and look for all the files that we have within this border and I know that I'm looking for safe tensil file here I'm going to put in asteris which mean can be anything any name dot save tenses so the ASX means that the name can be anything the extension will have to be save tenses now this line here will return a python generator and as it is we cannot use the generator object directly inside of the example class we'll have to convert it into a list and then pass that list into our input types in order to get access to it within the web interface so to convert this line into a list we are going to Loop through all the items in it I'm going to say four file in and because we want the individual file at the beginning will do fine and this entire expression has to be inside a list so I'm going to go in the beginning put a square bracket and the end close the square bracket this line here in Python it's known as a list comprehension you can use a traditional all Loop to do the same thing but at the end what you want to have is some variable name which is a list of all the items that we want to have okay so right now if we take this model and we put it inside of our class example it's not going to work because we are working with the path major the path Library will always return a path object so when we get this file here this file as well is a path object if we put the path object object inside the input types it doesn't know how to display it within this comy y buiild here what we want to have is when we click on this button we want a list that says model one model two model three and so on so all of these are text and text in Python is represented as strings so we'll have to convert this file object into a string and all we do is type in s short for string then put the object that we want to convert inside the round brackets now if this part was a little bit complicated do not worry regardless of whether you want to load save tenses or txt file images videos they will all work the same way all you need to do is change this part here to whatever extension you are trying to load okay so a quick recap here we started in this rude directory this is the line that is defining it and then we've join or concatenated some path going from root We join with Cofe one join with custom notes with the tutorial and then with the model now at this point this line here we are inside this model folder since we are inside the model folder we can look for all the save tenses file and then put all of them inside in this and because they are path object we are converting it into Strings so that we can load it inside the classic example okay I hope this was clear and you were able to follow along next I'm going to go into the input types and right now this example note by save it um no need to follow along here by go clear my canvas right click add node example node I can see that it is taking an image as input in integer field flow field printage screen as well as string field as inputs now we want only one input and that is just a drop down similar to this going back into the code I will have to remove everything that is within the required dictionary so remove everything from image all the way to a string field and we'll have to find the closing bracket to it so we have a round bracket here and then it ends here so all of this will go and we'll end up with just a return statement which has a start C bracket and C bracket required the opening C bracket and close C bracket okay now since I'm working with files here I'm just going to name mine as files can name it anything and in order to get the list we need to have a tuple or tle to do this we put in the round brackets inside we need to have a list so if we were doing this manually I would put the square brackets inside it I would do file one then file two and so on and because we already have our list here at the top under my models I'm going to copy the name contrl c go all the way down replace this placeholder list with my models list and then because this is a toble we need to end it with a comma to tell python that this is in trouble I can save scroll a little bit down in order for all of this to work anything that is within the input types required will have to be passed on to the test function here I talk about this how all of this work in my previous video so if you are having difficulty here please check out the previous tutorial I'm going to remove everything and replace it with bar so this name here will have to go here now of course first do not have Real Models to load so I'm just going to use print statements and load the file cck on Save go into files save and then restart config again always check your terminal to see if you have any errors in this case there's no errors I'm going to reload the web UI and click on clear right click add nude example example nude resizing the custom nude we can see that we we have a list that tells us that we have model one 2 three and four all of them were loaded hope you get the point that we are going from the root folder to a custom folder and then loading some files and passing that on like this in order to get a list in the web UI now similarly we can go at the top and we can just change this to jpeg or PNG and if you're doing any animation so you can have a folder here with some images and the SC image one and image 2. PNG now because I'm inside the images folder will have to change this to images and then the rest is pretty much the same we are going through the base folder or the root folder joining the base folder with all of these folders making sure that we are getting the full path or the absolute path to these images and then inside the function you can have some much which is importing these mes loading the images and do need conversion joining them together and preparing those images for your animation sequence now as a last example what you can have is text file so here created a folder called the Styles inside it have two text file one background one color and the same thing applies we go here we change this two Styles which is the folder here then we looking for text F inside color I can have red blue and green inside background I can do Sky the city in here I will be able to select between color and background which I'm loading here then based on what I've selected I can do some python manipulation so I can import random I can also open the file with open do file here I can do something like that this will return a list and I can do a random the choice on this list to get back a random color or a random backround and then I can have a custom node that will create a barable prompt so I can have a variable let say underscore color whenever I have underscore color it just replaces it with a random color base on what we've chosen from here now of course you're not going to import random from here you would import random at the top because you do not want to import random every time we click on the Q promp just import the package once and and use it as many times as you want that's the best practice and it's more efficient now of course there's a lot to cover when it comes to custom noes and it's basically based on your imagination and how well you have mastered the Python programming language let me know if you have any difficulties if this particular video was too difficult to follow along now this section may be slightly complicated but if you copy and paste this in any of your custom node and just change these should be able to load BS images videos or any models from a custom nude folder into your custom nude file now of course the functionality will depend on what you trying to achieve in your custom node with that I think it's a good stepping point I would like to say a big thank you to all of you who supported the channel last year continuing this year big thank you to all of you guys thank you I me nice day I will see you in the next one

Channel: Code Crafters Corner

Views: 965

Keywords: comfyui, custom nodes, python, advanced, models, files, pathlib, code, programming, workflow, automation

Id: x0lHCYEFyuE

Length: 26min 35sec (1595 seconds)

Published: Tue Jan 09 2024