Hey everyone, welcome back to the channel. In today's video we are going to talk about ComfyUI updates for the month of May 2024. There were a lot of updates throughout the year but in this video I'm going to cover only the month of May 2024 And also, I will
only cover ComfyUI as the base ComfyUI project. Not any custom nodes. Anything made by the community is not covered in this video. Maybe for a future video. We'll see. Let's get started. new features and support. We now have a webcam node. You can find it by right-clicking,
selecting add node, going into image and then you will see webcam capture. If you add the node at the top, depending on your browser, you will get a
notification asking you to allow webcam access. It may also be
referred to as camera access depending
on your browser. Now if you click on
denied, maybe by accident or intentionally,
you will see a notification telling
you that it's unable to connect to webcam and
permission is required. Now once again, depending on your browser, you will need to go back to your permissions. For me, I can simply click at the top and allow. Once allowed, the capture will appear as a live feed. You can click on capture to get that specific frame. And if you want
to use that frame in your workflow, you
may want to add a preview node
or connect it to a VAE encode and
then use that there. Now you will still need to click on the Queue prompt to send the image across the workflow. Now it is important
to note that after capturing the image,
your webcam will still be on, even
if you delete the node, even if you
clear the workflow. The only way I found to deactivate the webcam is to click at the top and just refresh the page. So be sure to check
whether your webcam is on or off, especially if
you added the webcam node, captured the
frame, and then you're spending 30 minutes
building your workflow. Even without the webcam node, the webcam will
remain activated. So once you've
captured the frame, just refresh the page
and double check. Now the next section is about a new model, the SDXS model. And here's the project. It's real-time one-step latent diffusion model with
image conditions. And I know that we have a lot of one-step diffusion models, but this is just another one. and right now ComfyUI
has added support to detect and
identify those models. As of now, we have two models, the 512 and the 1024 model. It's basically similar to SD1.5 and SDXL models. I will leave a link
to this page in the description
below for more details. If you would like to test the model, you can go to the link in the description below. Click on files and versions, then click on UNET. Download the Diffusion PyTorch model. It's about 1.26 GB. Then go back into files and versions, download
the text encoder model, and lastly you may want to download the VAE as well. There are two versions, the small one which is about 10 MB, and the large version which is about 400 megabytes. You will require all three files. Actually you will
require the UNET and the clip or
the text encoder. The VAE, you can use the default SD 1.5 version. Once you've downloaded them, go into CompVUI, models, for the UNET version, place it in the UNET folder, The VAE will go inside the VAE folder and then the text encoder model will go inside the clip folder. Once you have all three, go back to ComfyUI,
load a workflow. In my case I am going to use a default workflow. Now instead of using the load checkpoint node, we are going to add three nodes. The first one is the UNETloader. This is going to be for the model. So it connects to the ksampler. The next one is load clip and this one connects to the positive and negative prompts. And lastly, you can use a VAE loader. In my case, I am going to use the default SD 1.5 version. I found that I get better result using the default VAE. Experiment to find what works best for you. Now if you leave everything as is and you click on generate, you'll end up with a mess. This model requires one step. As for the CFG, the paper suggests or uses a CFG of zero. I found that one works best for me. Experiment on your own. Definitely check different CFG as well as step counts. As you can see, it is really fast and we get good images. Now, as of now, we have two base models. They are similar to SD 1.5 and SD XL. One can generate
512 and the other one can generate
images of resolution 1024. Now additionally,
there are a few updates, we have a
dream shaper model, there's the base
version, the sketch version, as well as
an anime version. You can download and test these models at your own pace, and let me know if you enjoy these models. User Experience Enhancements. Previously on Windows, you could press Ctrl + delete to clear the canvas, instead of clicking the clear button. This shortcut has been removed. We now need to use Ctrl + backspace to clear the canvas. Once you press Ctrl + backspace, it will ask you to confirm whether you want to clear the workflow. If you were used to
Ctrl + delete, you'll just need to switch
to Ctrl + backspace. It's going to be a slight learning curve there. There is also a new feature or a new button, Reset View. Let's make as if
you have a large workflow with nodes
scattered around. Clicking the Reset View, it zooms and centers on this blue outline, making navigation easier. Alright, next we have better zoom control. I'm sure you know by now that you
can use the mouse wheel to zoom in and out, but if you do not have
a mouse wheel, let's say you are using a laptop with a trackpad,
instead of using gestures, you can use control + shift and then drag up
to zoom in, drag down to zoom out. Now some of you may have tried the shortcut key,
control and the plus key but doing that will zoom the entire browser making
everything bigger. We now have alt and the plus key. This will zoom just the canvas and if you use the alt and the minus key it will zoom out. Next one is for masking or inpainting. Now when you use the default
mask editor, and I'm talking about the default one, not a custom
node or community added one, if you have the impact pack, you
will get some editor, but I'm not talking about those, I'm talking
about the default mask editor. Now once you go in, you will see that before they've added a thickness slider, we now have an opacity slider as well. It makes it easier to see where you are actually painting. By the way, you can also change the color of the paint. It goes from black, white, as well as negative. Now next one is about nodes again, but it's for the widgets. So when creating workflows, sometimes converting
the widgets to inputs is easier, especially
if you need to connect one output
to multiple inputs. When you right click on a node, for example the case sampler, before you would get the option to convert seed to input. Now you will have to go into convert widget to input. You will get a sub menu and in there you will see a list of all the widgets you can convert to inputs. Similarly, if you are converting from
input to widgets, you will get the list and then you can select from there. The next one is for LoadLora node and the range for this slider has been increased from minus 100 to plus 100. Better workflow and conditioning. Let's simulate downloading some workflows from an online source. Let's make as if
you have a .json file, you have a
ComfyUI generated image and then you also have an image that you've
downloaded online. The JSON file was created using ComfyUI, the ComfyUI image was also generated
by comfyui, which means it has the
metadata embedded in it. Now the last one, screenshot, is just a screenshot image, it does not have any metadata in it. So here in comfyui, let's say I want to load these files, I can drag the workflow.json file into comfyui, and it will load as you would expect it. Next, I can drag the image which was previously generated by ComfyUI and this one as well loads the workflow. Now, if I drag the
screenshot into ComfyUI, I get
this nice indication that tells me the screenshot does not have a workflow in it. Previously, it wouldn't do anything. It would just leave me unsure whether the canvas
was updated or not. I would have to refresh the page and check the workflow manually to see if something changed. Now the next enhancement is related to conditioning nodes and that got improved a little bit. If you previously had images using these conditioning nodes, you can maybe try it again and see if there
are any changes. Performance improvements. The first part focuses on speed enhancement. Now if you're one
of those people who likes to go into the manager and set a preview method, the latent2RGB method is now better. It has been improved and it is now even faster than before. So you may see a little bit of speed up when using it. The next speedup involves two nodes. If you use the image
sharpen or image blur nodes, by the way these are
the default nodes built in ComfyUI, these two
nodes were also improved, so you may see a
speedup when using them. And of course I'm not talking about 1, 2, or 3 seconds speed up, but it should still be noticeable. Next one is about using upscale models. So ComfyUI's resource
management will now automatically
allocate all free memory. based on how much VRAM you have. This was default behavior, but it's now better. Now next one involves those on low-end systems. If you have low VRAM, before COMFY was detecting that and it changes the VRAM state to a low VRAM by default. Now with the new update, ComfyUI will detect it but it will set the VRAM state to normal. So if you are affected
by this, if you're not able to generate
images, if you're getting out of memory
errors, you may want to change this behavior
back to low VRAM. You just need to edit the run NVIDIA GPU file. You can use notepad and add --LOWVRAM at the end. And just as a
bonus, you can also add
--always_normal_vrampause if you want it to always be normal VRAM regardless of the
memory available. Okay, now let's talk about optimization and compatibility. If you have an Intel GPU, there have been significant
improvements. Previously, Intel
GPU support was mostly in beta or
trial stage, but now it's fully integrated
and the GitHub repository provides instructions on installing the drivers and setting it up correctly. Next one involves the system login. Now in the terminal you will see even more logs as if it was not enough but some of these logs are actually useful. It tells you whether
there's a missing node, broken
workflows and so on. I was not able to simulate these to provide screenshot but in code it's going to look like this. It will also display the Python version, the PyTorch version and other relevant information. Now lastly, we have some bug fixes. The first one is for Mac users. Sometimes when
you were generating images, you would
get a black image. That was a bug, now it got fixed. The next one is for ControlNet. So previously, or for those on low-end system, when loading a ControlNet model,
it would sometimes take more memory than required. And some of you may have experienced that the control net would not upcast to the correct precision level. Now, this got fixed, so you may want to check back again and see if your workflow is working now. This one basically affect new installation of ComfyUI. The download package got updated. Basically, in the requirements.py, we have the correct NumPy version added to the file. So this ensure that the NumPy version is always correct. And last for today, there were issues reported by users when loading JPEG or PNG images, and that
got fixed as well. All right, so that was it. Thank you for watching until the very end. Let me know if you got any speed up or any improvements by upgrading to the latest version. Thank you for watching. I will see you in the next one.
