Real Time Camera to AI in ComfyUI

hey you do you want to learn how to generate AI in real time using your webcam like this then stay tuned enigmatic e hey everyone today we have something very special I've been wanting to test this out for a while and it's real time generation through a webcam or through just your camera in general uh through OBS or uh webcam that you connect to your computer through USB and it's all thanks to let me show you it's all thanks to this person right here toy XYZ this person has been doing a lot of really cool things with comfy UI and as you can see he even has right here uh open pose skeleton that's moving around and it's generating everything in real time and this is very cool he just released a new custom node that is pretty much yeah it's like real time webcam to come for UI generation so I'm going to show you how to install that I'm going to show you how to get started it's quite simple once you get started if you're not familiar with comfy UI I do have a video on how to use it uh there should be a link above that you can check out so that you can watch that and get introduced to how this works and then you can come here and continue on to install this new node all right so I'm going to put this GitHub Link in the description so you can find it down there uh again shout out to toy XYZ definitely go check this person out go follow him on Twitter so let's start by installing this new node we're going to get this link right here where it says get clone we're going to copy this whole link contrl C to copy then we're going to come to our comfy UI folder and when we come here we're going to go to custom nodes here what you have your url we're going to type in CMD and then enter you're going to get this new window that pops up and you're going to just press contrl V to paste the get clone and website link and you press enter it's going to install it into this folder all right so after that uh you want to start up your comfy UI you can can actually save this image here uh save image as and then you can load that image onto here you can either drag it or just load it from this button right here and then load it from there and then you're going to have the workflow already set up for you so there are a few things we need to consider like the model and uh in this case I'm just going to use aniverse you can use whatever model that you want to use for Laura you'll need a LTM Laura I'll put a link on how to download that uh from citi.com put it in your Laura folder and then uh it should appear right here for me it's named pytorch right now at the moment but it should have the correct name when you download it from here uh make sure you download whatever version you're using in this case I'm using uh 1.5 models so I put that in there I have to make sure everything else is correct so like the vae make sure you put the vae that you actually have and I think that's pretty much it we're going to uh you know this obviously is going to affect the generation and what's going to be shown so we'll see right now what this is going to generate uh and right here we have our webcam path we have to go to where we have our custom node where we just installed this node and then you're going to go to that folder it's going to become for you y toy X YZ test nodes you double click there you're going to go to capture Cam and then you're going to click on run bat once you do that it's going to open up this window here and you're going to have some parameters here so what you want to do is you want to make sure that you have an output if you don't put an output it's going to do I believe the default output that's right here but uh yeah I I think it's best if you just make sure that you create a folder and put it the output in that folder so make sure you select an output folder in this case I'm going to be selecting this folder I'm going to put it in here actually let me just put browse and then just press select folder right here and then when you press uh start then there should be an image that appears here here let me actually press start right here okay so you come here so you're going to see this capture.jpg file and it's going to be like kind of real time footage of what's happening so that's what's going to be feeding into here so now that uh I put this in here and I made sure uh it's it's called Uh capture make sure that this image was created in the folder that you put as your output so now we go to our queue we come here to where it says extra options and we're going to put Auto que so that it can keep generating and don't worry it's this is a preview window so it's not going to save it anywhere it's just a preview you can save it I'll show you how to save it later just by creating a save image node but uh yeah so let's just go ahead and cue this up and now you're seeing this image of a woman in real time so what's really cool is that you can just you know keep this going and just change the prompt like wearing a beanie for example it says photo of a girl wearing a gray beanie right now it's actually detecting a beanie and here I can actually even put some more embeddings maybe some some extra embeddings in here just to see how it affects it like yeah it's getting a beard so maybe I put beard this is like if you want to catfish somebody at some point right I'm just kidding yeah so you're getting this real image and it's I think it's really cool um let's see uh let's see like uh The Incredible Hulk Hulk green muscular and then I'll suddenly um I'm looking green and muscular uh yeah so all right so this is with a control net I do notice that when I add a control net it slows it down significantly yeah so this is pretty cool um let's see the fresh prince of balair oh oh my gosh that's kind of funny oh man this is really cool yeah all right so let's try some other models nice does it get my fingers [Music] yeah that's sick all right everyone well this is it for today um definitely try this out experiment with it go follow toy XYZ and enjoy enjoy this uh oh yeah to if you want to save this you know it's very simple you can't save it as a video I don't think um you can probably just save it as um just save image and it's just going to save the images uh I think that's just going to you got to be careful with that because then you're going to have a folder full of images and you know make sure that you uh are aware that so yeah all right everyone thank you so much for watching like always take care God bless and [Music] peace

Channel: enigmatic_e

Views: 15,935

Keywords: comfyui, animatediff, stable diffusion, generative ai art, ai art

Id: rl1znJNarAE

Length: 7min 24sec (444 seconds)

Published: Thu Nov 23 2023