AI Realtime Image Generation with Webcam & ComfyUI

hey welcome back to the channel where we do all kinds of fun stuff with AI face swapping AI art AI animation voice cloning you know creative AI we're back again with now a third sdxl video where we're doing realtime generation of AI art but this time we are using our webcam to guide the composition of the image and we're running it on our own computer to do this you're going to need to be in comfy UI so I'm going to go ahead and assume that you have that installed already if you haven't why there are countless videos on the internet to show you how simple it is but once you get that done I've included a link to the workflow for this particular process and you will need a webcam and you will need a processor that is pretty good now because this is not an installation tutorial I'm going to recommend that you go to the website for this particular custom node that makes this all possible and read it thoroughly because you do have options I invite you to scroll this page and read everything carefully it's not that complicated but you do need to understand how to use this what the different parameters are for the webcam how to get the webcam set up proper prop L because if you don't follow these instructions your webcam won't work and you won't get the results there's a few ways you can make this work and one way is using the webcam app that they provide now this is where you really need to read the instructions to make sure you get this thing set up properly but it gives you way more options than their option that does not require the webcam app this allows you to Define things like the width and the frame rate of your video where the other option doesn't give you those choices the other option doesn't require that you use the webcam app and the comy UI custom node talk to the webcam directly and all you need to do to use it is to download this image right here and drag it onto your comy UI interface and the workflow will load here's a high level overview of the workflow itself pretty simple I've got things spread out in an illogical way for the purposes of this demonstration as always the model we're using today is the turbo Vision XL super fast XL based on the new sdxl turbo 3 to5 step quality output at high resolutions I'm going to go ahead and use the recommendations here for sampler steps and CFG scale to get started with this so once you get the webcam up and running let's get into it and see what we can do now as always when we're doing this real time stuff we're going to make sure that we've clicked extra options and then we're clicking Auto Q so that the prompt just keeps feeding itself over and over and over and over again to give you that real time appearance keep in mind that the video of me right now is coming through my webcam at a very low frame rate but that's okay for what we're doing here now important thing to realize about this particular technology is it's not like a control net if you know what that is it's not tracking my eyes and my nose in my mouth and my skeletal structure it's more looking at values like color and brightness to determine what shows up on the screen let me just start this and I'll show you right now we have a positive prompt area and a negative prompt area nothing in the negative prompt at this time it just says man right here and I'm going to click on Q prompt and we start the process let's just confirm what we're doing here in terms of the steps we've got three steps and a CFG scale of 1.6 I've got the DPM Plus+ SD and the Caris scheduler and a d noise of about 58 we could play with that a good bit in fact I'm going to pop that up a little bit look at that keep in mind this is not supposed to look like me or anything like that there's no model and again it's not tracking my features so if I was to open my mouth it's not going to open its mouth I would have to say man with open mouth and then it would be open and then again the the position of my mouth isn't going to change anything so how am I affecting this image again look at how bright my face is when I move over this way way see the whole thing moves over but it's not following my face because it's a face it's following my face because it's bright let me show you what I'm talking about if I put something up right here but in this case it's going to show the hand here because I all I've got here is the prompt with man with open mouth but let's say I say man open mouth and dog it's going to put that dog there but when I put my hand up it's probably going to take see it's going to take the color and the shading of my hand hand and make the dog's skin lighter and I'm almost like I'm doing a puppet here see I can open the dog's mouth I can close the dog's mouth I can put the dog up like this you can have him howling at the moon right it doesn't have to be a person either it can be a corn on the cob okay so there's corn on the cob and again if I move it's going to follow the brightness of my face so what happens if I put my arm up let's see look more corn like this right so all sorts of creative opportunities so it does its best with what it sees in the screen and you can really throw anything in here and it's going to have an impact let's do something else scary man in a dark cave L whoops low light okay so now what happens if I introduce something white over here like this is ironic because it is a fake light switch but if I put this up here look it acts like a light on my face it doesn't block it because it's not looking at my features it just says oh there's lights there now if I move this over it might create something else like a lamp or a lantern or anything but just a source of light see very cool right let's use this what would happen if I use this red stop sign see now we get I got him the webcam is reversed so everything that's intuitive to me about where to put my head is all messed up let's get a little more creative giraffe with human head actually I want to do it the other way about woman with giraffe head okay there you go woman with giraffe head that I can move around like this nice so I don't know how practical this is but it's fun to play with now if I move my arm up in here what are we going to get look at that a woman now it's kind of like puppeteering how about ugly troll under a bridge again it's looking at my head to kind of place the troll if I move right in again you want to think that it's tracking my face it's not it's tracking light [Music] it's great when oh look at the claws on the hand that's cool so my hands are more affecting the light and how well the scene is lit if I fill it up with light color you still got the troll regardless of my face but it's lighter for as if I go like this and I move back in the back let it settle there for a sec I can do my light trick where is this going to be over here now I've created a tunnel over there on that side now what if I move it over here on this side there we go how about a scarecrow dressed as Superman how about scary scarecrow there we go that's more what I was looking for and by the way you this is set on a fixed seed and it's never changed so we could change it just pop it up there see if it makes any big difference I move over this way they'll move over that way only because of the light move over this way moves over this way for example if I leave the picture completely it's going to try to do that scarecrow cuz there's a problem there there's no guidance but if I do this look at that the the the brightness of my hand is doing [Music] that I move it over here how about a canary okay so there's a little Canary it does like to show people and so sometimes in the negative prompt I'll put people person man woman human child so there's a canary in the tree so maybe I can create some branches now if I say canaries it's going to try to do several of them right so that means this will probably make one this will make one right this one's up here this one's up here this one's down here how about something fun like squid there's the squid on the ocean floor so I can move him around the ocean floor you know what's really exciting about this is that probably this time next year by at least this will all be real time we won't have this delay steps who needs steps that's what we'll be saying to each other let's make it more interesting a purple scaly [Music] octopus on the ocean floor and let's make it be jeweled with rubies and sapphires look at that now isn't that much more interesting you add a little bit more detail we got a much more interesting image I move over this [Music] way so it seems to be adding more of a light source this time I feel like I want to do something with mushrooms gigantic Mush Room in the middle of a magical forest bending in the wind so now can I make it Bend in the [Music] wind can I bring more mushrooms up here no because I only said one mushroom how about gigantic mushroom let's make them multicolored so my head is determining that big cluster there so if I want to move the big cluster over here just move my head and then if I want maybe more mushrooms over here put my hand up Y and then they start growing and growing maybe just a really big one no it's turning into more there we go if I make it look like a mushroom this is craziness I'm doing this so poorly so that little ball has brought some light over to that side of the screen let's make this nighttime glowing there we go and now we'll get a little light source going I bet you how about this floating glowing orbs let me get the mushrooms on the other side when I bring this little thing in and no orbs just more mushrooms I guess my arm is what's screwing it up if I could just suspend the thing by itself then it could work as a 67 yearold detective New Orleans French order in the rain at night bokeh I know you are but what am I it looks nothing like Peewee but it's got a bow tie there we go that's what you would expect rattlesnake in a tree of Doom I don't even know what that means [Music] ooh that's cool do I have a [Music] lighter wow angry kitten and a ball of string so I'm going to put the string over here maybe up here now it's too big of a string whoa big can I make this string we got to go to the alien creature monster there we go ooh nice at a child's party so let's move back give it some options see there you go again it doesn't have to be like you're trying to track you just give it some room and maybe some lighting to come up with some ideas that is [Music] great she just brings a lot of orange and yellow to it yep Godzilla on a rampage in Las Vegas let's make it real bring it on home okay so there I am all glowy and orange determining where Godzilla is what happens if I bring this in oh an explosion happens wow I just am causing all kinds of Havoc just by doing this right but boom boom boom and then what happens if he's just back here oh the fire is building up and [Music] then okay too much fun if you got the computing power you can do that but there is a service online called cr. a that will allow you to do something exactly like this if you can get on their waiting list and be patient enough to do it this is in fact what prompted me to do this because I finally got access to it and I was playing with this on their site and I was like wait can I do this on my computer and I said yes and clearly I can and I decided to share it with you today and I'm glad that I did are you if you subscribe now I will not look for you I will not pursue you but if you do not I will look for you I will find you and I [Music]

Channel: Bob Doyle Media

Views: 2,303

Keywords: ai, artificial intelligence, chatgpt, creative ai, synthetic media, bob doyle, comfyui, webcam ai, create ai with webcam

Id: ZxseC0xzD_g

Length: 15min 2sec (902 seconds)

Published: Wed Dec 20 2023