Live link to ComfyUI Integrate AI with your 3D software Workflow Tutorial

Hello friends in this video I want to explain how to create these kind of uh connections between any kind of 3D software with confy UI here this is my 3D View and this is a render that is made with confy UI this is a floating window that I am placing that here okay I can move in the 3D space and I can take a screenshot and send a screen shot to confy UI confy UI automatically receed the new screenshot and after that it will try to process the um input a screenshot with control n and some other models and after some seconds the render with new view is ready for us okay this is my workfl and in this workf this is my total workflow I will try to explain all this the steps one by one also in this workf follow I can select a design style uh and the render will be down same as this side so it can help the peoples that are working in interior design uh okay so let me change the design style and cue the prompt let me move change this window and make the smaller okay Q prompt this time it's trying to generate the render with that style let me close the floating window okay the result is ready and you can see the render okay so let me tell you that what you need for creating this workf okay at first you have to install extension and confy UI with the name of mixlab nodes so go to the manager go to install custom search for mix and install the comy UI mix lab notes uh this extension is providing many useful notes but in this work follow only I am using two notes of this extension the first one is load image from pass okay it can load a screenshot from a directory that I am pointing that and it can automatically check the directory for making any change so each time that it is receiving some change again it will load the image automatically and start the queue so this note is very useful because it can take a file pass and also it has a watcher okay the second note that is very useful from this extension is floating video okay this note if you click here and select picture in picture okay then you can see that you have the final result on a fluting window and when you are going to your 3D software you [Music] can see this and for example you can place the floating view anywhere that you like and you can see here if I change my camera view and get a new preview in this preview it is it's trying to save a JPG file in my hard drive and after that the extension automatically will be load you can see it here the new preview and it's a starting to extract some lines some normal information and some deps information and after that okay it's surrender and if I minimize my Google Chrome I can see that something like this so sometimes I think it can um provide some idea and inspiration for 3D designers and also interior or exterior designers and totally it's very powerful this is another test um here in this file you can see everything is very very simple and I spend one or two minute to make this uh 3D file and also in this case I can send a screenshot save a screenshot from this View and after that you can see the render or if I change the camera view here okay this is my second screenshot you can after some seconds you can see the render and it was the style that I was selected uh for my render okay it's ready and you can see also in confy UI if I do some change on my control net and change the string of uh line art and normal and add some strange to the depths control net and do another cue you can see that it can do the interior design for us with more creativity okay it's on processing mm okay it's ready and here there are some style reference that I have collected in the folder and this time for example I want to se to select this uh wooden kitchen as the design style so I have to open my conf UI and direct my design style here I'm using the IP adapter and Q prompt this time it's trying to design this bedroom based on this image okay so okay it's ready and also we can place our reference here you can see that it's very very similar okay so in confy UI I want to explain to you that how you can to create this workf in confi at first I told you that you have to install the mixlab node and you should use the load image from pass this node mode and uh you should save your screenshots from any 3D software in a specific folder you should place the name of the folder here and after that enable The Watcher don't forget about the Watcher it's very very important uh after that here you can see that uh I am using three different kind of uh control net models the first one is line the second one is normal and the third one is deps about the depths uh I am using the new preprocessor with the name of deps anything and it can provide the three different model for provide for processing the deps if you have powerful GPU you can select L and for control net model for the deps I use I'm using the uh lose control net okay you can download the you can download this control net from install model here you can search for dips and you can find the control net for deps here okay control net box steps lose control fp16 okay it's very important okay let's dipe into the workflow at first I want to explain the key features of this workflow and after that I will try to provide uh more explain on outer parts okay as I told you the first feature in this workflow is looking to the mixlab note extension that you should install on the confy UI and you have to use this note for loading automatically loading the screenshots in your folders for example this is the folder that I have and every time that I am that I am placing a new file in this folder this note can can detect that because the Watcher here is enabled okay so automatically it can detect there is some change in the image and again it's trying to load the new image and automatically it will start the queue this note is very very important after that uh for making the floating window again I'm using the floating window mixlab node and if I click here here I can select the picture in picture and I can have this floating window and I can resize that so if you minimize your Chrome still you can have the floating window so these at first these two notes are very important in this workf after that here you can see that I'm trying to load my checkpoint after that I have uh three different control net with pre-processors line normal and deps okay uh about the depths it's very important to know that I am using the this model control net the Box STS okay it's very important to know that and uh it's new control net model for STD 1.5 for calculating the deps and it's working great for interior and also exterior design so go to manager install node search for depths and this is the name of the control net model that I am using control box steps loose control fp6 and for the preprocessor in the deps map I am using deps any anything and the vs 14 also can work fine for me but if I select with L it can work a little better but for example it need about five or six second more for calculating so if you have a powerful GPU you can select the VL um other things are very normal and simple here this is the IP adopter and I'm using the IP adopter for transferring the style from image input okay and also it's very important for IP adapter also I am using the IP adapter plus sd15 do saf and share model again you can download that from the install models you can write the name and download the model and for the clip Vision okay I am using clip with edge4 lion and also exactly you can download the clip Vision model from the manager uh in the in this note it's very important to turn on the IP adapter plus because I am using the plus model okay so if you are using the plus model don't forget to turn on the IP adapter plus and one another note is that I'm using this node prepare image for clip Vision because it's trying to fix the size and dimension of my style image and after that it's trying to send that image for processing with clip Vision uh for My Control net okay um I'm getting the screenshot from um from a JPG file here this is a JPG file and each time it can be in different size okay and um sometimes it can make some problem for me if I cannot check the dimension of the image so there is a very useful note with the name of Pixel Perfect resolution you can access this node from control net pre-processors and here this is the note Pixel Perfect resolution uh this note is helping me to convert WID or height okay it's looking to the minimum size of the image to some defined size for example if it take a image with the um maximum uh White of th000 or 2,000 it can change the size to 768 anything that you can write here or um if the height will be the lower value it will keep this amount as a lower value for the height this node is very perfect so I'm trying to uh use the Pixel Perfect resolution before starting the preprocessors you can send the resolution output to your pre-processors input okay for example here there is a node and you can adjust the resolution manually and if you click the here and convert resolution to input then you can use output of of the resolution of this node to input and it's working fine so it's very important for me because it can um save my GPU and also processing time so I did same thing for all of my preprocessors and also here I have one another note generate resolution from image uh from one of the image uh of my control net models I'm getting output and sending output to this node and it will extract the dimension of that image and exactly I am sending again I am sending the dimension to a empty latent so in this case when I am changing any screen screenshot with with any size my control net image will be adjust automatically and also the output size of my K sampler will be adjust automatically um these are the tricks and important uh parameters that I have in my workflow um one more time I'm trying to show you all the things one by one so this is my checkpoint that I am using realistic Vision V60 uh here I am using the freu I was test uh also I have added the luro LCM and also s AG but at this moment they are bypassed and if you want you can enable them uh but if you want to use the L LCM don't forget to adjust your samp L name also to LCM but here now they are they are not active here you can see that I have uh some positive and negative prompt and this is my control net I'm using apply control net advance and and also here you can see that uh the processing will be stop at before ending the process for example because I can adjust the end to the one but at this moment I prefer to work with about the 084 and yes there is not any other important parameter and you can see and exactly you can design a workflow like this and yes it can work for you uh just here I have some note excuse me okay here I have some notes about the a strange of the control net uh sometimes if you want to keep the structure of your screenshot okay it's better to adjust the weight and strange of the line and normal control net to some high value for example about 06 or 07 and uh change the depths value to for example 02 or 03 but if you want to get some creativity okay it's better to keep the line and normal in low value and increase the deps for example to something about 8 08 uh and also there is another note for um IP adopter for V of Ip adapter normally I'm trying to use something about 0.8 um but there is a trip there is a tip because if you re if for example if you're designing a bedroom and also you are selecting a bedroom of a photo for your style something like that can work fine but if you're um for example let me show you for example with this flow sample okay okay when I am q prompt with this flow [Music] sample okay you can see that it's trying to make some huge flow okay and everything is flow in this bedroom and we cannot see any bedroom in the image um because the concept of this image is completely different by bedroom uh and if you change the value for example to something about four in this case after that we can see the bedroom and um we can see some color and material and also some painting uh in similar style to the image so it's very important and keep that in your mind if your subject in your style is completely different so you have to adjust that so uh and yes there is not any other thing and this is very very simple up scale okay and also let me explain to you uh for the oper scale uh again I'm using the freu every time in SD 1.5 I'm trying to use the freu and here I used uh extension with the name of sdxl prompt Styler because I like the prompts of Sai photographic and some other thing and I have placed my uh prompt here and I sent them into some clip text okay and let me open these two okay these are normal text encoders and the clip is coming from my main check point okay you can see here vae model and the clip are going out and here I'm using clip model and also the VA uh here I used the uper scale for okay for my uh for doing the uper scale and after that I changed the size with up scale image by with about 0.54 so totally it will increase the size of my image about 1.3 or 1.4 X um and yes it's all for the case sampler I am using DPM Plus+ 2m and the Caris so I I think the result could be fine um and if you want also you can uh increase the escape by ratio to maximum the maximum number is one and I think the minimum number is 03 and yes actually this workflow can help me a lot in design process because sometimes I am looking for some idea for lighting and material so I'm ignoring the depths and I am working with line and normal but sometimes I need some creativity for interior design for example I have a wall that I don't know what should I do with that wall or um where can I place my f Furnitures or wall Arts uh so in this case I'm working with deps control net try that it's it's amazing okay the final image is ready and you can see that it's very very very sharp with high quality texture details and lighting um one another important thing in my case sampler is about the CFG value and I found that the CFG number four can work for me well but you can try and test different numbers so it's all about the workflow the main thing was load image from pass and enabling The Watcher using Pixel Perfect resolution for finding the best size for your image and using the get resolution from image for extracting the dimension of my pre-processor image and send them to my latent image using this IP adapter models and clip Vision adjusting the weight of the control uh IP adapter uh adjusting the string of the control net and yes it's all I hope you like that and also if you want you can share me the result of your work works and I'll be happy see you soon bye

Channel: Arch_AI_3D

Views: 3,361

Keywords: AI, stable diffusion, AI architecture, ComfyUI, ControlNet, architecture, artificialintelligence, Tutorial, ai news, ai revolution, ai uncovered, airbender, ar 3dsmax, ai render, ai render sketchup, ai render free, ai rendering architecture, ai render architecture free

Id: 7GLaiXreals

Length: 28min 2sec (1682 seconds)

Published: Mon Feb 12 2024