ComfyUI Architectural design plan workflow #comfyui #controlnet #ipadapter

hey today we will see how we can take a raw design plan of a certain space and by using comfy and stable diffusion we can create endless different variations and different designs that combine different colors or textures to this very specific space we will maintain the true proportions and measurements that exist in the original plan our workflow includes a positive prompt and a negative one interior design photo colorful you can move it if you don't want to muck color in your desyncs I have Laura for LCM just to make it faster K sampler that supports Laura and that's the base of our workflow note that in order for you to be able to upload any image in any proportion in our example the image is more panoramic something like 16x 9 so in order not to mess with the dimensions I added this node get image size you can connect your image on one side and get the exact height and width of the image you uploaded just connect it to the latent image right now I'm creating two variations for each CU we're doing and all of this goes through two control Nets located here let's go over them for a moment I will bypass this IP adapter group and as a start we will also bypass the line art which is the second control and I will stay for a moment with the control net called depth anything which is relatively new and came from the Tik Tock team I also use the aiio node which allows me to use all the pre-processors I have on the computer regarding the control net very easy to work with and let's do q and we need to get an image that is based on the depth of our plan if you look at this example of depth anything I will link to this demo you can upload any image you want and see how this model translates this image to a depth map which then based on this map we create the next images at the end of the video I will also show you how to install this model and use it in comfy and so you can start work with it I want you to see what we got here first of all the colors and shapes are very interesting but this is not what we wanted to get the textures we have here on the wall and all kinds of patterns that it seems a little unclear where they came from and that's what I want to show you if we look here I just created a preview image node that comes straight out of the pre-processor so we see th pre-processed image and how the depth anything translates our image when I zoom in I hope you can see it through the video there are Stripes waves like all over the image sta interact with the texture of the wall of the table and everything within the image and that's actually what affects the final generation it's probably related to how the image was saved and its quality so pay attention to that for now I will try to get rid of it by lowering the control net strength to of the depth anything let's see how much this helps us to with all these patterns as you can see once we have lowered the strength of the control net we are getting much clearer image I think that the lines we have on the wall are still related to control net so let's lower the strength a little bit more and also reactivate the second control net that uses line art here I also use the same node to load the line art pre-processor and simply use the line art model and let's put the line art on one just to see what it does and it seems to help us get rid rid of these textures or patterns that we saw earlier and we are getting something much more logical and much closer to the plan another thing I want you to notice is that because we are working with a relatively strong line art now we get some kind of separation from the line art look at all the black lines in the chairs and in the Contour of each straight line it is similar to The Contours we have here in the original image so to fix this I also decreased the strengths of the line art and I believe that now we will be able to solve the problem that the depth anything created for us and also the problem that the line art created for us we can see that now we are getting much cleaner image we don't have these patterns on the wall we don't have the black lines coming from the line art and now we can start to play with the colors and textures and for that we can do few things things one is to change the seed number and click on the Q button and we will get slightly different results in the final images another thing we have here is the IP adapter the IP adapter allows us to load an image and get a result similar to that image meanwhile we can see the result that the other seed number gave us a different and interesting result as you can see and we still maintain the the general design which is excellent now that we have activated the IP adapter I just want to show you that I am working with an IP adapter plus model which is currently the strongest in terms of the impact of the IP adapter on the model and we can play with the weight and the noise I also use Channel penalty which is another parameter that helps me give a little more strength to the IP adapter we need this combination because we are working with two control Nets that greatly affect our final image and this is the result we get with the use of the IP adapter as you can see the color is the same color that comes from this image that we load here we can play with the weights or use another image and get a combination of new colors and textures inside our interior design now let's see how to install the anything we need to go to this page on hugging face I will include the link in the description of the video and download the diffusion pytorch model after downloading the model you need to find your comfy UI folder the models folder and the control net folder here you should paste the model in my case I named it because it's easier for me to find it for the line art model go to this hugging faces page and here you can see the great designs of wi Studio the studio that provide for us the interior design plan you can find all the links in the description so this is one way to take an interior design plan and create all kinds of variations textures and colors while we are keeping the original dimensions and the details that come with the original plan and all of this without talking about the possibility that in painting opens for us hope you learned and we will meet in the next lessons you are more than welcome to ask questions comment and like and most importantly have fun bye

Channel: PixelEasel

Views: 6,445

Keywords:

Id: -YztmpHp8UA

Length: 7min 44sec (464 seconds)

Published: Mon Feb 19 2024