AI-Powered Sketch to Render: Transform Any Subject by ControlNet, ComfyUI Stable Diffusion Tutorial

Hello friends finally I made the work follow for converting any kind of drawing with any kind of driving style or subject to realistic renders okay let me show you here this is my workflow and I want to show a test here okay now I want to load the sketch of a car this is my sketch it has many unwanted lines this gauge is going to the control net preprocessor source and based on my experience now I want to select the scribble and connect that to the output so once that you are pressing the queue the processing will be started with different control net models okay you can see them here and after that you can compare the result and you can select the best choice really I think it's very useful because all the time I was testing that which control net or which pre-processors can provide an extract a good result for me but now it's very easy because only with loading one image I can see all the outputs from my different preprocessors and also all the old foods from my controller processors okay uh here you have to adjust your prompt for example now I have a prompt a car drift motion Loop and these are only normal prompts about best quality Masterpiece detailed and I just I added the sidewalk because I know that it should show the car in a street with some side work this was the reason [Music] if you replace the sketch okay then it can generate new designs and again I told you after that you can select the best result because based on your drawing Style each time different control net processors can provide a good result for you and it's dependent to you you can take the decision and here I am trying to show you the sample that how it is possible and with having all these nodes together I believe it's very easy to calculate the and generating the rendered image from your sketch now I want to use another car sketch for example this one and also here you can see many unwanted lines and totally the wheels are not complete and it's really a rawhand drawing cute prompt it's started and you can see the result in all of them you can see some dust under the wheels because in my prompt I wrote motion blur derived and some other key words and I think this is the reason that we can see these effects okay so now you can see and in less than a minute you have all the results and it's very useful okay the first one is scribble and in most of the time it can work very good and also lineart is not bad but to get a good result from Leonard you should have a high quality input and driving a high quality sketch is not an easy thing so this time let me use a high quality a sketch drawing and this time I want to use the invert you can after spending sometimes you can figure out that how you can take the decision for selecting the preprocessor output I will explain that but it will take some times and maybe in another video I will try to explain all the pre-processors but now let me start the processing okay now this is Leonard let me stop the following this is mlsd this is Kenny okay all of them are processed and this time you can see that the Leonard make a very very good result and it is very similar to our sketch everything is very very similar and the result is great but what's the problem every time that you want to get a very good render output you should use a very very good Hunter of ink it's the key yes and also here Leonard's anime also it's very good but for this sketch I am selecting the linards okay let me do the test with another sketch this time this sketch is very very tough and it's very easy everyone can throw in a sketch like this and this time I want to use the scribble pre-process over and for the prompt um foreign I want to use and it's not a car it's a truck and modern [Music] truck excuse me okay it's a modern truck and coupons okay this is soft Edge this is Linens this is Lino's anime this is called doctor this is mlsd okay so at first I'm going to review the scribble so I need the sketch here and I can compare the result I like that for the soft Edge I'll say it's good and the lines are follow the sketch for example here we don't have the wheel driving but in the generation it added that this is the Leonard and also the output of the linear is interesting this time animally North cannot work for me and also co-adopter and sketch because this is completely different style mlsds yeah it's not bad but it's completely different and also the Kenny um okay totally the form is similar but I don't like that and I prefer these options uh also let me search okay uh this time I want to change my prompt and instead of a modern track I want a futuristic track and cube root so you can see that the output is more creative go adapter sketch mlsd oh with this prompt I like the mlsd result it's very fun also the lineart anime is interesting oh really with the line art really notes futuristic track and soft Edge and scribble yes all of the more noise and let me do one more adjustment this time I want to decrease the strange of the control Nets I have connected all of them to this control layer and let's see what will be happen if we give more freedom to AI this time the AI has more freedom to change and modify the design and after that we can compare the results this time the candy is better mlsd also is interesting Leonard anime in art this time with some more details and some more consistency in the form of the body it had also additional parts soft Edge and this is the scribble okay so you see that it's possible also to convert this type of dropping so please let me do another test with a different type for example this one and based on my experience again this type of professional sketch and drawing um will be something nice if I use the line art or the Leonard animal in Earth pre-processor I am selecting this space on my experience and Q prompt oh I have to change my prompt because it is trying to make a truck from this sketch so let me cancel the processing and also here you can see the output of the animal in art yes we use the Leonard this time so at first let me adjust the string to one and again change the prompt so instead of the futuristic track let me paste that here I want to keep that card drift motion blur paste crumbs here and coupons so this time the AI trying to generate a car not a truck and with more attention to the line droppings because I changed this range and adjusted that to the one you can see that every control net will take about 10 or yes about 10 seconds only for processing and now let's meet check the results oh wow I like all of them not this one the core doctor it's completely different but I like that okay uh this time let me change my prompt and I want to delete the drift and also the Musha blur and I want to give some freedom to AI to change my a sketch so I am trying to decrease this amount so this time you can see there is not any motion blur or dust or fog inside the wheels and also it has some freedom to change the car shape Leonards soft Edge and Leonard's anime are perfect I like them okay let me change the category and this time I want to test a building okay okay and also I have to adjust the resolution because this time my sketch is a score a separation and this prompt is good modern house exterior Landscaping sidewalk and lighting so here I'm going to add them and Q David was about seven you can see it's going to calculate with with each different [Music] controller processor [Music] some of them are useful and some of them it's not possible to use them but I am trying to show all of them to you okay and let me take the sketch here yes you can see the handrails window door and also we have some lighting on the building we have some side work we have the landscape also here we have those elements with Leonard we have them Leonard Animal and generate something completely different sketch all seasoned usable also I like the mlsd result mm-hmm it's very interesting okay so uh this time at first let me turn back the strange value to one and let me select one outer I have used the lineart pre-processor [Music] but I prefer to test the because I think maybe the animal in art is more clear to me so I have connect the animal inert output to here okay and it's on Cube okay I like the mlsd yes I like that demo list is working um every time can provide good results with straight lines and for example when you have a building in modern style in most of the times mlsd can help you also the Leonard it's very very good because I can see very good consistency I have the door I have this smaller step I have these two window I have this big window I have hand red I have this wall I have this window I have the garage door yes it's the lean out the output of the Leonard with this parameter is perfect soft Edge it's completely indifferent style and also the scribble also the scribble is not bad but it's changed this window to some kind of door but but I like all of them most of them and yes the environment of the MLS this time is very good and it's very creative for example if you don't have time to make environment for your rendering or other things with this technique easily you can create some good presentation for your drawing okay okay so let me change the drawing and again cube root this time I want to generate for generate rendering for different building okay this time I like the soft the I like the scribble again I like the Leonard Leonard's anime phone mlsd also it's very nice in environment the also the lighting and atmosphere okay uh and this time I want to do a test with another drawing and here again we have lots of unwanted lines and when I have many unwanted lines in my sketch I prefer to use the output of the scribble you can test all of them on your computer because already I did that and I spent many many days so now it's very easier for me to take the decision and based on your experience you can select the best one so for example for arches for architectural drawings and a sketch to render you can see that for example using the co-adapter sketch or linear anime cannot make a good result for you the candy is not very bad mlsd again I like the environment and I like the lighting it's very nice no no Leonards perfect p o boom boom boom it's perfect soft Edge hmm not bad scribble also this it's normal because the consistency of the building is not very good and for example here I have only two window but this is a big one window it's not bad but the lean art was perfect the Leonard was perfect okay this time I want to make a higher quality okay so here I have also another um work follow or notes to generate the high quality image and I am trying to move these high-risk fix here and I will connect the conditioning Plus for conditioning late into late and and image to image start queue oh also I have to activate my sampler press Q in this node I am loading realistic Vision engine and for two time I am scaling up the render and with this workflow it is trying to add some more detail this was my old route in low resolution and this is my output in high resolution that is in processing and after about one minute about that I can see the final image in high quality good morning okay I have that so in compared to this version this version is very blurry and you can't detect them and see many details but this version is more clear and also more realistic yes okay so this is the first time that I have created a complete a really complete work follow for converting any kind of drawing or any kind of a sketch to render and this is the reason that I am trying to record a long video for you and too many different tests okay uh for the building it's enough and this one I want to test a portrait okay I want to use the invert image because the line driving is very clear [Music] and I have to fix my prompt woman portraits raw photography and tips of field tips of field also is very useful prompt because um can make some Blu-ray in the background and most of the time it is very useful okay Q prompt and start okay this is Leonard and again the lineart is the best but also the Leonard anime can provide a good output for reverse okay [Music] okay we have the results and let's compare them not bad but the hairs are not similar uh again not bad but Linux is perfect Leonard is perfect yes it's perfect Leonard Animal also I like that hairstyle sketch no list you know Candy note okay I like I think this time the Leonards and the Leonard Animal are fighting and the war is between them and outer processors cannot create a good result for us okay okay so let me change the drawing style to something like this and start the queue okay Leonard it's very good okay soft Edge it's not similar schedule it's not similar this timely North anime no no no no no no okay and please let me review the result okay I used the output of the inverse image so the AI exactly use this image but but I believe if I use the output of linear processor Phil processor maybe it will be better because the lines are more clear here okay let's start Q um this time the Leonard's anime is better but I think it's tilt The Head and the perspective is completely different but totally the form of the face is very similar um what the Leonard it's very very bitter yes it's very better also the soft Edge is better now but still it's not something similar and also about the scribble okay so let's move and do another test this time I want to use this drawing and it's a girl and I have to change my prompt I want to keep the depth soft field and just this time we're going to do it here and Q prompt foreign okay you can see that it's trying to fill the borders [Music] Leonard this is Leonard's anime okay uh I believe that I have to correct my prompt and delete the sidewalk because it is trying to place that girl in the street and this time I want to add a Garden and flowers and again start Q prompt okay I think soft Edge Leonard and Leonard's anime are the best for this kind of drawing okay and also let me do a test with this one it's a dragon and just I have to write a Rock on and coupon scribble soft Edge anime go without your sketch mlsd and chain okay this it did a very good job I like that soft Edge is very similar yes it's very very similar to the sketch highlight that Leonard it's better than soft Edge I like that lineart anime it's very very good and it has a very great consistency no no [Music] again the Leonard and lenault anime are the best okay so I I believe that now easily you see that it's possible yes it's possible to to convert any kind of hand-driving SK scribble or line to realistic rendering and it's very very useful it's very useful and also still I have some more customization method and also I want to tell you about that [Music] okay also I have one other type of drawback where is that okay please I have to find that oh okay I found it it's here yes this is a sketch and I want to convert that to a render and in my prompt as I want to write a building it's all and for the preprocessor I prefer to use the scribble Q product so you can see that it is trying to convert this drawing to something similar to a building some of them are useful and some of them are not useful this is an optional with soft Edge it's another option with Leonard and the holders are not useful and let me please check the drawings and now I use the processing with a scribble but also we have the lineart and it's not bad to test to do a test with animal in art Q so easily you can see when you are changing the line type for example you are changing your preprocessor after that the result will be completely different and this is the main reason that I created this work photo because for example in less than half hour you want to create the best option and then you can see all of these inputs and outputs together after that we can take a do you think you can take the best decision and also I want to do a test with pdnet because here you can see that it makes some nice output out slower outline for me Leonard cannot work also the Leonard anime sketch and the results of the emphasis interesting was it some kind of converting okay but uh I think the best model is a scribble here for processing this kind of drawings and yes I think this is the best solution and this time I want to change my preprocessor source and if you don't want to see the outer processors just you can select them and press Ctrl M on your keyboard so they will be mute so one more time I want to do a test okay and now I want to focus on control and Ctrl M Ctrl m only on soft Edge and scribble okay I told you sometimes it's good to give some freedom to AI with reducing test range of the control net so let me do that and some more freedom oh it's completely lost the building position [Music] and also with soft Edge okay about the prompt uh let me add these parameters Landscaping site work lighting um okay I'm sure that I have to increase the weight so with that prompt it's trying to imagine these lines as a building nearest Street including the sidewalk and other parameters and I can see that the soft Edge also cannot work for me so I'm going to mute that and only I'm focusing on the scribble okay uh here I control the seats with the fixed number but now I have to change that to a random number to get some more variation yes really just trying to use these lines I like this option I think I have to go with some uh incremental number and after that it will be easy to find the seed number because every time it is going to generate a building and after that the seat number will be changed and if I want to make a bigger scale I have to catch the seat number okay in the prompt I want to change the width of the sidewalk and also the name display lighting to [Music] to get some more options we can go here and increase the patch size so I can get four different version this one is also is very nice okay now I want to show you another option for defining the style of the building okay you know that we can use the prompts but also we have another way we can use an image as a reference okay here I have to load an image and after that in this workflow I have to connect the this conditioning to this part excuse me and Q prompt so it is trying to analyze this image and adopt the generation with the site that I defined okay for example this time you can see it's a Chinese building in that style and every time that I try to generate an option it will be in same style as my input and if I change the building style for example to something like this it can understand and after that it will try to create a building with those material to that style and something similar and if it is not successful again try with another cue and again if you think it is not successful change your prompt okay only here is the building and finally I am suggesting to decrease the vid so in that case it can change some lines for example it can ignore this line and it can make some outer dropping with similar to this sketch it's trying to place that on some other part because if you follow the perspective lines you can see that I am I should write in a building sidewatch and decrease the read so I think it's trying this time to place a building on the ground no cancel that again Q prompt okay and I think it needs some more feet about processor [Music] let me check also I have to check the manga to anime and one more time check that with scribble outfit okay it's better now it's better a little and now I can see that it is trying to capture this style but with these lines uh a little decrease and this time I want to change the building Style to a image like this it's a factory mm-hmm and you can see that it's trying to rebuild that factorial Style with this sketch okay and let's change this style this image hmm to us not successful and this time I need a different style it was a great reference uh let's redo some more tests for example this time with this style oh it's a nice building so with this seat let's active the upper scale um very good it is the problem ah I forgot that I have to connect the conditioning from here the latent and the image Q [Music] okay so you can see it is possible to convert this sketch to a building like this with this architectural style and this is the reason that I told you this is a very very powerful workflow and it can converse anything anything okay until the next video see you bye

Channel: Arch_AI_3D

Views: 6,048

Keywords: AI, stable diffusion, AI architecture, ComfyUI, ControlNet, architecture, artificialintelligence, Tutorial

Id: 9530DEWS3hE

Length: 80min 10sec (4810 seconds)

Published: Sun Jun 25 2023