Convert single image to 3D model with AI in ComfyUI with CRM, for GPU & CPU

welcome to this new video today I want to have a quick look in the newly published CRM comi custom notes it's a quick way to generate textured 3D models from a single image so far it is not available in the comi manager you have to install it by hand so you need to download it on the guub page I will link it below in the description a little bit tricky to get it running but nothing which is is impossible one downside is you really needed big GPU in my tests I used nearly all of my vram I have a 490 GTX 490 and the process used almost 22 gab almost everything I have so be aware of that you can also use it with your CPU if you don't have a GPU with the size of 24 gab you can also try the CPU uh version of this workflow so for now we use the Cuda version and afterwards I will do the CPU version the results are quite interesting I will guide you through the workflow we have to separate the object from the background and then process it to different images which represent different sides of the object which will then later on be converted and combined to our 3D model in my tests it was not as good as I hoped for but this is only the beginning and I'm quite sure this technique will improve over the time and uh we will quickly have better results in the future so far I didn't compare this to stable 01 to3 which is also a workflow to generate 3D models from images I will make a tutorial on this as well so we can compare the results I think in regards of the vram use they are almost identical both need lots of vram I will guide you through the workflow from scratch we're building it up step by step let's start right away so let's start by loading the image I take this little fella first we need to uh remove the background I use a note from the comi essentials pack this note needs a ram BG session the model I would say stays like this now we need to pre-process the image and The Mask to make it accessible to the further processing so we need the CRM pre-processor for poser connect the mask and I think would be great to have uh preview mask preview and let's have a look at the image as well so the CRM pre-processor simply combines the image and the mask and prepares it for further processing for the poser node which we will use next with the CRM poser config I keep all those settings for now after the CRM poser config which gives uh the settings for the later generation of the model we need our different steps we need the CRM post sampler and we need the CCM sampler the CRM post sampler will create six different views of our image front side view top view bottom view back view and so on which will later then used to generate the 3D model the CCM sampler will generate our normal map which will be combined with the side views of the object for the CRM post sampler we need to choose the pixel diffusion model I will link the models in the description below and for the CCM sampler we need the CM diffusion package now we need to connect the config as well let's give this a preview so that we can see what is generated there for this as well this would be generated later on now as a next step we need the actual CRM modeler I use the Cuda only version for now but later on we will use the CPU version as well we need the actual model keep this as CRM pth I will link this in the description as well and let's see this is in the wrong part it needs to be the CCM sampler needs in to be in the coordinates and the post sampler needs to go to the poses and then we can use this CRM viewer preview as a safe note for our 3D model which is based on the 3js JavaScript game engine and I like how open source Technologies come together here and um use each other's know how to benefit okay let's have a look I think we keep all those settings for now and should be able to generate something so let's give it a try so there we go I said I'm not 100% happy with the results but uh it's it's something and it's uh really promising I think so here you can have a three D view of the object and you can import it for example to blender or something else also it is usable in game engines or something like this let's try another one here you can see the first generated mask then the image without background and then the combined image and mask for further processing and the see poster config and here you can see all the generated side views and here you can see all the generated normal maps for a view and after that everything gets combined to the model so let's create the last one so let's change everything for the CPU usage we need this trm model are exchanged so let's give this a try and here you go so same result so if you don't have a big enough GPU you can try as a CPU generation might take longer to generate I will link everything in the description also the paper to the CRM approach and I hope to see you soon goodbye

Channel: Neuron

Views: 2,470

Keywords:

Id: DXiiVpkeKgQ

Length: 9min 11sec (551 seconds)

Published: Sun Mar 17 2024