Stable Diffusion TripoSR Transform Image To 3D Object - Is it Good?

hello everyone so today I want to talk about the new 3D models from stable stability AI that is partnership with tripo AI and create a new models called tripo Sr now tripo Sr is a 3D model that able to generate from a flat 2D image and transform it into 360° angles objects like this so when you can have an image for example we have a stable diffusions image we can using that and transform it into a 3D object and eventually we can maybe later we can use newer stable diffusions three models or Stable videos diffusions newer versions model and create animations using these 3D objects and how can we do that and we don't know when those new stable diffusions 3 models release yet but right now at this moment this Trio Sr models are released and happening already on the internet so you can download these models in hugging face or you can use official website to run this if your Hardware is kind of not powerful enough to handle this and you can use hugging face if you have powerful Hardware computers now in here so far I see is that they are training triples for 5 days on 22 gpus notes which is using 8 a 140 g g gpus to train these models so I'm not sure if we require something high in like that to run this but then let's try in here because why not right we can down download these models so basically you go to this hugging phase and click to the files and versions here you can download this ckpt files extension now this is the models for tripo Sr models files now we will talk about how to installation that in local machine later but first of all let's see what is this all about so stability AI our partnership with tripo AI and creating this model this is an individual company that is specialized ized in creating 3D animations they also created this service that allow you to use that online to just generate 3D objects in the website and there you have the web platform that means use their website on here to create 3D objects and you can use API call if you have backend server program you want to integrate that with your software you can do that as well and they have some pricing if you want to take a look at that if you really need that for online processing 3D objects and that is up to you so let's see they have some examples here as you can click the bubbles on here and they have showing these objects and then let's see like this one you see the explosions from the volcanoes and then you see hamburgers that is 3D so actually their 3DS are it's not just flat pictures and then create multiple angles without any layering and texture and actually you see all this Little Dot and hopefully what they claim in here is true that all these little dots creating the texture of 3D objects so here's the demos you guys can take a look when you have time and that's basically showing all the 3DS how can they use this AI 3D generators to generate such 3D objects and put that into an animations videos like this so that is pretty cool things now let's go to how to install this now in github.com I have seen a new project called so Trio Senor these models obviously we can use in comy UI already right now today all we did is install this custom notes and install the model files from this hugging face link and then we can run this but before that we have to do the installations so let's go through here we have to clone the project or you can download the CP files on the top of this hugging GitHub page and then you have to go to the subfolder of this custom node notes and go through this install dependency in command prompts so basically that is really easy you just go to each of this just copy this one and go to command prompt and copy and paste so after that we have to download the models files of course we are seeing this one in hugging face then we can restart our comfy UI and it's done so this is a demo of their very basic models to generate 3D objects from an image so one image are able to generate 3D objects and you are able to view these objects 360 degrees and ups and downs view of this object so let's try and install this so the first thing we got to do is we have to download this one for me I remember all the procedures in the installations already it's very simple so just click this download button and then we can locate this models file in our comfy UI models folder and check checkpoints now in here I want to rename this models file because I just don't want this raw models ckpt right and later on we don't know what is that going on so let's put that into triple Sr and then I give it an underscore and there we go now we are done with downloading the models right now and then we have to install this so basically what you can do is go to the custom notes of the comfy UI so I have that in here and basically for Windows you can go to here just copy this line and go to Windows type CMD and you got this command prompts open you got the GitHub installed in your local machine and whatever and you can do that and I just yeah just few seconds and I download the whole project already and it's a very small size actually it's just few folders and few files here very small size and then you go into the subfolder of that which is this one not in here in command prom so we got to do CD and then the subfolders name and we have located in there then we got to start the install dependency so all the requirements txt files are list every modules and dependencies that we got to install so it's all automatically download in here it's basically very easy simple you don't have to take care of anything just copy and paste and press the enter button so we can try and restart our comfy UI yes so we got this installed and let's go play around with 3D models in comy UI all right so this is a very simple examples from the comy UI custom nodes project in GitHub and as you can see there is a loader a model loader and then a sampler for this 3D modeling and then you have to load an image as a reference in image to create that so let's try something like this and see so actually for remove backgrounds I have already did something experiments in my comfy UI already so you guys can check it out using comfy UI essential this is another custom node that you can do remove backgrounds because I saw some of their examples in here do have using the background remove so let's see if we need that or not so after I refresh this button in conf manager and then actually I have did the right installations and I have that in here as you can rightclick the menu and you can see this Trio Sr this is the one we have the model loader we have the sampler and viewer that is the three models custom notes for these 3D models so we can start play around with this one and let's say we have sampler and let's say we have viewers okay so the viewer is pretty nice so let's try with the was notes this one is because this one removed background and transparency I want to see if this can handle transparency of the tripos SR viewers so right here we have the sampler so actually let's go to put that in the middle because sampling and then the beginning of here we have the model loader so model loader we have to select the one that we just download of course that is the triples ckpt files so I got that one already downloaded in here so I have renamed the files actually you guys have to rename this file as well if you want to use that because you don't want just a raw models files name and list all over your model loader so this is how I manage that so models connect to the sampler models that is is very easy simple and then the reference image actually I want to do this reference image from here the one that I'm totally removed the backgrounds so here's the preview image of remove backgrounds and I want to see how that will looks and things like that oh actually we cannot use this one because we need the reference Mass so this custom nodes do not have that so let's go back to the comfy UI essential this one we got the mess so this is a very complete solutions for what we need to do and we can delete those because we don't need those anymore this is the preview of that so let's bring it up to here I want to preview just see what kind of thing it will be like so we have to use this remg section so using this one you can remove backgrounds in this section just a custom note to help you remove background that's it and then we got an image and then the output the mass of the remove background area so put that in the reference mass and then also the image put that in reference image as well then we are good to go and this one we don't need that and right here we have to mesh so mesh is actually the 3D multiple angles data type in here so we have to connect this one so threshold 25 and then the resolution 256 I would not set this one right now and try the default settings which is this one and then we will try that later on let's see so the models once again loader here remove sections and we are good to go let's try it so we got something need to download because the first time I loaded using these features of remove backgrounds okay so we got this one actually it's like a clay toys mixing with a character and it doesn't look really good on here so far I test this one is kind of not performing good for human characters especially you have a very detailed and well this is a real pictures and then it kind of awkward if you see this one and after all found out that this model could use on games like Minecraft they are going to use that for similar like m inecraft game to do 3D objects so that's why their input image in here for the demo is not going to be a real human characters or very detailed 3D objects and they are using some simple objects to do their demos here so yeah I'm going to try with some simple objects like a rabbit or a dog or a robot like this one and we can try it so by trying this I'm going to use a text to image workflow this is a very simple one don't be surprised how can I create that just click this default workflow button and you can have this very simple text to image workflow then I'm going to paste that in here and let's see so I'm not going to use the save image instead I will be connecting the image output to preview image for this one then we don't need the load image in here so this will be a text to 3D objects simple workflow so what we have by default income VII of course is the purple glass bottle that will be our by default that one and then we got to try something like a real Vis or the ref animate let's try this one so we got the glass bottle and then we will have the glass bottle on here and hopefully this simple object shape is going to be worked so yeah there you go we have at least a better quality than the human characters there you go this is glass bottles in 3D objects but some of that is kind of missing in here maybe this model have not fully trained it in this first versions of their models hopefully they have an update later and let's try something like let's see a rabbit yeah there will be like an easier shape for this kind of text to 3DS or image to 3DS model to create a rabbit like that yeah that is going to be okay for this kind of models 3D models so as you can see there's the back here actually is generated by Ai and then they're referencing the front of this and actually we got the middle and the back of here but obviously in this first sampling Generations it's not going to be very detailed you got eyeballs and things like that in the other end of the image but actually seeing this Model start to understanding what is the object shape and the things on the opposite side of this rabbit maybe let's try a robot okay let's try this robot okay we got uh a pretty complex shape for 3D models let's see if this overcome this challenge yeah it's not going to be looking good for this one but it is still able to generate the front and then the sign here is still able to project the shoulders of this robot and yeah it looks pretty okay it's not too detailed for the draft of 3D objects is okay and let's see like a plant let's see plans something like that as you know that in Minecraft those 3D objects like simple objects like that is always appearing in the games and this model are working for those kind of objects Generations is okay and if you zoom out put that into one of the game backgrounds and it will be looking like that so yeah there you go this is very cool features and new 3D models from stable diffusions and tripo Ai and if you cannot run comi locally you can try that out in their website it's tripo 3d. a so yeah take a look at that and I will see you guys in the next videos have a nice day see you

Channel: Future Thinker @Benji

Views: 3,924

Keywords: Tripo SR, Stable Stability AI, Tripo AI, 3D modeling, 2D to 3D conversion, 360-degree objects, animation, web platform, API integration, installation guide, pricing, online processing, 3D objects, lifelike, transform images, innovative models, stable diffusion, ai art, stable diffusion tutorial, install stable diffusion tutorial, stable diffusion prompts, stable diffusion ai

Id: YjwYLXzsPr8

Length: 15min 46sec (946 seconds)

Published: Sat Mar 09 2024