Stable Diffusion OOTDiffusion For eCommerce In ComfyUI ( Installation Tutorial Guide)

hello everyone so in the last video I had promised to create a walkthrough on all diffusion AI model for changing clothes how to install it and how to run it in comfy UI this is the hugging face page now they have this demo page on their hugging space and you can try it out using their demo with preset model image and clothing image you can see different people wearing different styles of clothing here but but basically the image data they've used has been fine-tuned specifically for out diffusion because I found that after testing it some poses and angles of the AI can't generate good clothing demo outputs so I just want to test out some angles and poses which you may need to avoid when using oot diffusion to generate your e-commerce clothing demo images or if you want to use the IP adapter workflow that I covered previously you can try that out too the link is in the description below you can watch that video as well and try running things both ways testing which one works best for your use case you know sometimes there's a difference in which poses or model angles don't work as well in oot diffusion by the way here is one of the AI friend in Discord who is really doing e-commerce business and use the workflow created in the last video for his product images it looks amazing glad this really help and make things in production anyhow back to this tutorial like this one we have a tight t-shirt and a black T-shirt in the picture the model doesn't do really well with those as I can see the lower part of the t-shirt is tucked into the pants and this isn't the style of how a Loosely fitted t-shirt would drape right if you're wearing a fitted style t-shirt like that then sure it's okay to tuck it into the pants but there are some pros and cons with this or to Fusion so I'll demonstrate some bad examples especially ones to avoid when running this AI model the first thing you need to do is come to this OT diffusion models GitHub page it's called com fuy OT diffusion scroll down here and you'll see the instructions in both English and Chinese the workflow is pretty straightforward you load the UT diffusion model from GitHub as a custom node create another custom node called the OT diffusion gener node and then connect your model image and clothing image it will generate the output for you so let's go through the setup instructions here we're going to first create a cond virtual environment and install everything step by step but before that you have to scroll down to this part you need to install the Nvidia CA Library you also have to go to the visual studio installer and install the plug-in for your windows SDK also you need to to download the Nvidia toolkit 12.4 from here click on Windows if you're on Windows and then download the corresponding version for your PC back to UT Fusion GitHub scroll down a bit on this page and you'll see the visual studio installer section the author has provided screenshots showing which tools you need to install in Visual Studio basically that's the cuspus component and Net Framework component and and if you scroll down further you can see there's a Windows SDK listed let's zoom in to this picture and you can see we need the C++ component and the windows SDK if you have Windows 11 of course you'll need to install the windows 11 SDK instead and for my OS I am selecting Windows 10 SDK come back up to the GitHub page instructions there are a lot of steps that don't clearly explain what to do but if you have experience with a PC command prompt or a technical background in computers you'll probably be able to get through this without too many problems the last thing you need to do to boot up comy UI is to open a terminal or CMD to get the command prompt window don't use Powershell we need to open the regular command prompt window and then you'll need to find that Visual Studio path open up this VC vr64 4 dobat file from the command prompt and this will set up the necessary environment for o diffusion run and com youi then you have to make sure you went through all the previous setup installations properly and you can activate the cond virtual environment for OT diffusion the next step is to Kickstart the python main.py file and this will boot up your comfy UI but one thing I found is that you cannot use the portable versions of comfy UI you have to use the GitHub clone version instead that means you need to clone the entire comfy UI GitHub project into a folder without using the portable version because you have to create a cond virtual environment within your computer setup this is related to the comy UI cond environment for running the Cuda toolkit and the Nvidia Library which you need to get everything working in comy UI properly so let's get started with the first step in the visual studio 20 22 version I have the Community Edition here scroll down to the net and multiple platform development section now you can see the windows SDK options with Windows 11 Windows 10 and different dated versions of each SDK select the latest one available since my windows is Windows 10 I'll select the latest Windows 10 SDK and it will start installing after installing the Windows SDK you come to the command prompt like what the author instructed and create a cond environment once we have the cond environment activated you have to go to your comy UI folder and install the dependencies for comy UI here I'm not using the portable version of comy UI but rather the folder from the cloned GitHub project again I have also cloned the oot diffusion folder the GitHub project into my custom nodes folder lastly you have to install the necessary Library files for your comy UI setup I did run into one error with the Cuda Library so I had to uninstall the pytorch 2.2 version and then reinstall it using the command prompt mentioned in the official comy UI GitHub project page this is one of the most common troubleshooting steps using pip to uninstall and reinstall a different version of the pytorch library that process will take a little while after that you can try booting up comy UI using the python main.py command and it should start up properly let me show you the history of the commands in this prompt it installed everything at this stage it looks like everything is okay with no errors so I successfully installed pytorch and then I can Kickstart comy UI and it showed me that my graphics card is detected and everything started up fine but before that I forgot to install the requirements.txt all those dependency components so I had to go back and install those dependencies again then I started the comfy UI main python file and it successfully kicked off you can check in your command prompt if the O diffusion custom nodes are able to load or if there's an error message if there's no error that means you've successfully installed oot diffusion in your comfy UI in the comy UI page you can rightclick and see the oot diffusion items in the menu let's start a fresh workflow using these oot diffusion noes you can get the workflow from the oot diffusion GitHub page it's a very simple workflow you don't have to ask me where to find it or how to get it just go to this project page and there's a link to download the workflow Json file download that and drag the Json into your comfy UI interface the first time it runs it will download all the complex subfolders and files for OT diffusion into your local comy UI Drive there's a lot to download initially so wait a while and you'll see your image results the first image output looks a bit awkward from certain angles let me change to add different example image like this dancer wearing a green t-shirt but as you can see the output isn't exactly the same green t-shirt from the reference I can generate it again but it's still not the same so there's something you have to be aware of when using a diffusion you need to use a very similar model pose and angle to the reference clothing image you want it to generate for example I use this cheerleader demo image and it can generate that outfit naturally but look at the bottom of the dress it looks torn or broken in pieces that sometimes happens with this diffusion model because it's automatically masking and editing the model's image and look at this one um the model is at a 45° side angle like this it cannot generate and change the t-shirt on the models post perfectly for example with this model standing in a side profile view like this let's try one more dress example so you can understand what's happening basically if you use a model image that isn't facing straight forward to the camera like this one with a side view angle it will generate some weird artifacts when trying to edit and apply the clothing image it cannot generate and fit the clothing perfectly onto the body pose this next example looks better because the model is standing straight on facing forward this type of frontal pose works well for example this colorful dress output looks good because it has a similar basic shape to the original clothing on the AI model you can choose upper body lower body or a dress option for example when you load the diffusion model from The Hub you can also choose the full body option there are different choices you can test with various clothing styles for instance here I have a t-shirt but the original model image is wearing a dress you'll see it looks a bit awkward this dog t-shirt isn't going to generate perfectly on this model's post because the woman is wearing a dress and it cannot Mass the image flawlessly to apply the T-shirt some parts look odd especially around the hips and lower body area that part won't show the T-shirt fitting perfectly over the dress outline however if you use another model like this one wearing a tank top and jeans the AI can better mask the upper body and apply the dog t-shirt more naturally and with better performance so yeah that highlights some pros and cons of this diffusion model like with this image this dog t-shirt and a few of these model images are using examples from the diffusion models demo as you can see you're not going to get a perfect output on the first attempt you have to generate it a few times to get a good t-shirt fit on the model even then some details may not blend seamlessly with the T-shirt graphic let's try this same image and t-shirt using the IP adapter workflow that I covered before one thing I like to do is manually mask the subject better than just relying on the diffusion models automatic masking let me fast forward through the loading process there we go we have our result in the first sampling group we have four images as I mentioned before I like to generate four at a time to see which one performs best and I'll pick that output you can see it Blends in nicely using IP adapter to apply the clothing looks quite natural for the T-shirt shape and how it conforms to the body the second group starts adding more detail you can can see the dog graphic on the t-shirt is much clearer now a few samples in this set are doing really well with the AI image generation let's try a dress example next like this blue dress one good thing about IP adapter is that I can manually mask the area I want in this case while some examples here don't have the dress fitting perfectly I got a good one like this where the dress conforms nicely to the model's posst you can evaluate which method you prefer to use for your e-commerce product modeling photo editing or whatever your use case may be there's not always one definitive way and even using the oot diffusion model doesn't always generate perfect results most of the demo images they show feature very straightforward forward- facing poses it can't handle more Dynamic angles like a seated pose or side view as well so yeah you'll have to evaluate beforehand which technique performs best for your specific clothing items and intended output that's it for today's video I'll see you in the next one hey there future f Benji with a screen of glow crafting stories in the digital under toe pixels dance at the flick of your command bringing to life the vision so Grand future think you're Benji teach us the way in a world of zeros how to brilliantly play animations that flow like dreams in the night under the neon lights everything's bright

Channel: Future Thinker @Benji

Views: 4,646

Keywords: OOTDiffusion AI model, clothing change, e-commerce clothing demo, IP Adapter workflow, Comfy UI, GitHub, installation, CUDA library, Visual Studio, Conda virtual environment, clothing generation, poses and angles, clothing artifacts, fashion, content creation, e-commerce entrepreneurship, Stable Diffusion OOTDiffusion, OOTDiffusion For eCommerce

Id: 6cglM8ICha8

Length: 14min 38sec (878 seconds)

Published: Sun Apr 14 2024