GPUs for Stable Diffusion in the Age of SDXL - Best Value Graphics Cards for Inferencing

hey guys so in this video we'll be talking about graphics cards for stable diffusion and has been a couple of really significant developments one involving AMD one involving the RTX cards from Nvidia [Music] so we start off with the RTX transforming video the RTX 4060 TI has seen a significant price reduction of about 10 percent this is within about a month of it launching in July and that is significant because I cannot recall any situation where an RTX card dropped in price within a month of launch so the price came in at 499 we're seeing it at 449 from Zotac and it's not just Zotac we can also see one here from Asus was seeing one from Asus the Asus dual GeForce for RTX 4060 TI 16 gigabyte Advanced Edition this one comes in at 449 dollars as well so a significant price reduction very very soon after the launch which was at the price of 499 dollars and quite interestingly we also have one coming in from PNY which is coming in at 430 dollars now we'll take a closer look at that one this one from PNY PNY is one of the middle of the road companies for graphics cards from Nvidia they are pretty decent when it comes to warranties when it comes to customer services and with this one it started off at around 436 dollars when it launched a few days ago and I would say with this one with it selling directly from Amazon I think it looks like a pretty decent purchase I wouldn't have any problems purchasing this one from PNY so if you're looking for value at the moment I would say the RTX 4060 TI 16 gigabyte is good value and I will link to this one in the description in case you want to buy this one in the United Kingdom there have been price reductions as well so the price for the 4060 TI started off at 479 pounds and for this particular one it's the only one that I saw a price reduction for it's gone down 50 pounds to 429.99 I think the price reductions make these really attractive I will link to this one again in case you want to purchase it and I'm going to be talking a little bit about the performance of the uh of the different versions of stable diffusion invoke AI automatic 1111 comfy UI with an 8 gigabyte card and maybe giving you some ideas as to whether you want to try to upgrade if you're going to be using sdxl the new version of stable diffusion the one that has these big six gigabyte files now if you come and look at other locations in the United Kingdom we're looking here at box technology the prices are over 500 still so we really just seen one or two graphics cards drop in price this one uh this list here I'll link to this list here this is a pretty decent company they're selling it with OverWatch this is nvidia's way of trying to push these out of the out of the door I don't think many gamers that are purchasing the 16 gigabyte 4060 TI are gonna be that much interested in OverWatch I could be wrong but I think if you're purchasing a 16 gig uh graphics cards you probably want to play AAA titles with really really good graphics but hopefully the prices will droop down a little bit they are a little bit higher here than they are on Amazon AMD launched a couple of new cards since the last time that we spoke and these are the 7800 XT and the 7700 XT the 7700 is a pretty decent looking card at 54 compute units and so the 7800 also looks like a pretty decent uh card especially with 16 gigabytes of vram unfortunately I've not seen these actually selling in the shop so once I've been able to do quite a bit of price research on the Nvidia cards I haven't seen enough of these in the shops to be able to say really what they what the actual price you're going to be paying for them but obviously it does feed into the idea that the fact that they have just launched does feed into the idea that perhaps the price reduction that Nvidia have just announced might be related to the arrival of these new cars now another really important development is related to the rocm platform this is the platform that's used for running stable diffusion in Linux it seems that there is a development that has got quite a lot of the AMD Fanboys excited it involves the arrival of rocm to the consumer graphics cards so rrcm has generally speaking been associated with the workstations the professional cards which cost quite a lot more than the Consumer cards and uh what veronics are saying here they have a quote from bamsi bupana who is a representative of uh AMD and she says we plan to expand our CM support from the current currently supported AMD our DNA 2 Workstation gpus to select rdna 3 workstation and consumer gpus and is that consumer GPU part that has got a lot of people excited because obviously this means the ability to run some of the code that you need in artificial intelligence on these CPUs and here on for ronix they do compare rocm say about Nvidia is NVIDIA meanwhile continue supporting Cuda across their entire span of consumer and professional products going back Generations from launch day and what I would say is look Cuda which came out more than a decade ago 15 years ago even five years after it came out there was some reluctance to adopt to adopt it by some of the large software houses it is now the Dom one of the dominant platforms and it's one of the key reasons why Nvidia is so powerful in artificial intelligence so it takes a long time for momentum to build up Behind These platforms and there's actually a page here which discusses high performance Computing it is the sort of workstation professional level area of AMD and they discuss HPC rocm and there's a section here on hip where which is called heterogeneous Computing interface for portability now they mention here that hip like Cuda is a dialect of CC supporting templates classes Lambda and other CC C plus plus contract constructs now this is not an area where I'm very very strong so if you guys want to jump in with some comments about what all of this actually amounts to feel free to do that in the comments one I will just say is that the community seems to be excited about the idea that rocm could be used to emulate in video Cuda so enabling code compilation for either AMD or Nvidia cuder environments there may be a degree of compatibility between rocm and Nvidia Cuda does depend to some extent on whether developers are willing to work with rocm in this particular way for those of you who are really interested in this I will link to the originating article which has really sparked off this this discussion here what I would say is I'm not entirely as enthusiastic as some of the AMD guys I know from past experience that can take a bit of time before for any platform really gains traction and begins to be used in a way that significantly improves workflows and whilst I was looking for a little bit of color on this whole situation I came across this piece a bunch of people discussing uh the technology one guy here this is the AMD forums it's not an environment where you would expect people to be hostile to AMD and what this individual band balance says is if we compare it to Cuda even on Linux there are numerous exam there are numerous issues graphics cards in the field of artificial intelligence are lacking and leave much to be desired if you're only considering changing your code due to vram let me tell you that for instance in AI work environments like stable diffusion and Nvidia graphics card with half the ram of an AMD one performs much better and has better workload management than one with double the Ram from AMD so hopefully that adds a little bit of perspective a little bit of color to this whole discussion feel free to jump in the comments as well with your own ideas about this if you have any opinions I will have a brief discussion of the situation with company UI invoke Ai and also with automatic 1111 on an 8 gigabyte system and if I'm just going to summarize the findings I would say if you are intending to run invoke AI on a system similar to mine or if I was running a invoke AI uh the the new version with sdxl I'll be looking to upgrade my system I'll be looking to run the system with 16 gigabytes of vram the Nvidia RTX 4060 TI 16 gigabyte version and the same would apply also with automatic 1111 with Nvidia with the config UI I could get by with an 8 gigabyte graphics card for quite some time because it does seem to be very well optimized for running with stable diffusion and lower amounts of memory so I I have a slightly longer piece which I will play towards the end and maybe that'll give you a little bit more insight into these conclusions some links in the description to these graphics cards that we've discussed and hopefully I'll see you guys in the next video so I tested the performance of an 8 gigabyte graphics card out RTX graphics card with invoke AI 3.1 with the latest automatic 11 11 that's 1.6 and also with comfy UI and what I found with SDS Excel was that I was not able to satisfactorily complete a render in invoke AI on this machine with 8 gigabytes it just would not perform so I would need a 16 gigabyte graphics card to perform satisfactorily with invoke AI I was able to complete a 1024 by 1024 with the stable diffusion 1.5 meaning that the problem was the the amount of ram the amount of vram because with that stable diffusion 1.5 you're obviously dealing with you're dealing with a model for about two gigabytes with stable diffusion XL you're dealing with six gigabytes for one you dealing with six gigabytes for the refiner so it's quite a lot more memory that you need to actually complete the process satisfactorily and comfort UI complex images like this which require a lot of processing with multiple models and also using Laura files these take about a minute or so to complete whilst a standard 1024 by 1024 takes about half a minute 20 minutes 20 seconds to 30 seconds and that's the fastest that I've got on this machine and within automatic 11 11 I was able with a lot of optimizations to get the render times for 1024 by 1024 down to around one minute 50 seconds to one minute and that required a huge amount of optimization but I was able to do that whereas with invoke AI I was not able in spite of a lot of attempts to optimize things I was not able to get that performance Improvement without the optimization it was taking two two times as long as that maybe sometimes four times as long as that too as long as that to complete a render so so far I would say probably with comfy UI I feel eight gigabytes is comfortable with sdxl it becomes a little bit uncomfortable with automatic 11 11 but we may see more uh improvements over time and within VOC AI there's definitely a lot of work that I would need to do to get it running in any kind of reasonable way with sdxl now I did try the new graph menu inside of invoke AI but that didn't seem to support sdxl as yet foreign [Music]

Channel: Pixovert

Views: 4,457

Keywords: SDXL, Stable Diffusion SDXL GPUs, Stable Diffusion SDXL GRAPHICS CARDS, Stable Diffusion SDXL RTX NVIDIA, Stable Diffusion SDXL RTX 4060 Ti 16 GB, invokeai GPU SDXL, Automatic1111 GPU SDXL, ComfyUI SDXL GPU

Id: zZm3E0Zx6Oo

Length: 13min 38sec (818 seconds)

Published: Mon Sep 04 2023