How to Choose an NVIDIA GPU for Deep Learning in 2023: Ada, Ampere, GeForce, NVIDIA RTX Compared

What GPU should you buy in 2023 now for this video I am going to make two assumptions the first is that you will be buying an Nvidia GPU the reason for this is just their dominance in terms of support by the major machine learning libraries the second is just their proliferation across the major Cloud platforms the next assumption that I'm making is that this GPU is going into a machine that you are building or installing basically is the GPU power in the same room as you if we're talking servers then that's a completely different proposition now in this video we're going to talk about things that are my opinions for what I like to use in deep learning and machine learning you may have different opinions and the discussion on these videos I learn a tremendous amount so please let me know in the comments what you think of anything that I say here and if this is helpful to you please give me a like maybe subscribe to the channel certainly helps a lot all right let's get into it the first thing that you need to think about is since we've eliminated the server line do you want to go for the Nvidia RTX or the Nvidia GeForce line the Nvidia RTX is more expensive this is the professional line of gpus this was called Quadro and days gone by and there's still I believe one Quadra that's in the active lineup here if you're going to get this more expensive line of GPU I can really think of only two cases that you would want to consider it over the GeForce which is really more the focus of this video if you'd like me to do a video entirely on Nvidia RTX let me know in the comments I would be glad to look at that I have an Nvidia RTX a6000 in that machine and really really like it the two reasons that you might want to consider the professional line in my opinion one is you need a lot of RAM on your individual GPU and secondly that you're going to perhaps get more than one of them so let's look at those One By One The Ram you can get on the Nvidia a6000 which is the Ampere version or the Nvidia a6000 Ada Edition which is not yet out as of the recording of this but will be coming soon those both come in 48 gigabytes of RAM the best you can do in GeForce is half that so if you want that much RAM on a GPU and have that flexibility it's it's worth getting and that's kind of the category that I fall into for having the a6000. the second thing to consider is do you want two of them it's just their compactness and their ability to stack several of them together the a6000 that I have in that machine just takes two slots so if you're going to put two of them together that's four whereas if you're putting two 30 90 or the the 40 equivalent you're you're going to be taking six because it's three by three so they're really designed to be denser and that's what you start to get when you go to the professional series and then like Hopper the server class so let's jump right into GeForce we'll make use of this chart a lot in this video here I have the Ampere 30 series at the top this is the older one and the newer Ada Lovelace 40 series we are really at a turning point like we usually are between two technologies you have the 30 series which is ampere now being replaced by the 40 series which is Ada we're at a point where both of these are worth considering depending on price point if you just want maximum maximum performance and you don't care about price point by all means get a 40 90 or get an a6000 and maybe wait for the Ada Edition to come out of it that's that's your best bet but if you're trying to hit a specific price point like yeah I can spend this amount of money on a GPU then let's look at really what is going to get you the best GPU for that dollar amount I'm not going to talk about the dollar amounts of the actual gpus because if I do that this video is obsolete before I even upload it because there's a lot of factors if you're just looking at the msrps you'll make a decision and then you'll go on Amazon you'll go on NewEgg wherever and you'll be disappointed because they're going to cost more they're going to not be available so you've got to look at current market conditions and what what is going on there with supply and demand so you see a lot of things going on on this chart you have the Nvidia Cuda cores that is really going to determine how fast the GPU is going to do the Deep learning the clock cycle is important as well but fundamentally it's going to be how many Cuda cores you have memory size is important that's the amount of stuff that you can have loaded data the parameters of your neural network into the GPU as it's running and then notice the memory type the memory type shows how fast your RAM on your GPU is going to B the 30 series you'll see that there's two classes the gddr6 is slower than the gdgr6x in the 40 series there's just one type of ram at least for now so failing everything else what I really go for on a GPU is memory for deep learning it's looking at this chart on the memory size I would say 12 gigabytes is really sort of the sweet spot where a lot of things are going to work really nice for you if you drop into this eight gigabyte range which really only exists on the 30 series it's good to maybe get started with but you're going to quickly run into some limitations I used a 3060 TI even did a video on it it was very very challenging for me to get stuff even beginner-ish stuff working in that eight gigabytes the RTX 3060 with the 12 gigabytes that's a really interesting card to me I never found one of these actually in the wild I wanted to experiment with it if you can find it and you can get it at a great price I would recommend taking a look at that particular card now moving into the 40 series everything is above that 12 gigabyte sweet spot so like a 4070 versus a 4 4080 if the 47 DTI which is a very new one and is getting some Flack currently on its price point but if you can find one at a rate that the price point makes sense then might might certainly be worth looking at Nvidia has reported that the 4090 and the 4080 both speed wise beat anything in the 30 series so going strictly on memory I would look at a 30 60 RTX with the 12 Gig if I can find one and the price point is there then it's really a good deal if if that's all you have in the budget then I would move up and look at maybe a 30 80 or above on the 30 series and think about how much of that Ram you can really get into there I would buy the most Ram really I could because that's going to Future proof you the best but then look at the price point if the 4080 or the 4090 is getting closer that you that you can justify it then you can get some additional processing speed and it can get done quicker again all my opinions let me know in the comments what do you think believe me I learn as much from you as you do for me okay once you get that that GPU what is the next thing you do while you subscribe to my channel so that you are up to date on all of the latest technologies that you might want to use with your GPU because that's what I focus on is machine learning programming and often gpus come into that mix thank you for watching this video and please give me a like And subscribe if this was helpful it helps a lot thank you very much and have a wonderful 2023

Channel: Jeff Heaton

Views: 50,191

Keywords:

Id: F1ythHjdWI0

Length: 9min 8sec (548 seconds)

Published: Thu Jan 05 2023