Build your own Deep learning Machine - What you need to know

yo what's up world michael here in this toy is war machine so war machine is a pc that i personally built to take on deep learning and reinforcement learning tasks before building my own machine i actually wanted to buy a rebuild machine because i didn't want to go through the trouble of having to build your own PC i found a couple of companies that sold these type of machines which are lander labs and bison tank they have really good machines I really really really liked the machines but I don't like their prices so I thought hey I'm computer science degree how about I try and build my own can't be too hard right it turns out it's actually not that hard there are a lot of reliable resources online on how to build a PC what I found lacking or resources on how to properly build your own deep learning rate so in this video I want to talk about everything I learned on how to build your own deep learning machine towards the end of this video I want to talk about the pros and cons of training on your own personal machine vs. training on the cloud so some requirements for me when building war machine was I wanted some beefy so that means that ton of CPU cores and as many GPUs as possible I wanted to maximize my dollar because I wanted to ball in the budget where machine is inspired by the product so de Atlanta Labs the parts reflect linden lab parts but the prices do not reflect Linden Lab prices currently war machine has 12 cores 24 threads and NVIDIA r-tx to atti and 32 gigs of ram it will be expandable to have 4 GPUs and 128 gigs of ram ai training rigs require particular part so let's walk through the different components so at the heart of training deep learning models is the GPU GPUs are super fast at competing deep learning algorithm unlike CPUs with a very small amount of complex cores to do complex tasks GPU have hundreds or even thousands of simple cores that are super efficient at doing matrix multiplication which is perfect for deep learning the most reliable brain for deep learning is Nvidia cards that's because NVIDIA has something called a CUDA SDK which is just a software library to interface with their GPUs when picking out a GPU to get the most bang out of your buck you want to pick a GPU with something called texture cores so in terms of cores or something in MIDI and vetted that just specialized processing units to do specialized matrix math if your GPU has tons of course dad nil enable you to utilize fixed precision or have precision training so mixed precision and high precision allows you to do bigger batch sizes faster training and bigger models at the time of making this video tensor cores can be found in the nvidia r TX models I'm pretty sure all future models after that should have the tensor course as well so one of the most important things when getting in CPU is GPU memory any amount memory you need for your GPUs is actually dependent on the model sheet really not if you only want to train models for embedded devices which are smaller models then you can probably get away with a GPU with less memory but if you plan on training big models like in the NLP domain let's say Bert or Qi PT then you'll want as much memory as possible having more memory open up doors to you guess it bigger batch sizes faster training in bigger models also if you plan on Doudna multi-gpu setup like me make sure you choose GPUs with glower style fans so big mistake when building a multi-gpu setup it's not getting blower style fans lower style fans expel heat outside of your case so it's very important for temperature management I've seen a lot of threads online on GPU throttling due to overheating of your GPU because in in multi-gpu setup they don't have blower stop fans for war machine I went with NVIDIA r-tx to atti from Asus has 11 use of memory and blower style fans I plan on buying three more GPUs when you know my youtube blows up okay so that's it for get should be used to sum it all up make sure you have sufficient memory for your use case blower style fans if you want to do a MotoGP you set up and tanks of course you know for the best bang for your buck now let's talk about the CPU so more machine is equipped with an Intel I 9 10 9 20s has 12 cores and 24 threads it also has a clock speed of up to four point eight gigahertz CPUs are mainly used for data loading in deep learning the more threads on the CPU means that your training script can load more data in parallel so this is actually really useful when you train on big batch sizes because then your GPU doesn't have to wait for your data CPUs are also super important when you want to get to reinforcement learning in reinforcement learning most of your computation is done in your learning environment so you want to have a big enough CPU to tackle that task also in reinforcement learning if you decide to use large neural networks then a GPU also help if you're only planning on doing deep learning though then the most important thing for your CPU is to make sure it's compatible with however many GPUs you want so when choosing a CPU ask yourself these questions do you plan on doing reinforcement learning then you need a beefy CPU to train faster do you only want to do deep learning then you can get away with a smaller CPU but just remember that more threats you have enough CPU then the faster to data loaded do you plan on having a multi GPU set up then make sure your CPU is compatible with however many GPUs you plan on having so I end up going with Intel but I heard a lot of good things about the AMD's as well AMD are a better bang for your buck so I understand why you would choose that option my Intel im9 1090 20x is a very capable CPU of doing both deep learning and reinforcement learning so it works out and perfect for my needs now let's talk RAM a huge mistake that people make when choosing RAM is trying to get one with a high clock speed high ram clock speed it's like a marketing gimmick best explained by Linus tech tips in this video in deep learning a higher clock speed on your RAM has negligible improvement so you're better spending your money elsewhere what is actually more important is the amount of memory you have on your RAM you want to make sure you have a minimum of as much RAM as you do your GPU memory I went with a Corsair brand that has a clock speed of two thousand six hundred and sixty-six megahertz in 32 gigs of RAM when war machine is complete I plan on maxing out my ram at 128 gigs because you know I'm an extra like that okay on to the motherboard when choosing a motherboard make sure you have enough PCIe slots for however many GPUs you want also make sure your PCI slots have enough space to fit your GPUs each GPU generally takes about two space of the PC slots so war machine is equipped with an X $2.99 Sage motherboard I chose this motherboard because there has support for foreign GPUs and I also support my CPU of choice the only thing I wish it has is onboard Wi-Fi but I mean I connect with the ethernet cable anyway so it works out let's move on to your storage for storage there are two important things one you need to make sure you have enough space for all the data and the models and two you want something fast if you want faster data loading for storage if you want to optimize your data loading speed you'll want something fast like a solid-state drive solid-state drives are more expensive than standard hard drive so it might be helpful to buy two drives one SSD and one standard hard drive you can use the SSD for the main OS and also any data of interest that you plan on training on and then you can use the slower standard drive for long-term data storage and long-term long storage war machine is equipped with an mbmb samsung 970 Evo SSD the SSD has one terabyte of storage for the second drive war machine has an 8 terabyte standard hard drive from CKD ok now on to the PSU or the power supply unit for the PSU you want something with enough wattage to support your entire system a new rule of thumb is to take the required wattage for your CPU and all of your GPUs add those together and then multiply by a hundred and ten percent that should give you a general amount of how much wattage you need for your entire system make sure your PSU has enough PCI slots for all of your GPUs or machine is equipped with a 1600 watt PSU from Rosewill even though I don't need all that watch now because I only have a single GPU I will need all of it when I complete my set so I bought it for future proofing ok let's talk about cooling your system you'll for sure need a CPU cooler so to reduce fan noise I recommend buying a water cooler also if you have the budget you can also look into water cooling your GPUs water cooling your GPUs would make for a super quiet system if you stick to air cooling GPUs just make sure you have the blower style if you plan on doing a multi-gpu setup your machine is currently equipped with a corsair h 115 i Pro which is a water cooler for the CPU for the GPU I just have this day blow us off fans okay now for the case for the case this is pretty much just up to style so choose whatever looks good to you but make sure all the parts of it so some extra cautious our longest money I spend I went with a case with a lot of airflow so my components don't overheat I went with a Corsair 540 ATX case it has ample air flow I think it looks pretty cool and it's the exact case that lambda Labs uses so okay when shopping for parts I use a tool called PC part picker GC part figure has this feature where it checks for parse compatibility when you're building your rig so you know you don't screw up it's not perfect though because when I was putting all this together it told me that my case is not compatible because the parts will fit but I'll purchase it anyways because these parts were similar to what lambda Labs we're using and they managed to fit everything inside the case I still recommend it use as a compass so you can feel confident on the parts you purchase okay at the end of my training rig cost me a little over three thousand dollars which is still a lot of money but it's a very versatile and capable machine so I shouldn't have any issue tackling most deep learning problems once i purchased my three extra GPUs the entire bill would cost me around seven thousand dollars which is still about four thousand dollars cheaper than what you can get at Lind labs okay now many of you may be curious why build my own training ring can I just use the cloud and pay-per-use well yes yes you can but there are benefits to training on your own machine number one cost savings if you frequently use your GPU for training then training on your own machine can actually save you money long term if you're renting a b100 from AWS that's about three dollars an hour if you constantly train for the entire month it's gonna add up to about $2,100 a month you can build your own machine with that money and use it forever number two it's faster to train on your own machine your hardware is actually faster than the cloud that's because the cloud suffer from slow i/o because of all of virtualizations the cloud uses a bunch of extractions and virtualizations to optimize their own hardware and that causes slowdowns when you're training deep learning models both antagonism research with training on the cloud versus training on your own hardware and they found that cheap consumer hardware trains almost just as fast at the top cloud GP you could get now there are other free GPU options like Google collaborative but you're limited on a time you're able to use those GPUs so then your options are narrowing down when you want to start an experiment I do highly recommend you using them for most people starting out on deep learning though okay so that's the guide on how to build your own AI training machine if you found this helpful and if you haven't already please hit that like and subscribe button do you have experience building your own machine let me know your experience in the comments below alright peeps see you later and as always thanks for watching [Music]

Channel: The A.I. Hacker - Michael Phi

Views: 156,656

Keywords: deep learning, machine learning, buildapc, pytorch, tensorflow, nvidia 2080ti, lambda labs, bizon tech

Id: Utwnm2kjYAM

Length: 11min 57sec (717 seconds)

Published: Fri Jun 26 2020

Reddit Comments

[u/llN3M3515ll]

Interesting video, I am a dev that really hasn't done much with ML. Guessing the choice of CPU / Platform was mainly due to PCIE Lanes since you wanted to maximize the number of supported GPUs. I was a bit surprised Ryzens 3900x and 3950x had so few PCIE lanes, seems like you really would need to step into a 3960x before you could get all the lanes you need for multi gpu support for team red.

I guess its all up to what you're doing with it, but if you're spending $1,000 on a processor I would definitely consider looking at the 3960x. It's $300 more, but you're effectively doubling your core/thread count, as well as having additional features like pcie4 and having some additional PCIE lanes. This link outlines the raw specs side by side.

Caveat you probably should have noted is that this is really only a build for a multi GPU build. If you only ever plan on buying one GPU (and hence don't need the PCIE Lanes) there are far more cost effective platforms.

I have had that case you used for several years now, and I do like it. It provides good airflow and is super easy to work in. The downside is its big, heavy, and the cable management could be a bit better. Overall it has been a solid case though.

[u/Luxenburger]

It would be great to have a powerful computer that can do my deep learning work. Are you afraid if it being outdated in a few years?