Installing TensorFlow/Keras CPU/GPU w/CONDA (July, 2020)

hi this is jeff heeton welcome to applications of deep neural networks with washington university so to get going in this class you need to be able to run the examples run your assignments and get them running it's working in python so you're using tensorflow with kira's in python you can install this all onto your own computer and in this video i talk about how to do this to see all my videos about kaggle neural networks and other ai topics click the subscribe button and the bell next to it and select all to be notified of every new video if you don't want to install a lot of software on your computer or maybe you're not as technical with dealing with the command line then i recommend google colab either way you're going to have access to a gpu which is a graphics processing unit now you may not have a gpu that's compatible on your actual computer it needs to be nvidia and it needs to be one of the the later ones you can google the capabilities of the gpu in your actual machine and find that out the gpu is it's a card basically there i'm using a titan which is fairly high-end one in this computer but i recommend you maybe try this on your computer try it in collab see see what works best with you in this video we're going to see how to install python with tensorflow and cara's on your computer i'll cover both how to do it with a gpu and just straight up cpu due to some advances in anaconda python those two processes are really pretty similar okay let's start with the installation process okay i'm going to show you now how to install the files that you need to actually use tensorflow for either the cpu or the gpu we'll cover both and this allows you to run the class examples for my applications of deep neural networks class even if you're not following along with my class this will get you a conda environment set up so that you are in a good place to actually begin working with tensorflow and curas i'm starting with a brand new fresh install of windows so that's why it's saying activate windows i do actually pay for windows it's just when i install it for just something that i'm going to throw away within a day i usually don't activate i don't want to confuse the microsoft activation server which it would probably figure out okay so this is a clean build i do have chrome installed i am mostly familiar with chrome i'm not that familiar with edge so i want to not be tripping over unfamiliar software and that really doesn't matter to the install what we're going to do is if you search for jeff heaton and github you can see my github here and the files and instructions that you need are here and if you go to t81 applications of deep learning this is it go ahead and go into well while you're here at the main page go ahead and click code and download a zip you can also use get clone depending on how sophisticated with get you are but this will work just fine so it downloads that zip file while that's downloading i am going to go ahead and go into the install folder that i have set up here i've got a couple of these here if there is a newer one here probably try to find the youtube video that goes along with that because this does change this is the july 2020 version i will try to keep that one there as long as possible i'll only re-record the video or create a new file of that if it changes so much that this video is just no longer relevant but i tend to re-record this at least once a year google always makes some sort of change to how this goes that it causes me to re-record so we'll go ahead and open this one essentially what i'm telling you to do here is install miniconda you can use anaconda if you prefer anaconda has every package and its brother known to man installed i tend to like to be more minimalistic but this is a personal thing you can install whichever one you desire but let me go ahead and go to the miniconda link i'm opening another browser so that i can get back to it we're going to install python37 for windows the mac installation is really mostly the same as this video i'll probably record a mac specific version of this video at a later time okay we're going to use python37 and 64-bit if you happen to have a 32-bit machine tensorflow is not going to work anyway use google collab also if you're just not a software installation person you're more just a coder not not a command line hacker google collab is great and highly recommended so let me get this downloading this should be pretty quick but i'll fast forward through it okay once you've got that go ahead and run it i just double clicked it click next agree i prefer to install it really just for me i don't tend to have multi-user computers but and that directory is fine it's under my user directory always remember where that is because you may end up going there to adjust things later i do vary from the default on this one i prefer to add it to my path environment you can use the anaconda prompt as well both of them should work without changes to these instructions the only case where this might mess you up is if you have other software on your machine that is dependent on a version of python they previously installed and changing that global path is going to force them to use this new version that you're installing so be aware of that i'll click install the anaconda installation will take much longer than the miniconda installation so this should go through pretty quick all right and we're pretty much there so click next and i don't need tutorials and i don't need to learn more about anaconda alright so now that's installed before we get too far with this that file that you downloaded i've got a couple of files in here from different things i was experimenting with on this build but you can ignore those this folder here which is what you downloaded from my get repository i'll go ahead and open that i'm going to just drag it to the desktop for now needs to unzip it and i'll go to my user directory and just copy it into there so now i've got it ready to go these are all of the class examples that give you a few files to test it and other things there's really just two files that you actually need you can you can download those as well if you don't want to download the whole thing so you successfully installed miniconda now you need to open a command prompt cmd this might work in powershell too i'm not that familiar with powershell and i'm just going to do conda install minus y jupiter we'll go and fast forward through this this might take a bit okay jupiter is installed now what we're going to do is create a conda environment an environment holds all of the python libraries and everything that you need for a particular project you get a base environment that you start out in i tend to just use that just to run jupiter and i use the underlying environments to for each of the custom configurations that i need for various projects the other thing that is great about these environments is you can automatically install the gpu drivers that you need and it just greatly simplifies the installation and it prevents collisions you might have an old environment that is using old cuda drivers a new environment that's using newer ones and it just really takes a couple of hours off your installation time so i'm going to go ahead and create a new conda environment i'm just going to copy this and paste it into here now the important things are tensorflow is the name of the environment you can call it whatever you want to just make sure you change it in the subsequent commands so that everything lines up and then this is the version of python that we're using 3.7 the versions of tensorflow that i am using for the fall 2020 semester that i'm setting this up for require or at least can use python 3.7 i will up this in the future as i modify these instructions each semester but for now it's python37 so i'm installing this say yes we'll fast forward through this it's pretty quick okay it is done so now we're gonna do conda activate the command that i have here is the same that they're suggesting here so conda activate tensorflow and press enter notice your prompt changes you're now in tensorflow now a lot of these commands that i have down here require that you be in this tensorflow environment that you created whatever you happen to name it and won't work and if you're not so make sure that if you're finishing this over a couple of passes that you always go back to conda activate tensorflow and a very common error that people always ask me about is you'll get a dll error and if you get that it's usually because you're not in the environment so now let's run this condo install nbconda this just gives you some needed tools so that you can link this new environment to jupyter notebooks jupiter notebooks are basically the editor we're using in this course i'll fast forward this and it's done now we're ready to install tensorflow now as of the recording of this video we're right around the tensorflow 2.0 to 2.2 version the platforms that we're dealing with in this course are google collab 2.2 windows 2.1 linux 2.2 and mac 2.0 generally i found that mac is the most behind as far as what conda has released for tensorflow versions and linux linux and google collab stay practically neck and neck are the more current ones now if you're using windows 10 you can actually install a linux subsystem in your windows environment and actually install the cuda drivers to go in conjunction with that i'm going to do a separate video on that if you're a relatively advanced installation not afraid of the command prompt user you may want to do that i think that is sort of the best environment in windows but i will soon have a video on that and i'll link it in the description but we're doing windows so as of the writing of this windows is 2.1 of tensorflow that conda supports now you can use cpu only or gpu and cpu gpu is great if it just magically works if it doesn't magically work you've more got to install the drivers and everything and get cuda and coo dnn and all these various things working cross your fingers and hope that this works if you are doing it for gpu if not i have another more hands-on video showing how to literally install all the drivers that might be an option but i will be honest with you installing all the gpu drivers can be a headache and there's various other options for using the gpu like docker images i'll probably do videos on those in the future i do have a gpu on this machine i have a titan rtx so i am going to go ahead and use this command but if you wanted to install this just for cpu use this command if you want to be really fancy set up two environments and that's what i actually have on my work machine one for cpu one for gpu and then you can switch between them so i'm just going to paste that in there by the way quick way to paste is just right click make sure you're in your tensorflow environment and run it potentially if things are going to go wrong this is this is a good place for them maybe to go wrong so if you do get an error here go ahead copy paste it and google it you'll probably find something on stack overflow and hopefully that will guide you you can always post it to the comments here and maybe somebody will have an idea i'm going to go ahead and say run i haven't seen every thing that can possibly go wrong so i may or may not be able to help you but if it's something i've seen i will post a reply in the comments and also look in the comments see if anybody has had the same issue as you and got a response from somebody possibly me we're going to fast forward this this takes a little while all right no errors so that's good now the next thing we're going to do we're going to install some additional libraries that my class needs but these are all very common i i suggest maybe using this as a starting point they're located in this tools.yaml file and that is inside of that zip file that we downloaded earlier or you can just download that file directly if you don't want to download everything and you can see it right there i am going to basically copy this command so and then paste it into here again make sure you're still in your tensorflow directory if you've closed your dos prompt and reopened it then you might not be there so we'll go ahead and run this this takes it a little while to install these this is installing things like scikit-learn and other other things that we need for the class i'll go ahead and fast forward through this okay and we're done you don't need to reactivate it just ignore what it said there so long as you're still in tensorflow or whatever you named it you're fine now this is a very important command here if you don't do this you won't see your new environment in jupiter and you won't be able to use it so we're going to run this command here now if you run this command and you get an error like it can't find the ipyn kernel that means that you forgot probably to run this conda install nbconda i get that question from time to time so just so you're aware of that i'm gonna paste that into there this happens quick so it's done now what we're going to do is we're going to run jupyter notebook now on windows it is very very important that you start your jupyter notebook in the environment that you're planning on using it's a little silly that in jupiter notebook you can pick between all these environments but you have to have started in that one so it's it's almost redundant on windows but nonetheless this is how windows works there is a github issue open about it so maybe they'll fix it at some point then i'm going to run jupyter notebook and by the way if you try to run this from outside of tensorflow you don't you don't do the conda activate tensorflow you'll get a dll error i'll demonstrate that actually since it is a very common error that that people get so i'm gonna run jupyter notebook it's gonna launch a web browser and i'm going to go ahead and open up class 1 overview now in you do need to select i called it tensorflow here if you called it something else you need to do cond kernel change kernel and pick the one that you that you have which is which is right here so we'll go ahead and actually it looks like my instructions named this 3.6 even though we are using 3.7 so i'll i'll update that that's just a label it it does not matter but i'll do kernel restart and run all and it's going to run this part here which is critical that tells you the version that you're running of everything so python377 so yeah i need to update my label there which is easy enough and then the key here is the tensorflow version as of this video 2.1 is the latest that you can get on windows unless you literally install everything kind of from scratch without kind of helping you which is which is not horrible but it's doable notice gpu is available so i've successfully installed this for gpu if you installed it for a cpu it'll say gpu is not available so this is this is how you get this up and running for gpu or not for cpu now let me show you a very very common error hopefully you're not getting this error but i just want to show you some of the most common things that can go wrong now if nothing's going wrong you're done please subscribe and i hope this was helpful for you now if you're if you're getting other errors here's one of the most common ones i'll do conda completely like you're just starting up believe me i make this mistake all the time and i go into the course files and i don't do a conda activate i just go jupiter notebook now in mac and linux this will this is fine there's no issue here but in windows this will cause an issue so i'm gonna go into class one overview just like i did before and i'm gonna do kernel restart and run all and notice the error that i get here by the way this is the error that you would probably want to copy and paste and google on but dll load error it just means you did not activate your environment prior to running this so just just be aware of that thank you for watching this video now this seems like a hairy process that maybe you've tried and didn't work that well feel free to use google collab for the class assignments and examples it really works quite well i test everything with google colab so you should be fine if you find this kind of thing interesting please subscribe to my youtube channel or give me a like if this video helped you out thank you very much

Channel: Jeff Heaton

Views: 65,445

Keywords: jeff heaton, nvidia, titan rtx, washington university, tensorflow, keras, cuda, conda, deep learning, neural network, python, deep neural network, wustl, machine learning, artificial intelligence, deep neural networks, GPU, data science, python neural network, google colab, gpu, python introduction, python introduction deep learning, pandas, tensorflow 2.0, anaconda, colab, google colab tutorial, google colab tips and tricks

Id: PnK1jO2kXOQ

Length: 18min 27sec (1107 seconds)

Published: Wed Aug 05 2020