How to Run Python 3 and Jupyter Lab using an Anaconda Docker Container and VS Code

hello everyone in this video i want to show you how you can set up a development environment on your local machine where you run python code in a docker container now there's a few reasons why you might consider doing this and the number one reason is for reproducibility if you run a container that's based on a commonly available image or an image that you've created and can distribute with the world then other people can reproduce your code results as if they were running their code exactly in your environment now if you're new to docker no worries there is kind of a learning curve i would say because immediately when you start working with docker and start seeing some of the terms you're like oh okay so it's like a virtual machine and it is like a virtual machine but there are some key differences that make a docker container not a virtual machine so what i hope to do in this video is just provide an introduction to getting docker and solving your machine and starting to develop python code in a reproducible environment called a docker container along the way i hope that you pick up some key terms associated with docker and see how it is like a virtual machine but still different from a virtual machine and see how you might use it in your future development all right so let's get started the first thing that you're going to want to do is download docker desktop so you can google download docker desktop and it'll bring you to a page that looks like this docker.com products docker desktop now you can see that docker is cross-platform so choose the download for your operating system so i'm running on a mac right now so i'm going to download for mac all right so we'll need to save this somewhere i'm going to save this in my downloads folder this is the installer for mac docker.dmg now this looks like it's about a 550 meg file so looks like i've got about 30 40 seconds left in my download while this is downloading i will equate what we're doing to a little bit of your background you might have in running virtual machines so with a virtual machine you have a host operating system which in my case is mac and you have a guest operating system which might be linux or windows or ubuntu or something like this so you download something like virtualbox or vmware or qmu or some kind of program that will run on your host operating system so you can run a guest operating system so essentially what we're doing is downloading the container version of virtualbox okay we are not going to be running full-blown virtual machines though we're going to be running docker containers which are much lighter weight and don't include a full-blown guest operating system like a virtual machine all right that was enough talking to cover my download here i'm going to click docker.dmg and i'm going to run the installer if you're on another operating system make sure that you run that installer if you can't simply launch it from your web browser then go to that folder where you downloaded it and try and launch it from there all right looks like we've got a little bit more time to talk about docker while it's essentially extracting and repairing the docker installer so think back to virtual machines and how you have a virtual machine disk image right usually like an iso image iso that represents essentially the image for the operating system or a definition for an operating system that you could use in order to run that operating system okay so you typically launch something like virtualbox you load in that disk image or that iso and it extracts that image and launches it as a running virtual operating system now we would call that an image and then when we run that image we would call it a virtual machine now with docker we have images as well these are static files that represent oh here we go drag and drop these are static files that represent our application image okay so we're not actually essentially packaging up and using an image for a full-blown operating system we're packaging up the entire tech stack and dependencies for an application or a set of applications okay that's what makes docker more or docker images much more lightweight than a virtual machine images is it's really everything you need to run an application versus everything you need to run an operating system all right so i'm essentially copying docker into my applications folder this is a one and a half gigs so we'll watch this progress bar but it'll take a little bit of time all right back to virtual machine analogy so when your virtual machine is running your guest operating system you can install additional programs you can save files to the virtual file system etc well with a docker container which is an instance a running instance of a docker image you can do the same thing you can install additional programs you can modify the file system so if you delete that container then you essentially only delete the instance the image will still be there and you can create another container running from that instance so looks like i've successfully installed docker so i'm going to launch docker desktop now so how you do this is dependent on your operating system so i'm going to simply hit command space bar to bring up my searchlight you can also do this in the upper right hand corner here all right so here it is and i'm going to type in docker and it looks like docker.app i could launch it if you're on windows you probably have a shortcut on your desktop or you can simply go to your start menu and then search for docker all right it says docker desktop needs privilege access to install its networking components and links to the docker apps you will be asked for your password i'll hit ok and i'll type my password in here if you're on a windows machine during the install you might be prompted to enable wsl 2 features make sure you keep that box checked you might be prompted at this time to install those additional features alright so now at the top of my task bar here i've got a little entry here for docker desktop it says docker desktop is starting i'll click on dashboard to see the state of things all right perfect and you can see on the left side we've got two entries here container slash apps and images so while we were installing docker i provided hopefully somewhat of a clear overview of the difference between an image and a container so an image is essentially a static definition for your app and its tech stack and then a container is an instance of that image so you can think about this in terms of object-oriented programming right the image is like your class definition and the container is like an object and what's cool about these containers is you can start and stop them kind of similar to how with objects you can serialize them to disk and then deer serialize them later if you want to kind of reinflate them and use them at run time what we want to do now is we want to create a docker container running from a docker image that we fetch from the web the docker image that i want to fetch from the web is called anaconda 3. if you've heard of anaconda 3 before you'll know that anaconda 3 is a python 3 distribution in fact it's the most common python distribution for data science and scientific computing it contains a lot of really great libraries like numpy scipy pandas matplotlib seaborn scikit-learn etc so i'm going to google let's do anaconda 3 docker image and the first hit here is for docker hub so docker hub is essentially a site that hosts docker images so here's continuum io this is the organization that maintains the anaconda 3 python distribution slash anaconda 3. so this is a docker container with a bootstrap insulation anaconda based on python 3.5 that is ready to use all right so here under usage we've got some commands for docker that we can use to download and run this image all right so we can follow these instructions here but i've got my own instructions that include some flags for the run command that are really helpful i think when getting started working with docker so i'm going to flip over to my own notes here but i did want to show you the source of the docker image we are going to be downloading all right so i've got instructions here called create anaconda3 container from an image and so i've got this docker command here that is meant to be run at the command line so terminal on mac linux or command prompt on windows and you will have to grab a different command depending on what operating system you're running the reason why is because i've set it up so that it uses this little environment variable pwd which just grabs the present working directory or the current working directory that you're running your terminal in but on windows you'll need to use percent cd percent instead of dollar sign pwd all right so here's what we're going to do let's copy this command and then head over to your terminal or your command prompt and let's make a directory and this directory is going to essentially be mapped to the home directory of our container when it's running so i'm going to make a directory i'm going to call it anaconda 3 temp all cd into anaconda310 and i'm going to paste this command here so let's go through the command so docker run is going to attempt to run this image right here continuing i o slash anaconda 3 colon 2020.11. so this 2020.11 this is the particular tag or the particular release of this image that we want to run so let me show you where that 2020.11 comes from so if you go back to our continuum i o slash anaconda 3 page on docker hub okay here's the overview you click on tags and you'll see that there are a lot of different releases of this particular image and to uniquely identify them they are tagged so the latest released always has the tag latest and you can see that this latest release was last pushed to docker hub a month ago december 1st at 205 pm if i look at the next tag 2020.11 okay last pushed a month ago it is actually the same image so the latest tag always refers to the most recently pushed image and so here that corresponds to this image the one with tag 2020.11 so rather than just grab latest we're going to grab 2020.11 so that we all have the exact same one right because in a few months there might be another uh latest right but if we grab that one it'll have different versions of python libraries and whatnot than the 2020.11. so let's grab 2020.11 and we will use this one all right so docker run continuum slash anaconda 3 tag 202111 okay what's all these extra flags in between here so the dash i dash t this is going to launch an interactive terminal that is essentially a bash shell connected to our running container where we can issue commands once the container has been launched so that's really helpful to immediately get a cli or a command line interface to our container once it's launched the dash p here is essentially setting up publishing a port on your container to your local machine or your local host so essentially what it's doing is it's taking uh hostport 888 and it is mapping it to container port 8888. why do we need this in here well one of the common use cases for the anaconda 3 docker image is to use it as a container to run jupiter notebooks so jupiter notebooks run on a local host server and you typically specify a port like 888 that that jupiter notebook or jupiter lab runs on all right next is dash v and i alluded to this earlier when i was talking about how with virtual machines you can set up a shared folder between your host and your guest os well you can essentially mount a directory in your container that is mapped to a folder or directory on your local machine so that's what the dash v does here okay it's going to mount the current working directory that's the pwd here colon to the home directory of our container next the dash dash name flag here this is going to set the name of the container to be this anaconda 3 cpsc322 i can change this if i want maybe anaconda 3 underscore temp since i'm going to be deleting this container after the video all right so let's press enter it says unable to find image continuum io anaconda 3 tag 2020.11 locally so it's gonna go pull that from docker hub so this is really like a get pull where it's fetching some data from a remote repository so this will take a little bit of time we'll let it run all right so it looks like the poll is complete and you can see that our prompt has now changed before it said g sprinting is macbook pro desktop percent sign because we're working with a z shell here but now i've got this root at and this long hash here and a pound sign so here i'll do a pwd and you can see that i'm actually operating in the root directory of my container so this is really cool any command i run here at this terminal is going to execute in my container not on my mac all right so i'm going to head over now to our docker desktop dashboard so we can see that we have fetched this image and we have a container that is running based on this image all right so here's docker desktop dashboard so under images i now have one local image there it is here's the tag and it is in use you can see why this took so long to download it's 2.71 gigs okay remember this is an image so this is a definition that can essentially be inflated or instantiated as a docker container to run the image all right so now under containers i've got one running here anaconda 3 underscore temp it is based on this image with docker desktop there are a few commands that have essentially been wrapped into the gui so if you kind of hover over your container you can see we've got a few here open in browser cli stop restart and delete so if you ever need to grab that command line interface because you've lost it for some reason right right when we run it here we've got it you can always press this button here to get a new cli when you're done running your container you can stop it and you can also restart it when you're done with the container completely you can delete it okay what i want to emphasize is this command here docker run it runs a new container from an image so let's say i exit out of here and then i run this command again i'm going to take off the name here okay now i've got another docker container this one was assigned a random name let's exit let's run it again okay i've got another docker container okay so every time i run the docker run command it's creating a new container from the same well from whatever image you specified here so for us it's the same image okay so let's stop it let's delete this last one and delete this second last one and now we're left with this anaconda 310 the original one that we made all three of those containers were nearly identical because we didn't change anything in them so let's restart our anaconda 3 temp i'll hit start here i can also run docker start and then the name of the container from the command line all right so let's say we want to get a command line interface to the anaconda 3 temp container shell i'm just going to go over here and click this cli button you can see the docker command that is used in order to give us this shell so we can do docker exact dash it if we wanted to launch this commandlet interface from the command line all right so now in here we essentially have a shell to an anaconda 3 python environment so let's try python dash dash version and we'll see that this is running python 3.8.5 so i can launch python and now i have an interactive python terminal where let's say i want to print out hello world okay so if you are following along and you do this you'll be running the exact same version of python and all of its dependencies included in the anaconda 3 python distribution as i am there won't be any discrepancies it will run exactly the same all right so the next thing i want to show you is how you can launch jupiter notebook or jupiter lab from your anaconda 3 container so recall earlier i said that we needed to have our port 8888 published on our host and our container machine so that we can open up jupiter lab in a browser on our local machine that is tied or running in our container so let me show you how you can do this so i've got some notes here on how to do this so launch jupyter lab using the container so in the container cli which we have open we're going to run this command to launch jupiter lab and if you'd rather launch jupiter notebook then just replace lab with notebook and then do ip this is our local host here's the port to launch it on launch it without a browser uh allowing uh root access and setting the notebook directory to be the home directory okay so this is on the the home directory of the container which we have set up to be tied to our anaconda 3 temp director we made on our local machine okay so like normal we're going to be prompted to copy and paste this url into our browser in order to access jupyter lab so i'll do this all right so this looks like it's running uh locally on our machine and it is except that this port is tied to a server listing on port 888 on our container so when we run say some python code in here we're going to be running that same python 3.8.5 that's running in the container so let's try it let's do a new notebook and i'll just print out hello world okay so i run it prints out hello world fantastic uh let's print out another one just for fun so it's different than what we did at the command line with interactive python okay so i'm going to rename this i'll just call it uh testnotebook.ipynb and this is saved in the root directory that jupiter lab was launched to be in so its current working directory is actually if we go back to our command it's actually where'd it go right here the home directory and recall the home directory when we created our container is mounted or tied to our present working directory that we ran our docker run command in so if i head over to my other terminal here anaconda 310 and i do an ls then i see right there test notebook.i p y and b how cool is that all right i only have one more thing that i want to show in this video and that is how you can get an ide like vs code connected to your container because sometimes we like to write our code using terminal editors or command line editors like vim or emacs and sometimes we like to write our code using a graphical user interface like an ide or integrated development environment so let me show you how you can do that so let's say you have vs code installed on your machine if you don't have it installed you can go to this page code.visualstudio.com download and download vs code for your platform now if you worked with vs code before i would encourage you to try it out it's a really lightweight editor that is becoming increasingly popular and it has a lot of extensions that you can install into vs code to get some really advanced functionality in fact in order to run vs code on our local machine like my mac and connect it to our container so that we can use that anaconda 3 python distribution inside of it we're going to have to install an extension all right so let's do i'm going to launch vs code inside of my anaconda 3 temp folder all right so what i'm going to do in here is i'm going to go to this bottom icon here which represents the extensions for vs code and if you haven't already installed the python extension i highly encourage you to install it all right what we want to do is we want to install the remote containers extension so it says open any folder or repository inside a docker container and take advantage of visual studio code you can read more of a description about it here all right i'm going to click install and just like that we've installed the remote containers extension which if you saw gave us this little green status bar icon down here in the bottom of vs code that says open a remote window so we're going to click this now and we have all of these different remote containers extension options we want this one attached to a running container so remember if you have a docker desktop dashboard open you can click on containers and see all of your containers that are currently running you can connect to any of these containers from vs code once you've installed that remote containers extension so i want to connect to this one so i can use that python 3.8.5 installation that's inside of this container all right so i'm going to click i'm going to do attached or running container vs code is going to go see okay what containers does your dr damon have running anaconda 3 temp perfect it's going to launch a new window you can see in the bottom left-hand corner that nice kind of green remote containers extension status bar icon is saying okay here's the progress we're opening the remote and bam we're connected to this container continuum i o anaconda 3 2020.11. you'll occasionally get these little pop-ups saying your service running on this and this is available just ignore that okay now what we want to do is we want to open a folder in this container namely we want to open that home directory in the container so click open folder we'll do slash home there it is hit ok and now what's really cool is our home directory in our container is the current working directory in vs code all right so when you launch vs code it may prompt you to select your python interpreter sometimes it just pops right up you can see it down here in my status bar it says select python interpreter so if you click that or command shift p will open the palette the command palette in vs code if you're on windows ctrl shift p and you can always just type python select interpreter to bring that up okay so the python interpreter that you want to choose is this one right here the 3.8.5 this is the python interpreter inside the container for python 3. all right so let's test this out i'm going to make a little main dot py and i'll print hello world for the third time and if i run this hit my little play button up here okay i see hello world and i'll just make sure that my python here is python 3.8.5 so this is not the python installed on my mac this is the python that's included in the anaconda 3 image that we've created our container to run based on all right when you're done working with this remote connection you can just come down to the bottom here click on this green icon and choose close remote connection and that will remove the connection from vs code to that container well hopefully you learned a lot in this video we covered a lot from the basics of what docker is and how you can compare it to working with a virtual machine to actually installing docker installing visual studio code setting up an image and a container oh i have so many things open uh setting up uh the continuum i o anaconda 3 image by pulling it from docker hub and then inflating a few containers from that image running them and then connecting to that image with graphical user interfaces like jupiter lab and vs code and just having straight command line access to it via shell alright that's it thanks for watching

Channel: Gina Sprint

Views: 2,179

Keywords:

Id: cK7vgjOntqM

Length: 31min 1sec (1861 seconds)

Published: Sun Jan 17 2021