Containerize Python Applications with Docker

what is going on guys welcome back in today's video we're going to learn how to containerize python applications using Docker which is a key skill in the industry so let us get right into it [Music] alright so for this video you will obviously have to install Docker on your system maybe you already have it maybe not if you don't have it you just have to go to docker.com get Docker or docs.docker.com you will find a link in the description down below here you can choose your operating system in my case windows and you can just follow the installation process it's quite simple you download Docker desktop for example you will need to have the windows subsystem for Linux 2 backend or the hyper-v backend so one of the two virtualization features or environments I have a video on the windows subsystem for Linux already so if you want to know how to install it you can watch that video otherwise you can also just follow the installation guide here on docker.com but you will need to have Docker installed this is basically the containerization environment you could say and the basic idea of using Docker or containerization in general is to package and deploy applications in containers and making everything more consistent and reproducible so if you have a contain Mainer discontinue will just run it will just run it's a virtualized environment you could say and it will just run without you worrying about the compatibility with your system you can package and run applications in those containers and this overall improves efficiency and scalability so it just makes sense if you have something that you want to be able to run everywhere you can just ship it in a container and if the environment supports running containers you can just run it everywhere that's the basic idea of using Docker of using containerization alright so once we have Docker installed we can go to the development environment of our choice in my case pycharm and we can write a simple python script that we then want to containerize it we then want to dockerize and in this case I'm going to do two different containers I'm going to do two different applications that we're going to containerize the first one is going to be a simple python script that runs a socket so a server you could say and if you connect to that socket it will give you some information and this information will be taken from a third-party module so that we have some dependencies here um so this is not necessarily a useful script this is just something that works and I'm going to show you how it works in a container the basic idea is we're going to have a server socket and if you connect to that server socket you're going to get uh one feature some data from a data set that is loaded from scikit-learn and the only reason we do that is so that we have a dependency on scikit-learn so that we can see how to install something automatically in the container because otherwise we would be just working with core python modules socket and something else maybe so we're going to start by importing the core python module time we're going to import also the core python module socket and we're going to import we're actually not imported we're going to say from sklearn dot dataset data sets I'm going to import load Iris so if you don't have scikit-learn installed you can just open up your command line you can type pip install kit learn like this and then you will be able to do this import as well but the good thing is when you ship a container you don't even have to have it installed on your system so I can actually once I have the container uninstall so I could learn so that I don't have this python library on my system anymore but in the container it will still know and have so I could learn so I can still run the features of scikit-learn even though it's not installed on my system because it's running in a container that's the benefit of this and the basic idea is to say data equals load Iris to then say server equal socket socket Socket Internet socket so AF Finance socket TCP socket so sock stream and then server dot bind now this is here important we need to bind it on a special interface here on a special host here which is the zero in string of course uh in in quotation marks of course 0.0.0.0 so we're not hosting this on localhost we're hosting this on this IP address which is important for the docker container uh and we provide a port for example four times nine so this basically means the server is now listening on this host on this port and now we can say server.listen to actually start listening and all we're going to do is we're going to say while true um client and address is going to be equal to server.accept so we're going to accept all the time incoming connections then we're going to print connection from and we're going to say that the address just connected then we're going to say client sent we're going to send something to the client like you are connected and then we can also add a backslash n because um on the other side in this case for the demonstration I'm not going to write another python script I'm going to run a uh telnet client and because of that we're not going to use the print function so it's not automatically going to do line breaks which is why I'm going to send them right away so I'm going to send your connected line break and then this needs to be encoded again the specifics of what we're coding here is not to are not too important because the focus is not on this script we're actually building the focuses more in the containerization and we're building the script to have something that is listening on a host and that is using third-party packages that's the basic idea you can also feel free to use another example if you want to so clients sent and what we're also going to send here now is we're going to send the First Column of the iris data set so we're going to say data of course in uh curly bracket we're going to say data and then we have the key data and here we have everything from the First Column so the column with index 0 and then we're also going to add a backstage and we're going to encode this um and then we can do something like time sleep two seconds or something and then client sent you are being disconnected or something like that and of course this needs to be encoded as well because we can only send bytes via sockets then we're going to close the connection with the client and we're going to listen for new connections so we're going to accept the next connection uh this is an endless loop so that is the script we can actually run this now without a Docker container so it's just listening we can then use a telnet client now on Windows if you're using the talent client or if you want to use the telnet client this is a deprecated feature an insecure feature you can also see so it's disabled by default if you want to do this for just debugging or demonstration purposes or just playing around with it you can go to Features so turn Windows features on or off and you can scroll down here to the telnet client and check it and then you will be able to use telnet in the command line on Linux it's simpler if you're doing this on Linux you can just use netcat so NC to connect to that host import and we're going to just open up the command line now and we're going to say telnet now I'm not sure if this already works with a host we provided but let's see if I connect to localhost or 9999 there you go you are connected we get the data again doesn't matter what the format is we're just getting some data here um and that is basically it so this is how this works and we want to now take this and reproduce the functionality in a Docker container so we want to have a container that we can just ship to someone this person might not have scikit-learn installed they will still be able to get the data from scikit-learn because in the container we will have all the dependencies and everything we need and this is done with Docker and with a so-called Docker file so what we do now is we create a new file and we call this file Docker file with a capital D and you can see already in Python we get this icon this Docker icon and what we're going to provide here for this script is quite simple we're going to say from and here we need to provide the python version so it's going to be python in my case 3.9 this is the version I'm using you can of course also provide python 3.10 3.11 or some stuff like slim or certain snapshots from previous or something but I'm just going to go with a classic 3.9 and then we're going to say we want to add some files and the only file that we want to add here is the main dot py file and we're going to add it to dot so to the current directory and then we're going to just run a command and this command is going to be pip install scikit alert so this is why I wanted to use a third-party package because if you don't use a third-party package there is no need to run pip install but by specifying this by specifying run pip install scikit-learn this is going to run this installation command when creating the docker container and in the docker container we will then have scikit-learn that's the basic idea and then the last thing quite simple is we're going to execute the command and the command is going to be python and then the second keyword is going to be dot slash main.py so we added main.py to the current directory and here we're just running it that's the basic idea um and in order to now build this container we need to have Docker installed as I already mentioned we need to have Docker running as well so if you have Docker installed you should have this Docker desktop app you want to run it and once this is running your uh you have the docker Daemon running so Docker is running in the background so you can use it and all you need to do now is you need to open the terminal or the command line um we're going to navigate to the current directory here and in the current directory I hope you can see this well enough I'm going to zoom in a little bit in the current directory we're now going to run the command that is necessary to build a container and this again only works if Docker is running so what we're going to do here is we're going to say Docker build Dash T and then the name of the container for example um I don't know Iris RS dockerization or something like that and then we're going to specify the dot to again Target this current directory for the build process I'm running this and then the docker container is being built you can see here the building process is running we will also see I think that it's going to install scikit-learn there you go run pip install scikit-learn we also added main.py you can see how all of this is happening and then after a while it will be done and we will have the docker container ready so now we have this Docker container it has this name uh Iris what do we call it Iris dockerization and we can now run this container as well by just saying Docker and then we can say run and if we want to use supports if we have some networking stuff that is running we need to map the ports as well so if you don't have ports if you're just running something you don't need to specify anything Network related obviously but if you have in this case a server listening on a certain Port you need to map that port to a port on your system so you want to say Dash p and the port 9999 shall be mapped to 9999 otherwise it won't work um and then we also want to say okay what are we actually running we're running the iris dockerization and then this is running you can also see here Iris authorization is running in the docker desktop graphical user interface environment and now even though nothing is running in pycharm even though I'm not running the actual script here in pycharm I will still be able to uh just say telnet one two seven zero zero one there you go so I can still get this connection even though um it's only running in the docker container so you can see that this works and I can also this is the most interesting thing now I can delete this now um what I can do here is I can also go to my command line and say pip uninstall scikit learn so this removes scikit-learn from my system and you will see that this is the case because I will not be able to run this script anymore I will not be able to run this because it says no module named SK learn so I don't have it on my system however I can still go into the command line and execute this last command to run the iris doctorization I can still open up my command line I can still connect telnet 127001 for localhost and you can see that I still get the data because in the container scikit-learn is present even though it's not active on my system even even though it's not present on my system I can do everything because in the container I have this reproducible environment I don't need to worry about dependencies I don't need to worry about installation everything in the container is there and I just have to run it and I can use it that's the benefit of using Docker containers all right now let us move on to an additional example we're going to containerize a fast API application and this is oftentimes the sort of hello world example for beginners when it comes to dockerization and python if you Google how to containerize python applications using Docker you will oftentimes find examples using fast API and because of that I wanted to include such an example here as well and for those of you who don't know fast API is a simple framework that allows us to easily build rest apis I have a video on it as well and the basic idea is you specify a couple of functions you map them to a certain endpoint and then you have your rest API without having to use full-blown web Frameworks like flask or Django fast API is as the name already says the best library to build a simple API fast and we're going to build such a simple API now with just a single endpoint and we're going to containerize it using Docker so in order to be able to work with fast API we're going to use pip to install fast API and you UV corn even though I'm not sure if uvicorn is not automatically installed using fast API or by installing a fast API but you need those two things and then you're going to have to import UV corn and also from Fast API import fast API like this then we're going to create a simple application by saying app equals fast API and then we're going to provide just a single endpoint just to so that we have something and then we're going to containerize this whole application and we're going to call the function here main function even though okay this name is probably misleading let's call it uh Central function or something you can call it whatever you want and we're going to just return a dictionary here that says neural is the key and 9 is the value so this is our Json object and in order to get this response you will have to send a get request so app.get to the root endpoint so just slash that's the basic idea and now in order to run this what we do is we say if underscore underscore name equals underscore underscore main underscore underscore if that is the case we're just going to say uvicorn dot run and we're going to run the app we're going to specify the port 8000 and we're going to say that the host is again 0.0.0.0 and that is basically it so this is our fast API of course you can make it more sophisticated you can have multiple different endpoints with multiple different HTTP methods supported and those do some data science work or something but this is the most basic example which we're going to use here for the containerization I can just run this now and then you can see that this is running now on zero zero zero eight thousand I think I should be able to access it on localhost there you go and you can see I get neural and nine that is our API now and we can containerize this API with Docker so I'm going to close this now and usually you have a structure that is not just a main file you usually have something like a source folder so you have SRC and then this main file is inside of SRC for example um and then we're going to also have again a Docker file so we're going to have a simple file Docker file with a capital D again and in addition to that this time we're going to provide also a requirements txt file so that we have a file with all the modules needed so we're going to create a new file here requirements.txt and the basic idea is here you just list all the modules all the dependencies of your application in this case uvicorn and fast API so just uvicorn and fast API if you want to have a specific version you can just say equals equals and the version number but we're going to just specify the names now and of course if you're using pandas or scikit-learn or matplotlib or anything you just put it into into the requirements and then you have to install it of course with a certain command in the docker file but we're going to do this um here so again we're going to start with a simple from python 3.9 we're going to then say the working directory is going to be let's call this I don't know neural Dash API and uh this just specifies the working directory the context of this whole thing and we're going to then copy the requirements txt file into uh the current directory and we're going to also copy The Source folder in this case there's only one file in the source folder but in general we're going to copy the whole source source folder and we're going to say that we want to put it into another folder that has the same name so we're getting this folder here and we're putting it into a new folder that has the same name that's the basic idea here we're using the copy command for this and then we just say pip install Dash R requirements txt because this file is now added to the current directory we can just use it in the PIP command and once this is done we just have the same command so we just specify again Python and then dot slash Source SRC slash main.py and this is the docker file that we need here now what we can also do is we can um we can basically use the same build command now but we can also specify a name so I can go into the terminal here I can navigate to the directory and I can say again docker uh built Dash T neural Dash API for example um and then we're going to when we're running this we can specify a name so if you go to the docker desktop here you can see it just assigns a random name determined Diffie or or difi Diffie I think and then the actual container but we can also specify such a name manually we can just go ahead and say Docker and then again run we're going to map 8000 to 8 000 again since this is a an API running on a port we need to map the port again this port here um and then we can just specify dash dash name and we can say I don't know my container name and then I can specify neural API and I think I didn't forget anything so that should work there you go you can see it's running and here we can see my container name so I can just use it from here now I can go again to localhost and you can see the API is running even though it's not running on my system so once again I can go open up my command line say pip uninstall fast API and then also pip uninstall ubicorn there you go it's gone I can also stop the container now again hopefully there you go it stopped I can just run it here again from a desktop so I don't have to always type in the command but you can see the API is still running even though I don't have fast API on my system it is in the docker container and because of that it works so that's it for today's video I hope you enjoyed it and hope you learned something if so let me know by hitting the like button and leaving a comment in the comment section down below and of course don't forget to subscribe to this Channel and hit the notification Bell to not miss a single future video for free other than that thank you much for watching see you in the next video and bye [Music] thank you [Music]

Channel: NeuralNine

Views: 65,140

Keywords: python, docker, python dockerization, python docker, container, containerization, python container, python containerization, python docker container

Id: 0TFWtfFY87U

Length: 20min 51sec (1251 seconds)

Published: Mon Dec 19 2022