Python Executable with PyInstaller - Day 28 - 30 Days of Python

now one of the coolest things about software development is to package up your application so you can share it with somebody else and they could use it like think about vs code we actually didn't install electron or node.js the two technologies that for sure drive vs code we just installed vs code and that's really cool because it makes it really easy for everybody to just run that program we want to do something very similar with python and we're going to be doing that today in day 28 using a tool called py installer pi installer and we're not going to be creating a gui or a graphical user interface like vs code instead we're just going to be able to execute some python code as a bundled package so let's get to it [Music] in this one we're going to be using pi installer or py installer to bundle our python app and all of its dependencies in theory into a single package and pi installer doesn't support 3.8 yet or python 3.8 so if we actually scroll down on pipe.org what we'll see is only 3.5 to 3.7 are supported so you're going to want to use one of those versions by downloading it on python.org or however you download python in the first place and just finding the system that you're going to want to work with now for us we haven't done anything in python that python 3.6 couldn't also run we've been using 3.8 but 3.6 is definitely useful still and it will be for years to come i mean they even still have python 2.7 listed which is incredibly old now um so let's go ahead and jump into vs code and do the next caveat that we have which is well it's a couple things first let's go pwd and what we want to make sure is when we actually use our project we use an underscore and no spaces so pwd or dir if you're on windows you want to make sure that there are literally no spaces in here the next thing is we want to go ahead and say python 3.6 and dash m v e and v period so we're going to be using v e and v as our virtual environment manager instead of pip env because of how pi installer finds things vemv to me has been much much more useful than pipim v with pi installer at this point maybe in the future that's going to change but for now we just need to put all of our virtual environment stuff in the same folder that we're working on it doesn't matter what system you're on just make sure that you see all of the virtual environment things in there in other words if we actually type out source bin activate on mac and linux your virtual environment will activate if you're on windows you'll do dot slash scripts slash activate okay so now we are finally ready to install uh py installer so we'll go ahead and do pip install py installer now if you accidentally install py installer to your entire system you're going to want to uninstall that you want to make sure that you're using py installer inside of the virtual environment that you're working on because if you install it globally there's going to be conflicts and stuff like that so just do your best to not install it globally so the next thing is we just want to verify that py installer is installed and we see you know this sort of error when we write it out where it's just giving us a bunch of advice of what we can do right so you could also probably do python-m py installer after it's actually installed but in this case it looks like it's not working but i have had that work before on other systems if for some reason yours is not okay or rather if this is not so now that we've got that it's time to actually build some baseline project before we even use pi installer but there's really no purpose in going forward if you're not able to get here all right so we're going to use flask and waitress to build a web application so let's go ahead and do pip install flask and waitress waitress is similar to g unicorn where it's actually going to handle running our flask application this is something you can use in production as well on a web server like a production web server but we're going to use it for our application so now that i have that installed in day 28 i'm going to create several folders here first is the src folder next is the actual flask project name whatever i want to call it in my case i'll call it cfe os as in it's a very simple application that is going to be cross-platform inside of src i'm also going to make a wsgi dot pi file and then i'm also going to make one last thing which is going to be data okay so any sort of relevant data or things that i might need to package for my project are going to go in that folder so first things first let's go ahead and make our flask application and inside of here i'm going to make two files first is an init file to turn it into a module that python knows about next i'm going to go ahead and create main dot pi and then inside it in it i'll just do from main or rather from dot main import all okay so now instead of main i'll just go ahead and do from flask import flask no surprise there probably and i'm also going to import the path lib so import path lib which is a fantastic alternative to using the os lib which is also great but path lib is better so i'm going to go ahead and give my base directory being equal to pathlib dot path that's capital p path of this file and then we can go ahead and resolve this and then get the parent element i will actually return this base directory from flask so we can see it so let's go ahead and create our flask app and i'm just going to call this web underscore app is equal to flask and we use underscore underscore name this is very very common with flask on how you initialize a flask application and then we will create our index route by doing at webapp dot route home page there and then the available methods hopefully your following along here if you're not i will explain some of this in just a moment and we'll call this index and i'll just return that our you know dur is this string of base stir and then i'll give a 200 response okay so if you're not familiar with flask the general idea is we create a flask application and then we add a route what this is going to do is it's going to allow us to have a http site going at our local host with some sort of port let's say 5000 and then on that home page right so if we change this route to like abc this would be abc now i'm not going to go into flask too much here other than some basic things like that because that's not really the purpose of this but that's just generally speaking how flash works so this is a incredibly minimal flask application so now that we've got that we can create one more aspect of this flask app and that's our wsgi file so inside of here what we'll do is we'll import waitress so we'll go ahead and do from waitress import serve i'm also going to import from cfe os we're going to import our web app okay so cfeos is this folder right here waitress of course is what we installed and the reason it can grab that web app is because init is importing everything from main.pi which has the web app itself okay so then we can serve this with waitress with underscore underscore name equals to underscore underscore main and then we just do serve and this is going to be our web app web underscore app brother and then we can set the host which in this case i'm going to actually declare 0.0.1 just like that so 127.0 0.0.1 that's a very common place for our host to be and i'm going to set the port that i want to use in this case i'll just go ahead and say 5002 and then we'll give the number of threads we want to use definitely check out their documentation here i'm just gonna go ahead and use two threads their documentation uh recommends how you're gonna wanna thread this based on the system that you're working with two threads even one thread is probably fine but we're just gonna go ahead and stick with two threads so this will actually serve our web application so let's go ahead and just give this a shot it's really simple so i'm still in the root of this project or day 28 and i just want to run this wsgi file it's really simple we just do python src slash ws gi dot pi hit enter and i get unknown argument port oops that should be a lower case port not uppercase try that again and now as we see it's serving on localhost 5002 we can open that up and that gives me a absolute path to the root of that project right so the root of the flask application itself so it's going into this folder right so that's what that's what this base directory does so if we want to get the you know the data directory for example what we would do is say data dir equals to the baster dot parent so this goes up a level to src oops so the parent directory would be this src folder from cfeos and then we would do slash data okay so then we can actually grab this and call this the data and we just pass in a string the reason you want to turn it into a string is because you don't want a path live path object so we can cancel that server out with ctrl c run that all over again and refresh in here and oops um oh these just reorganized but there you go now we've got our data dir and our main project directory but of course this is actually not how we'll grab things from our data directory what we need to do is have relative paths because of pi installer so the actual source code itself is fine but actually where we store data is going to be a little bit different than this so let's go ahead and do that now there's a really good chance that your project will have other kinds of files that aren't python files so like images or videos or machine learning models those resources need to be loaded a little bit differently with pi installer applications so let's take a look at how to load it with just path lib and then we'll adjust it only slightly to load it where it's a little bit more robust and ready for pi installer so the data directory that i have is inside the src folder so it's outside of my primary project here and i just put an arbitrary image of beach.jpg just to illustrate this purpose so i'm going to go ahead and say img path is equal to that data and it's just main or rather beach dot jpeg okay and of course i can actually bring this into my response down here and i'm just going to bring in whether or not it exists so image path dot exists okay so that's a cool thing about path lab is you can just do dot exists to see if that's a real file so if we restart the server with control c press up and enter we can take a look and what we should get is a true value for that image path so what i want to do now is actually change this only a little bit to where it will take for account our py installer you know eventual data file so inside of cfeos i'm going to make a new file in here and call it resources.pi of course you don't have to call it that but to me calling it resources is telling me where these resources live so there's a couple things i want to import so first off i'm going to import pathlib and i'm also going to import sys and then we're going to define git resource path and this is going to take in a relative path now when i say relative path i'm going to say it's going to be something like this so relative path being relative to my entire resources so all of the resources for this project that i'll add into my build i need to know what that relative path is so my resources for this image that path will end up being data beach so i need to actually be able to load that single thing in and i can use pathlib here as well so it should be that string as is it should also be able to do pathlib.path something that you probably have seen before as well um but also relative path being like os path dot join data and beach dot jbeg and so i need all three of those things to be able to work for this relative path so it's actually not really that hard to do first off we are just going to re declare the relative path as rel path and i'll just say pathlib.path and then passing in that relative path now do not put resolve here resolve is going to attempt to find where that actually exists again there's a relative path so just making a path lib um actually path then it will be treated the way we need it to the next thing is where data is going to live now in development the development base well it's actually identical to this right here right so this base directory is where src is it's identical to that except one more thing okay so we actually want to have just dot parent as well because if you remember also back in main we have base der dot parent so this base directory is actually the root of the flask project itself and then the data dir goes one directory higher than that so our actual base path is that directory so i call this the dev version because what py installer adds to this whole system is an attribute added to our sys package and that attribute is underscore m-e-i-p-s-s this will actually give us the relative path when we build our project so my actual final path will be base path equals to s or get attribute from s-i-s-y-s from sys or the default will be that dev base path and then what we're going to return is the base path and the relative path okay so you know if this attribute's not there it's going to default to whatever that is and of course this is our dev path so this should actually work once we build our pi installer application but it will also work now with our development environment so now what we can do is actually use this on main.pi so let's come back in here and do from.resources import this git resource path and now my data directory i'm just going to literally pass in get resource path for that data directory now you still might need this base directory for relative route of the flash project for so many other things but now that i've got this get resource path it shouldn't have actually changed the value of it especially if you didn't change this to resolving right you definitely want to resolve the file itself and getting two parent levels up because of where resource up high is now if dot pi was in another directory lower you would absolutely need to adjust for that in here so let's go ahead and restart this run it and refresh and it literally the exact same data so again if we actually change this to being resolve and rerun that we now get false in a completely different location um so yeah you definitely want to make sure you take account for that all right so here are the build scripts that we're going to be using for our operating system now of course on the left we have mac and linux and on the right we have windows the general rule of thumb is if you need to run it on that operating system build it on that operating system so run it on windows build it on windows run on a mac build on mac run on linux build on linux now you can actually build it on linux using docker as well so you don't actually have to have the entire linux operating system to build up for linux you can just use docker and one caveat for mac users is there's actually a couple binaries that we need to include here which are the tk and tcl frameworks we just include them like that now this reference will be on our github and if for some reason these aren't installed you can either install it using homebrew or using xcode and just install from there so the next thing is we activate it we call py installer and then we've got our entry file so this entry file or entry point is the python file that we are going to be primarily importing items from which we'll look at in a second and then dash f actually turns it into a single file one single file execution so if we look at that entry point which is wsgi.pi we have a number of other imports in here right so we have the cfeos module as well as the waitress package that we installed with pip so those things are already being imported pi installer actually looks for the things that are imported and will attempt to package those into the build script as well right so that's going to do it automatically but in that same vein you can actually add in some hidden import items here as well and i've found that every once in a while you add it in as a hidden import and a lot of times it actually will for sure do it where sometimes if you don't have it as a hidden import you just assume that it's being imported you might actually run into some issues so so when in doubt just add it in as a hidden import you shouldn't run into additional issues that i'm aware of so that's true on both of these right next line is the name of it so if we look at both of these of course the names are only slightly different right and you can call it whatever you'd like to call it cfe os windows or dash mac just kind of makes sense for these environments one i'm going to build for mac one i would build for windows the difference one of the differences is actually how we add in data it's a string here the first half of this string before the colon on mac has the relative location to where this build script is and then where its destination should be so it definitely has a colon between those two i'll discuss that in a second windows users very similar but it has a semicolon instead it's like why doesn't it do the exact same thing i don't know but that's how it works and also you want to actually use the paths that are system specific right either forward slash or backslash depending um and then of course that hit an input and then the last thing is this clean call so what clean is going to do is actually just remove the old builds and then add it in so let's say for instance you actually wanted to add another directory you wanted to use another directory well it's actually really simple you would just use add data again and then image like let's say images and do something like that okay so you would do the same thing on windows as well now i don't actually need to do that here but that's just the general rule of thumb but if you go a little bit further you can actually get specific file types and this specificity is probably a good idea when in doubt like what are the files that you for sure want to bring over just add dot jpeg star.jpg or star.mp3 or mov or pickle or whatever it is that you want to bring over and then the destination directory now when i say destination directory i think the directory that holds these items should be the same as the destination so if you're using data you would want to grab that one and if you're using subdirectories you would also want to call that so say for instance we did data you know images jpeg i would just then do data images like that okay and all these things are going to be worth testing out on your own as well to make sure that that data is actually coming through and we'll see what that what happens with that now do realize that this src in front here is going to be relative to my build script which is outside of where src is if it was inside of src we would have to change these things ever so slightly but i like keeping it outside of the entire project itself because i think it makes a little bit more sense to do so because of where we want our final project to be built to so without further ado let's actually run this command so i'm going to go ahead and do chomod build.sh this of course or rather chomod plus x build.sh this of course is just for the mac and linux users if you're on windows you can just run dot slash build.ps1 that's new windows users for mac and linux build.sh so by now it should be built it actually won't take that long because this is not that big of an app and then what we see is this dist or distribution folder that will have our actual executable so if i close out the one that was running and do dot slash dist slash cfe os mac or dash windows if you're on windows notice that it's serving this file and now when i refresh i've got my image path being true it shows me a brand new data directory that is not relative to this code whatsoever right so that's a key distinction with those resource paths that we had this is now in a brand new location but yet it's still actually getting that file which is denoted by just knowing that it's there it's like actually finding it um which is actually really important to do but as it stands right now i could actually share this file if you have a mac you could run it directly on your mac you might have to give permissions to run it but you absolutely could and notice that it's only 7.7 megabytes so that's i mean it's kind of big i guess but it's also not that big considering that python's everything we needed for python is in there flask is in there and then also waitress so it's got a lot of capabilities for only 7.7 megabit megabytes which i think is pretty cool uh windows it might be a little bit different of a build size but that's not that big of a deal one other thing you might want to do is actually add a icon to this executable so you can say icon equals to icon.icns that of course is for mac if you're on linux it's i believe it's a png file if you're on windows it is an ico file and ico and then with the tick there okay so what we've got here then with these icon files you want to actually put them in the root of your entire project directory where the virtual environment is as well as the build script i put them there because i've had the best luck putting them right next to the build file you probably could put it into src or in the data but i really like putting it next to wherever i'm building it because it just seems to work better across all systems and create an icns file or ico file just google how to do that it's actually pretty easy to do um so now that we've got this i will say that there are a number of caveats with building this the first thing is actually using a data directory or data file type that just doesn't exist so if you tried to build this now you should actually run in to some errors now even if i don't see an error it's going to happen from time to time that if you're using an invalid extension or you're using something else like an invalid folder that's actually not there you will get some issues like this right here right unable to find star.abc like i literally do not have any files inside of data that has the abc extension um but that would also be true with a folder that's not there and in this case it actually didn't even build the project at all so that's a big one the other thing is that the hidden imports now i have waitress in the hidden import even though it's not a hidden import it's an import that is very clearly there now for me when in doubt i add the package i need in this hidden import now this hidden import is really good when it's a package that's being used in your project but maybe not on a top level package like if it's a dependency to one of your packages using hidden imports is often a very good thing but luckily when you actually build it when you go to run and build it it will often give you pretty good errors related to not being able to find a package but then of course when you actually run the application it will also tell you hey cannot find whatever that package name is and of course this is actually one of the challenge of pi installer is some packages just simply can't be packaged up inside of pi installer just doesn't work i've had a number of issues with some asynchronous packages i've had a number of issues with machine learning packages um but just generally speaking it does work really well there is overall a an amazing amount of support to package all kinds of pi pi packages or python packages in general one of the things i've also found that if it's not packaging up very well you could always bring that package into your src folder in other words go in to the virtual environment or rather into lib and finding whatever that package is and dragging all of that python file into your actual src folder and then modifying it as you might need now just generally speaking you don't want to do that you want to use the virtual environment and you want to use pip install to do all those things but if you need to use the site package and make changes so it does package up and work with pi installer just make note of that and then also make sure that inside of your src that you uninstall the you know site package version of it i've had some great effect with that as well so now that you have a general idea of how to package a project in pi installer the biggest question is where do we use these things the biggest question is when do you actually use this kind of packaging now for me personally it's when you bundle it into a desktop application of some kind or a graphical user interface so if you have an electron application and you need to run some python code specifically this is when you actually go about doing it you build pi installer to do that the other part is i can actually send this to anyone they can double click and open it what that's going to do is open up terminal for them not that they would know exactly what that is but then it would actually run that server right so if they have other applications that need to use what's in that server they would now have it right so then i could just navigate to whatever that location is and there you go there's that application running again but as far as like actually using this to deploy on a web server well you totally could do that but i actually wouldn't recommend that being the best option it's a much better option to control the operating system environment because of all of the potential problems with building and running the pi installer app on various systems right so like i don't want to have to rebuild this app every single time i'm going to deploy it that might take a lot of time and the app might start to get really big further some of this data you know i don't necessarily need to repackage and send to a web server every single time you could just either leave it on the web server or leave it somewhere else that that brings it in and then leaving it like this as far as using pi installer is just not ideal for that and then there's something called docker that actually bundles the operating system level even better than this and just essentially runs the code so pi installer is great for a lot of these like smaller applications now if you're a network systems engineer you could probably use pi installer to distribute small little python applications or even big ones but python applications that you want to run across a bunch of different servers and pi installer would be a really easy way and unique and consistent way to do that without too much overhead right so it actually bundles the app pretty well into a fairly small app but to me it's actually just putting it into a desktop application or gui application so that i can with reasonable certainty know that the python part of that gui will absolutely run correctly every single time now the biggest application to me is actually running any sort of machine learning application that i don't want to re-translate into something different right there are ways to change a machine learning model into other kinds of executable code but it adds another layer of complexity which pi installer even if it is a little complex does definitely take away some of that stuff so anyways that's it for pine installer please let me know if you have any questions on this and let me know what you end up building this was day 28 so thanks for watching and i'll see you next time

Channel: CodingEntrepreneurs

Views: 12,223

Rating: 4.9327731 out of 5

Keywords: djangourlshortcfe2018, install django with pip, virtualenv, Django Web Framework (Software), Mac OS (Operating System), Python (Software), web application development, installing django on mac, pip, django, beginners tutorial, trydjango2017, install python, python3.8, django3.0, python django, web frameworks, install python windows, windows python, mac python, install python mac, install python linux, pipenv, virtual environments, 30daysofpython, beginner python, python tutorial

Id: 3xuN9JQ7j1Q

Length: 32min 42sec (1962 seconds)

Published: Mon Sep 07 2020