How to Build Python Packages for Pip

okay so we're going to work through actually building a python package that we can install through pip so when we're installing a package through pip we're actually installing a package from this website here the python package index so if we just search for something we all know pandas we'll see here that we actually have all these different pandas projects now the reason there's so many of these is because loads of people upload projects that say pandas in the name as you can see the one that we use and we know is this pandas 1.2.3 and we head on here and we can actually see all of the information about that package now what we want to do is actually build our own package and upload it to this so that we can pip install it and build it in a way that other people can use it without any issues so to start i am going to use this package that i put together quickly for this and what it does is it simply generates a ascii visual based on a few images i've included in here so these synthwave visuals and all this package does is you initialize it and you press generate and it will generate ascii versions of these pictures at random so it's pretty useless but it's a good example now the first thing we want to look at here is the directory structure so we have this aesthetic ascii directory which is like our project directory we can name this as we want and then inside here we actually have the package directory so this is where we put all of our code and in here we have this init python file now this is very important we need to have these files in order for our directories to be recognized as python modules or python packages and then inside here we also have the synthesize pi file and this is where all the code is going for me and this creates a module within the package that we can import now you notice here my init file is empty and this is something you'll see quite a lot it's just there to tell our package builder which directories are parts of our python package or python sub packages and then of course we have the readme file here so we can summarize this as this we have on our top level we have the project directory and then we have the readme file and then we have our code directory inside there we need to have the init file and then we in this example we have two modules we have module a module b now when it comes to actually importing one of these modules and using it in our code what we would do is we'd write something like this so we'd have from and in this case it's called code directory obviously we wouldn't call our package code directory unless you really want to and then we would import a for the a module or b for the b module so in our case this is going to be replaced with ascetic ascii and this is going to be replaced with synthesize and we'll come to actually testing out later but for now let's just work through the other files that we need in our directory i don't want to focus on the code so much here so we have our readme file and we have our code directory the other files that we need are all configurations subfiles and the most important of those for setting up our package configuration is the setup config file so that looks like this and if we go ahead and actually open this file so we need to create it here so setup dot config and there are a few parameters in here that we'll need to add now for this project we will be using these so we have metadata which is just the description or the data explaining what our package is actually doing so this sort of thing will be displayed on the python package index and people will be able to see the name of it the version the author they'll be able to get in touch with you through your email and they'll also find a link to your website so for me obviously i'm just using git up here and all of those things are pretty self-explanatory but there are a few that i think are a bit more abstract so first we have the long description and you can write something here but it's a long description so i find it's better to include a file and you include your readme file okay and then this will display on the front page of your python package index site and then we also need to explain that our long description is a markdown text file so that it is processed correctly next we have our classifiers and these are kind of like types so we're specifying the programming language that we're using it's python it's python 3. we are outlining the license that we're using and for that we have this oc approved and we're using an mit license which we'll cover pretty soon and i want to specify the operating system and for us it's independent you don't have to have a specific operating system to run this so that's just the metadata behind our package and then down here we have some of the more important things for actually building a package so here we're specifying the packages that will be included as dependencies and we use this find here and what this would do it will go through our code here and find any package dependencies that we have and this is provided by cell tools which is what we're using and i'll explain that in a moment we also want to specify that we're using python and we require either 3.7 or greater and the reason that we need 3.7 or later is because we're using a package that came with python 3.7 in order to include all of our resources over here so that's our dependency on that version and then here we're including package data which again we are including because we have all these extra files that are not code files that we need to have included within our package and that's why we've added that in there so that's our setup config file and alongside that we also need a pi project dot tumble file and i mentioned before in the setup config that we're using setup tools and in order to tell our package builder which we'll use later to use setup tools in the building process we need to specify this pi project dot toml file so we'll create that file here pi project dot tumble and this is the code that we'll need for it so we're saying here for the build system it will require setup tools which is the library we use to actually set up and process the cell dot config file and then python wheel and then here we're just specifying the build process that we'll be using from sub tool just build meta okay and that's that's all we need and that is just read by our builder and it tells it how to build our package which essentially tells it to use setup tools and process this and follow the instructions that this file is giving you and next up is our license now the license is quite important and it's very important that we actually include a license file because if we don't include a license file we can get into a lot of issues in terms of people not knowing if they're actually allowed to use our code or not and especially if you have people contributing to your project if they contribute and you still don't have a license file then it's quite odd because legally you're in a sort of gray zone as to whether you can actually use the package that you built so a license is very important to include but it's very very easy to actually set that up so all we need to actually do is we go on to this website called choose a license.com and we just select what we want for our license so for me i want it to be simple and permissive so anyone can use it and it gives you a license here to just copy and paste across so i'm using the mit license go back into project and we just create a new file and it's called license like that and we just copy that across change list so it's 2021 james brooks and that's the license sorted so it's pretty easy now this is all great and we have our code but there is one modification to our code we need to make in order for this to work so in our code at the moment we rely on reading these images which are in the same folder at the moment now of course we could have put these in a different folder we probably would normally but when we are building a package we need any resources that we want to include in that package to be contained within a package directory which includes a init pi file so that's why they're all included in here now we have a few images and we also have this font file which is a true type font and what we do down here usually is that we use the pill library and we just open the image like this so we're selecting one at random so it's one of these at random and we're just opening that and it's pretty straightforward now when we build a package this won't work anymore because this is a relative path so you will almost certainly get issues so what we instead need to do is include these files within our package so we've already done part of that we've included them in this init file but there is one more step we need to include a another file here which is called manifest dot in and this manifest dot in file needs to include any resources that are not code files that we would like to include within our package so to do that we need to write include and then here we give the name of our python package or sub package so it's aesthetic ascii and this is referring to this directory name here and inside there we want to include all so it's wild card for all files that end with png which is our image files now you will notice that we also have some jpeg files as well so we'll just copy that and we'll add jpeg as well now that's our image files but we also have our truetype font file so we add that in as well and that's our manifest file now back in the config we know that we've included the manifest config file through this include package data at the bottom so if this was false this would not be read in and our actual resources would not be included with it being true it knows to expect a manifest file and then it reads this and includes these within our package so that's great and then in our code we need to use a different approach to actually opening these images or other files and to actually do that we need two different libraries in this case we need import lib and from import lib we need to import resources and alongside import library sources we also need to import the i o library so this import lib resources specifically for reading in resources or files that we've included within a package the i o library is for converting the bytes that we will read when we import the resources into readable file like objects because the code that we use down here reads our images and font from file so this allows us to pretend that the bytes that we've read into our python code is actually a file rather than a set of bytes so with that this code here this line instead becomes this which is a little more complicated but this is just what we have to use so here we're using the resources that we imported and we're using open binary it's in the ascetic ascii package so we need to include that that is the directory name here and then this time we're using the same thing again we're just using the image and we're selecting one at random and then we're just importing or reading that like with any other file and then when we use this we would expect a file path to be passed to this function so we imitate a file path by using this i o bytes i o function and passing the image binary to that now that's our images that's sorted and then we also have our font down here as well okay so at the moment what we have we have our font path and this is just reading from file so we've got this image font true type and then we're reading it in there now we're going to be using the same thing again we're going to be using the open binary function so we do with resources open binary again we include our aesthetic ascii package name and we also include the font path which we've just defined there and we again read that in just like we did before so we'll call it fun we do fp.read and then we need to initialize that font and what we were doing here was reading from path before so again we need to use the i o library to imitate that so we do i o bytes i o and font and with that all of that will now work so that's everything set up and what we do once we have it all set up is we first test like works locally so the first thing we need to do is navigate to that directory so for me it's in documents projects test aesthetic ascii and once you're in there we can just do pip install dot and this will do a local pip install from the current directory okay so you can see that that worked that's great and then here you'll probably want to just test your new imported package so for example for me i would go from aesthetic ascii import synthesize and then i would go through the code for actually testing i'm not gonna do it here but as long as that all works well then we should be good to go so at this point we do python module build and this package this module will actually build our package and what you will find in the directory here is that we'll get a new directory called dist which means distribution and this will contain two files so we can have a look at the moment we have the tar file and now we have the wheel file as well now these are two files that are going to be uploaded to the python packaging index and they are our package so after we've done that we need to upload our code the first thing we do before we upload it straight to the python packaging index is we upload it to the test version of python packaging index just to make sure that everything is working and we find that over at test pipey.org and of course you will need to register here so you go to register enter everything you need you will need your username and password when you're running through the upload process in your prompt window so make sure you have that set up and we'll also need to pip install twine which is a package that will allow us to upload everything to python packaging index now i already have it installed so i won't go ahead and do that instead what i'll do is python module twine upload repository test python packaging index and here we need to pass everything within our distribution folder and we run that and everything will be uploaded to our python packaging index once we've done that we want to actually install our package but from the test python packaging index so to do that first because we already installed our package we actually want to uninstall it and just uninstall that and then we can install it again but through the test package index so to do that we just pip install and then we set the install index to the test python package index so we just do test pipey.org so simple and then we specify our actual package name which is pathetic ascii and we should find that that will actually run and install and then at this point this is where we want to test again now if you are using resources and everything with your package that is something to definitely check that everything is being read correctly and it's just functioning now once you're happy that everything is working correctly this is where we finally upload it to the actual python packaging index now this is the same process again so do python module using twine we using upload and we also type repository now this time instead of writing test pipey we write pipe and again we use our distribution we execute that it'll ask you to log in and then we're done and we'll be able to see our python package over here so we'll be able to search for it or you can just come over here to if you've logged in click on your projects and you'll find here we have our actual package now i can view that and it will come up with the readme file on the side here we have all of our metadata so we have the mit license the author and that includes a email link there as well which python version it requires the language and so on we can also click on the home page which will take us through to the github repository so that's all set up as we would want it to which is pretty cool and most importantly we can actually just pip install it and start using it so first i'll just pick uninstall and now all we need to do is pip install our package and it's that easy so that's everything that you need to know for actually building your own python package it's a really cool thing to be able to do and i would definitely recommend learning how to do this and building your own packages so thank you very much for watching and i will see you again in the next one

Channel: James Briggs

Views: 42,366

Keywords:

Id: JkeNVaiUq_c

Length: 21min 7sec (1267 seconds)

Published: Fri Apr 02 2021