python packaging: basic setup.py and declarative metadata (intermediate) anthony explains #057

hello and welcome to another video in this one i'm going to cover a very very basic python package and set up you know python packaging metadata and then i'm going to show you declarative metadata and why i think it's better so let's just jump into that so to start we're going to be making a small python package that we're going to distribute i'm not going to be using the source layout but i will link the source layout video below in the description um but we're just going to start with setup.pi i know you know pi project up tallmall you should use the brand new fancy stuff but we're just going to use setup.pi for today uh yeah let's make our package first uh i'm just going to call it i don't know hello world or something like that so maker hello world this will be the namespace that we distribute and we're going to give it an init.pie so the traditional actual python package there's an unfortunate thing about python in that package means kind of two different things uh one of them is slightly more correct like a python package is a folder that contains an init.pi file but a lot of people including myself because you know i'm wrong and it's easy uh refer to things that get distributed on pipi as packages although a more correct term is probably distribution or something like that but whatever i don't actually care um we're also going to add a um a little main script here that's just going to if name equals main exit main and if you know if i was making a package that's actually this simple i would probably just do it in a single file uh but i wanted to make a uh actual python package that you know demonstrates this dealio and yeah i think that's enough so let's jump into setup.pie i have a little cheat sheet so i don't forget everything so i'm just opening that up off screen so we're gonna start with the most basic packaging metadata first and we're gonna be using setup tools for this setup tools um is kind of ubiquitous for creating packages now you used to do this fun little dancy thing with dish utils.core and some other stuff but uh for the most part dish utils is you know not happening anymore oh i forgot to turn off my keyboard cam oh well you'll see my keyboard game for this from zebra tools import setup um and we will call the setup function and this is where we pass in the metadata for our package and setup is essentially just side effects that package those things up and we'll start with name uh we'll just call our thing hello world and we'll give it a version uh i usually i usually uh you know be a little uh skittish and start with version 0.0.0 but it's usually a good idea to just start with version one just like get get on your semantic versioning from the beginning i know semantic version that's a whole different thing that people are up in arms about you can use whatever versioning you want but we're going to assume we're using some amount of magic versioning here um description is a good one to set this package contains some sample hello world code i don't know something like that uh what else do we need we need author so author would be me anthony sitili and there's usually author email as well i'll add my email address that i use for open source what else do we have oh we also want a url um i'm not actually going to push this to github but if i were it would be github.com acetylene hello world what else do we have we have install requires uh which we don't actually have any dependencies any runtime dependencies for this but you would fill them out here you can just leave it in an empty list although i usually prefer to just delete this line if it's the default value because what's the point and the next thing we need to do is list the actual python packages we're going to be including but we're going to be using a special little tool that does this for us and that is the find packages helper from setup tools from setup tools import find packages now what find packages will do is it'll iterate over the file system here and it'll look for things that have init.pi files and add those to the packaging metadata automatically now normally you could just you know list them manually packages equals hello world and this would work just fine however as your package grows it's very easy to forget to add new things to this and i can never remember whether it's you know hello world dot foo or is it hello world slash food i just you know can't be bothered oops that was not what i intend to do let's get this back here okay uh but yeah i can never remember what the format is for packages and it's very error-prone to try and synchronize it with the file system manually so there's this find packages helper find packages and oops and by default you can just call it like this just find packages print print i usually have a directory for tests and have some test code in there test foo dot buy one tests has ended up high sometimes and often i will also have a testing directory [Music] which has like helpers in it testing and that'll also have an inner dot pie um what the heck happened there so often i will have other name spaces in my source code directory that i don't actually want to package and unfortunately via then python import setup tools if we do set up tools.find packages it's actually going to find the tests and testing directories but we don't actually want to distribute those namespaces to people that install our packages because otherwise they'll just get this you know garbage testing namespace that happens to be the helpers for my um for my package so we're actually going to do is we're going to exclude those and we'll pass the exclude here and you can just do test star and testing star i like to do a tuple here it doesn't actually matter if we do test star testing star you'll see that now we only get that package that we expected here we're no longer including tests and testing so that's packages and the last thing that we want to do is add uh and there's a whole bunch of other metadata here but we're i'm just doing kind of the basics here the last thing that i want to add is a entry point and this is going to make it so that we can run the hello world as a program directly without having to run it as a module and the way you do that is with console scripts console scripts and you give the executable a name so maybe hello world cly or something like that and you pass it a dotted module path with the attribute at the end so in our case it was what helloworld.main yes hello underscore world dot main and then the function inside of this model is called main and this is our basic packaging metadata let's actually install it and see what it does marshall and vm we do pip install dot that will install our package so you can see now if we do pip trees we'll see okay that was not what i expected uh okay i guess pip has changed their pip freeze output to something that oh no i expected to do hello world equals equals one but apparently they have changed some stuff but that's fine whatever um maybe there's like a pit freeze legacy or something i don't know anyway it doesn't matter anyway we've installed our package and now we can do hello world clay and you'll see that it prints hello world and this actually ran our program okay so that's like the very basics of setup tools metadata now i'm going to convince you that you should never do this hopefully uh and that you should instead use a declarative setup um and i'll show you two tools that you can use to make your declarative setup easier to manage the first of those is if you have a classic setup.pi based distribution you can use a tool that i wrote called setup pi upgrade open that up and show you guys the readme of that github.com setup pi upgrade spoilers it just showed the other tool we're going to talk about but yeah it takes the setup.pi and automatically upgrades it to declarative metadata what declarative metadata is is instead of having code in setup.pi code inside of pi so instead of having code here i mean you still have to keep these two lines in this case there are other cases where you don't need this but we won't go into those but instead of having code you instead store your metadata as data similar to this and the nice thing about data versus code is it's really easy to rewrite data there are tools that will load this into you know dictionary like objects and you can modify them and then write them back to disk whereas with your classic setup.pi example which i can just grab the one that i was looking off over here with this it's really hard to modify these values automatically because you need to do a code rewrite and rewriting code is not so simple in python there are tools for it but i don't know i haven't found one that i like so far but yeah difficult to rewrite this automatically whereas metadata is really easy to rewrite so i wrote this tool called setup pi upgrade and you basically just pass it a directory and it will upgrade your setup.pi automatically so if we take this and we tip install setup pi upgrade and then we run this set up pi upgrade dot you'll see that it has rewritten setup.pi and setup.cfg and you can see that it automatically migrated all of that data that all that metadata into this metadata format so you can see like we got metadata with all of these fields so we got this find packages which is now this specialized find colon thing and you can see that the exclude argument uh was was sent here and we also got our command line down here as well and setup.pi just becomes these very very simple two lines and actually you can leave this out in some cases um if you're using pi project for instance but we're not using bi project so you still need this little file um so that's set up pi upgrade there's another tool that i wrote called setup cfg format install setup cfg format and this is a code formatter that takes your setup.cfg and kind of normalizes the output it also adds some fields that are helpful so if we run setup oops actually activate the virtual setup cfg format on setup.pi uh setup.cfg yeah not on 7.5 it's like what what happened um where did it actually change uh i don't know what it changed i think it reordered some stuff um but it also does some stuff like if you have a talks.ini which has a test of lists you know let's say you're testing against python 3.7 set up cfg format i thought knew how to do this test as why do i not remember this option anyway there's there is some option where it'll automatically format out all of your classifiers also if you have a license file workspace we'll just copy a license here um oh was i calling it wrong no i called it right uh but you can see here like it will automatically add license information if you have a license sitting around and it does some magical detection there but anyway that's uh that's set up done by kind of a crash course as well as declarative metadata uh i have switched all of my packages to using declarative metadata because i think that it's much better than the code based approach but hopefully you guys found this interesting i will link both of those tools in the description so you can check them out if you guys have additional stuff you want me to talk about leave a comment below or reach out to me on my various platforms but thank you for watching and i'll see you guys in the next one you

Channel: anthonywritescode

Views: 10,944

Rating: 4.7333331 out of 5

Keywords:

Id: GaWs-LenLYE

Length: 12min 35sec (755 seconds)

Published: Thu Jun 25 2020