Writing Awesome Command-Line Programs in Python

Oh so our next speaker is Mark Smith who's a develop train of a fan duel in Edinburgh and apart from that mark also runs Edinburgh Python user group and his talk is writing awesome command-line programs in Python hi thanks very much for coming to my talk I'm going to spend the next 40 minutes or so talking to you about the things you can do to take your online programs to the next level and some of the modules there in the standard library and they're available on the net to help you do that so just quickly to talk about my favorite subject me my name is Mark Smith I work for Fanjul and we're hiring we're looking for smart people so if you're interested that come and talk to me afterwards and I'm pretty much called Judy on my social networks for reasons that aren't as interesting as they sound and so that I wouldn't normally put the word awesome in a talk title but this talk is vaguely based on a book that I read about this time last year called build or some command line applications in Ruby so I was stuck building a command line application in Ruby and I'm not a ruby developer so I kind of need to look around and I came across this book it really is excellent so if you're stuck in the same situation I highly recommend it but what really became apparent to me while I was reading it was that a lot of the libraries that are out there in Ruby there are similar libraries and Python pythons just as good if not better for writing this kind of thing so that's really what led to this talk so let's just talk quickly about why you might want to write command line programs because I think this is the only talk on the topic at this conference so it seems like kind of minor concern and yet I'd be surprised if there are any developers at this conference who don't write command line programs regularly to simplify their job I mean we're developers we like writing code it can't all be web applications so I would say the reason to write a command line application is that in many cases a command line application is the best application for the job we have a habit of thinking of them as inferior to give applications inferior to web applications and I would argue that's not necessarily there are definitely some applications that benefit from windows and buttons and menus but as developers we all know that we can automate processes programs are run by programs and if we have something like bash or even other scripting languages other programming languages we can glue applications together in really powerful ways and besides it's much easier to write command-line application than a full web application so why would we want to spend any time writing a good command-line application why not just chuck a main function in our script and be done with it and that's because that I would say that you want to do that you want to think about this do this properly because scripts are never throw away they never get thrown away you write them you think you're going to run them once and then you need to run them again in a week's time then you need to read the source code to work out whether it really did what you want might need to make some modifications so I'd so if you start from the beginning with the idea that this is a reusable application then when somebody comes up to you and says hey you've got that cool script that does something I need then it's already documented they can understand the output they don't need to actually modify the file to it to run it so let's just talk quickly about what makes up a command-line application so I suspect most people just kind of fell into using and then writing command line applications without really thinking about what makes one and more importantly what makes it good easy to use reusable command line application so this is your program it's just a program it's just a bundle of functions maybe some classes this is the thing that actually does the useful stuff but without an interface it's not any use so let's start sticking in command line interface on top of it so the first thing you probably think of are the command line arguments there are flags or things that begin with minus or minus minus and data that you pass on the command line to run the application so I've been talking to people about the topic of my talk show in a week and I've been getting this this look as if to say you're doing a talk on Arg pass a 40-minute talk on our bus but there's a lot more to it than that we've got standard in standard out standard error and it kind of looks like we finished but we haven't finished we've got signals so I can I have a raise of hands of anybody who's actually caught a signal in a command line application oh wow that's way more than I expected okay I'm not going to go into great detail on signals what I'm going to kind of skim over that because I got far too many slides and and then we got configuration so configuration is not specific to command line applications obviously we have that GUI applications as well but I think it's more important so in GUI applications you tend not to the user tends not to see the configuration it tends to be hidden behind a preferences pane or something like that whereas with a command line application if you're providing configuration your users probably going to have to write that they're going to have to know where to find it they're going to have to know how to modify it and so the decisions you make and more important and then finally we've got the exit code which is a number that's returned when the program's finished to say whether it whether it did with supposed to or whether it encountered an error so I'll start with command-line parsing there's a bunch of options there are three in the standard library and you'll find that most people who aren't experienced Python developers pick get opt partly because actually it's not alphabetically at the beginning anymore of course is the beginning it's it's the first one they see and they've probably written some C or they've encountered essentially the same interface in other languages and so they just kind of go oh that's that's what I'm looking for but get ops horrible it's really horrible to program it's not really pythonic and it doesn't validate your arguments very well and I'm going to talk about opt past a lot because it's really been superseded by our boss arc Pass offers all the same functionality as far as I'm aware plus some extra functionality so I'm going to talk about our past briefly in the third-party domain we have a few options as well doc opt is interesting because it takes the opposite approach to anything else I've ever used doc ock do you actually write your help text so the kind of textured expect to be printed out when you ran running your script with - - help and then da Copts will pass that work out what arguments York your program should expect and then it builds you an object that will pass those command-line arguments so it's kind of clever but having used it it's kind of fiddly to to get the syntax right when you're writing the help and I found it just actually took a lot longer and it wasn't really easier in the end but I quite liked the idea plus it's cross-platform so it's been implemented in more than one language so if you if you for some reason they're writing a script that is implemented multiple languages that may well be a good option for you Clint is much more than a command-line parser Clint is a project by Kenneth writes and it's a whole framework for building command-line application so I highly recommend it okay it covers a whole cross-section of stuff and we're going to cover in this talk I admit that the one thing I don't like about it particularly is the command line parser but but it's a good option if you're already using Clinton and you're probably going to want to use the command line parser that comes with it then is click I been creating a bit of a buzz recently it was released by Armen ronica a couple of months ago I haven't really used it it looks pretty powerful Armin's very good at these kinds of libraries and I believe he solved a lot of localization encoding issues so if you're having if you're going to be running under lots of different locales then click is probably quite a good option for you and then finally we have Kampf ago which I don't think is a hugely popular library but I like the approach that it took and I think it's it's simpler for certain things than arquebus I'm going to quickly cover that with one slide after after talking about other parts so I'm sure we've all used odd paths in this room so I'm not going to spend a lot time on this but you instantiate an argument parser and then you tell about all the arguments you expect some of them I'll be compulsory some will be optional and then when you finish doing that you actually call parse augs that will go talk to sis ugly extract the arguments work out which flags correspond to which values and then hand you back a namespace object which contains the extracted values if it fails if somebody's passing invalid value it'll it'll essentially it'll print out the help text and finish with antara code of one so here we've got an option optional variable called optional parameter called name that's what the NRS question mark means and so if it's not provided I'm just replacing with word well this is just a silly HelloWorld script and then it prints hello and the value of name and that's what happens when you run it and so because it's optional we can run it without it uses weld we can run it with Douglas and it'll print hello Douglas sorry skipped ahead so we get the same output as we would if we just extracted that single variable from from our clean so you might say why we bother it's it turns out that that's that was only one line more than the code that just extracted it from the first item of ography but I you get this when you run - myself so although we didn't specify - my cell and we didn't actually provide any help text anywhere in there we didn't say what name was but instantly we can give the user a picture of how they run the application I think that's really powerful I now no longer write any scripts that don't use OGG parse or something similar simply because when somebody comes to up to me and says that script you gave me how exactly do I run if you've got documentation somewhere you can say run it with - - help it it'll tell you what you should be doing and then come back if that's if that's not enough so here's more complex example just covering some more of the details of arc pass at random so here with the first one it's we've decided there should be a boolean value there's no value associated with verbose really it's just on or off and so we say store true an outlaw store it's true if it's available false if it's not available and then the second argument number we've got this type parameter when we're passing an int so interest just a standard Python int type but you'll remember hopefully the int is also a function that takes a string and returns an integer value and that's actually all you can pass in any function like that for add argument under the type parameter so anything that any function that takes a string and returns the actual type you want you can pass in so then when your namespace will actually contain the types that you're after rather than you having to constantly convert things it's nicely reusable yes sorry so our boss also supports sub commands so if you're writing a much more complicated application without the the traditional unix single use specification then if you want to write something like git we've got where you've got sub commands like add push pull fetch remove sit-stay you can do this it tends to be very verbose it works very well and it's kind of worth doing you have to be quite careful about how you factor out all the parsing for the different commands that you don't just end up with one huge block of code in some ways it's configuration more than code so this is where comp code is actually quite good complicate aches a much more lightweight approach and in some ways it's not so powerful but if you're writing this kind of application you just want to kind of get it done quickly comp ago can be a really good option so I have no idea whether it's supposed to be pronounced comp ago by the way I don't really know what that means so here you instantiate an application that's essentially your parser and that provides you with the decorator and then you just use that decorator for any function you want to be an entry point into your application so in this case we've given our application a greet sub command which takes an optional parameter of two so it just passes this out of the function and the same with ungreased so I added another set another sub command now we've got we can call this program with ungrate and we can again we can choose to call it with a parameter or not if you don't provide a default value for the parameter then it becomes it's a compulsory parameter and you have to provide it if you're running the ungrate command and then when you run it it looks like this so it's not quite so nice it doesn't have positional parameters as far as I'm aware but it's really really lightweight it's really easy to write it doesn't take up a lot of lot of space in your code so if you're already using third-party libraries that complicates probably a good option so that was command line passing quickly well bin tour let's talk about input and output so i/o is often an afterthought it's just kind of you think oh yeah I'll dump some Princeton since my script but good i/o can really transform a command-line application from something that is just a hacky script and it it doesn't take a huge amount of work it just takes a little bit thought and possibly some libraries that are out there that can really add some power so just before we move on it's worth talking about standard out and standard error because I suspect most people don't really think about them standard out is for the output of your program it's so if you think of your program as being a pipe and it takes some input and it spits out some output that's kind of the data that you're producing you might is perfectly valid to have a script that doesn't produce output it's perfectly valid to have a script that doesn't take input but if your program produces data in some form standardout is where it should go standard error is probably the worst name it could be given as a streams that is standard error is not an error stream you can use it for errors but standard error is really for updates on how the program is running it's the kids it's kind of information on what's going on it's it's you can assume that the user is probably going to see standard error whereas if they're piping the output of the program to another program or a file they're not going to see that so it's they get no feedback on how the programs running so the way I think about it is that standard out can be piped pipe to a file or another program whereas standard error can be locked so at the least it goes to the screen where you can save it to a log file that's really with its full so let's talk about logging logging is written the right way it logs to standard standard error by default because it's ultimate an informational thing logging is a really powerful library I wish I had time to kind of go through it but this is an idiom that I use in command line programs quite a bit and again it's not a huge amount of code and it makes a big difference in the rest of your program in terms of making it easy to to provide informational information to your user so I use basically I call basic conflict to just set up logging I set it to warning by default so anything that's info or debug won't be spat out to the user the reason I do this is because program should be relatively quiet by default and then you can provide a flag to allow that allow the program to be more of a boast so that's what we've done slightly further down we've just checked if they want it if they want more information I've set it the two levels down to debug you could set it down to info it all depends on kind of how how you like to use logging levels is up to you and then they show the last four lines are just kind of how you then use that through your program but the nice thing here is that logging just takes care of how much information is spaced out to the user after that it it knows the level that should be spat up so it doesn't invade your actual code you can use logging as you would in any other kind of program and it will just do the right thing after after running this block so another thing that's worth thinking about that we often don't think about is are you talking to a user so I think we often assume when we're writing the program that actually standard out it's just going to go to a console and input is probably gonna be fed to us from a file I don't know which is slightly slightly weird that we have that that sort of different approach to the two streams is really easy to do in Python the main file types standard in standard out standard error all support is a TTY as a module and that just tells you that that stream is attached to some a screen somewhere so the thing you want to think about as a result of this is so it was just taking a picture it's so this is the result of running that program so if you run it on its own it just says not none of its pipes to anywhere it's just coming to the screen if you pipe it to another program in this case cat then it says the output is pipe to cap it obviously not standard standard error and finally we're just piping input and output so it's just just testing that that all works but the thing you really want to think about with this is you can change the format of your output based on whether it's going to a file or another program so if your information is being printed to the screen you'll may concern is that the user can read it so you can think about formatting stuff in aligned columns you can think of color and you can think about the type of information they want to see how they want to read it if you're piping to a file or another program you want something that's easily possible so you could switch your output to JSON and CSV it's something that's easy for another program to pass it becomes data rather than kinds of information so I just mentioned color color is also surprisingly easy and weirdly the Ruby guys do this all the time we don't tend to see so much color output from Python programs you can just use standee and standard ansi escape characters but i don't advise that i recommend use a third-party library called Colorama so caller Amma provides some constants for for changing the stream and all it really does is spit out a character to sister to stand it out and so here I'm said I'm printing out red and I print out some text and standard that will continue to be printing red text until I send the reset token so the next line actually adds a background color of green so you then get red text with green background and that's kind of what it looks like when you run it and you can see how this would make a huge difference to a program if you've got errors coming out in red and you've got warnings coming out and yellow and you've got normal types coming out and white it just means that as you're reams of information stream passed you can kind of see that that something happened that's actually interesting rather than just information so it makes a big difference with long-running programs and it makes a big difference with programs that spits out a lot of information the problem with this is that if with that script if I pipe the output to a file it adds that it keeps the tokens it and you don't really want that you want to store plain text when you're saving it in a text file and the way that you do this and this isn't something that's really promoted and documentation is you call Colorama in it and you pass in strip equals true and that just means so in this case we're just saying if it's going to the father another program strip out all the characters color AMA one of the neat features is it wraps standard out and standard error but when you initialize it and it means that if you print out those fancy color codes that are only supported on UNIX not in Windows if you're on window if the programs being run on Windows it'll automatically convert them to the Windows equivalent so just with the kind of a single line of code you've essentially made the colorization at least of your application cross-platform which can be kind of nice user credentials is something that's supported in the standard Python library batteries included and I'm just going to quickly cover it it's got to functions get past get past prints out password to the screen and then it allows the users to type in a password when they hit return it returns that as a string and it doesn't actually print the password to the screen yep get user is quite different get user is not an interactive function so it's it's slightly confusing get user will actually talk to the operating system and try and obtain the user name of the user that's currently logged in so in this case I was logged in as mark and so it's just picked that up and again these can people find it useful if your script is communicating with a database you don't want to have your password stored in the script that's fundamentally insecure you could end up committing that to version control something so if you want it to be a professional application you should be asking the user what the password is or perhaps you can store it in a configuration file progress is another one so if you've got long-running task I can't count the number of times I've run a program that doesn't doesn't spit out very much information and you kind of think has it died should i cancel it is it going to start again is it a network problem is it just a temporary thing will it will it carry on working so I mean progress is a really nice easy way of just so you know we're we're in the middle of something and it's still going you can see progress you can get an idea for how long it's going to take to complete as well so there's a library called progress bar which has been around for a while and I think is really really need some love and it's been forked so it's received some love with just in a different place but this is progress bar 2 if you pip install progress bar 2 you get a module called progress bar increasingly and then you can instantiate this and you can use it in a couple of ways so in this way I'm wrapping an iterator or a trouble and just each time through it knows that it's an extra 80th through the process and it will just print out this this progress bar on the line with a percentage in front of it you can do much more complicated things if you want to so here it has this idea of a list of widgets that make up your progress bar line this is a little bit messy but it demonstrates a few things so you can have strings and you can have what they call widgets which are kind of updated each time so in this case it's going to print out the string loading at the start you can see that in the comment at the bottom and then it's going to print out the percentage that it is through and it's got space and you can print out progress bar and then the last two kind of interesting so ETA just keeps a running calculation of when when the process is going to finish so that's that's kind of that would be something that you have pain to develop itself and file transfer speed assumes that every time you're updating you're actually updating with the number of bytes that you've read and so it keeps a running tally if the number of bytes per second which again is kind of you know it's a professional level of update so if you're writing a program that's streaming stuff from one place to another that's that's quite nice information for the user so with i/o you want to think about adding a flag to specify output so I said you can automatically decide and you probably should but always let the user override that because they might not want that json written to a file they might want the output that they used to on the screen for some reason and the same with verbosity unquietness add a flag to allow them to choose how much information they see and also be responsive so tell the user that things are still going on don't just leave them with a blinking cursor on a black screen that would be that'd be bad so let's talk about configuration we have a bunch of options for configuration and we have way more just out of the box in the standard library than I thought I thought until I scintilla I went looking I kind of thought of any files in JSON I suppose I thought about CSV but it's at yeah we've there's a whole bunch of stuff that we go out of the box so he can got any file so you can use environment variables we can use JSON out of the box CSV there's some rudimentary XML parsers and Python not fantastically performant but but fine for config but I don't think anybody wants to write XML for configuration we've got Apple plist built in on all platforms which also I don't recommend or obviously we could write our own parser using regular expressions or something horrible too and again just just don't do that third part we've got Yammer which I've been working I've been working with ansible recently and Yunel turns out to be quite a nice format to write it's got all the same data types as JSON but it's actually it's kind of nice to write and then we've got Java properties which you probably only want to consider if you're interacting with Java applications but I'm going to talk about any files and they're kind of they're kind of not exciting and they have a lot of us UNIX developers and probably thinking isn't that a Windows thing but I think that they're hugely powerful and I think people kind of underestimate the power of and possibly just just don't use it in the right way so I wants to cover it so the conflict parser module has a few different conflict parser objects in there I recommend you safe config parser because that way your applications there's some security holes and some of the others they're there for historic reasons more than anything else as far as I can tell and the way that user is you instantiate your config parser at which point you will have an empty configuration object and then you call read and read doesn't take a single file name it takes a list of file names which is kind of counterintuitive until you understand really what it's designed for and what it does is it goes through that list of paths and if the file that that path points to is there it will read it in and it will merge whatever data's in there with with it with whatever's in your config object and then it will go on to the next file and if that's there then it will read it in and merge it in and if it's not there it will just skip it it doesn't there are no errors if it doesn't file it find any of these files it expects to be getting a list of effectively optional files files of optional existence so the way that you use this so talk about these parts so the first one is as a defaults file that one assumes is provided with this script it's expected to be in the same directory it could be in users though it should be it could be in slash etc or something like that but is essentially a default configuration and then on top of that we've got the you per user configuration so if we're picking tool dot any from the home directory and then on top of that we've got conflict dot any which is in our current working directory I haven't specified an absolute path for that so that's just kind of where we are so that's yeah that's your project configuration and these might look like this so starting at the bottom so the first one that's loaded is defaults dot ini' and that assumes normally when you run this script whatever it does it's going to be talking to localhost on eighty eighty and and then it's I'll talk about your L in a second and then what it does is it loads tools tiny if it's not there it does he doesn't like it it doesn't complain but if it is there it loads it and it overrides port so there's like inheritance with objects so now we have local host and port 5000 and then then we load in the project configuration for this project we're pushing everything to WWE rockstar guru love those new domain names and so now we've got that host we've got port 5000 but we still have this URL value that's generated with placeholders so it's built up from host and port so if we print these out we got the host that I said the port that I said so you'll notice that also we can call get in so it doesn't basic casting for us so some passing if we want it it's not a huge benefit but it's kind of nice and then we got URL which is I generated value and you'll notice that it's generated on demand so it's got the values that we inserted in later files so ninja rock star wasn't around in the in the file that URL was defined in but it still pulls it in anyway and that may not seem like a huge benefit I mean it's a bit ugly as well but what it means is that if the user wants to use an HTTP URL they don't need to change any code they don't need to go into that defaults file and change it they can just they can set URL in their personal config file with a project config file and then it will use that and again they can use these patterns they can use host and port in there so it doesn't need to be they don't need to start from scratch again so it's kind of powerful that's that's config so I'm going to kind of skim through signals a bit talking really fast actually I hope hopefully can follow me signals so it's not nice code but you can set up a callback essentially so when the signals received from the operating system you can get it to call your function and what that first line is doing is it's overriding the default behavior so when you're when your function returns this is sig info which is a non-standard flag I actually remember how you pass it to your program but it can be kind of useful if your programs not spitting out in the output and user still wants to get some progress didi does this you can pass in a sig info using a control letter sequence and and then it will just print out as progress before which is which is kind of nice but the truth is probably most of us aren't going to do that very often we're usually interested in sig int which is keyboard interrupt so when somebody hits ctrl C to finish your program and that is actually caught by Python anyway so that isn't it Norden it doesn't doesn't by default just quit your program it raises a keyboard interrupt and this is why in all my command line programs I wrap everything with the keyboard interrupt because the default behavior is to print out a stack trace and exit with an error which which isn't really what happened so you generally want to you want to handle that and not really do anything I'm not really sure whether the correct behavior would be to make sure you're exit code is 0 when this happens because it's not an error so I'm just going to run very quickly through code structure and packaging because I didn't think I had enough time to do more so all projects are different and a lot of the stuff I work on are actually single file scripts and then there's there's not a lot to it you could add a setup dot P way to make it installable but you don't need to do much and often to be honest they just end up sent by email which is not great because any of these versioning you use the ability for people to update things like that but there's nothing magic about command line scripts I recommend you setup tools rather than just you tools because that handles dependencies installing any dependencies you might have and it also includes some some extra features specifically for command line writing command line programs so here we've got the set up py that we've written I'll I've got an example on the next slide and we've got our actual script which is my tool and that should be tiny I should be really really small so we should be writing all our code to be reusable we may be writing a great awesome command-line application but but one day we might want to take that functionality and stick a web interface on it we might want to take it to take a GUI on top of it we we might want all three at the same time depending on how the user starts the application so really you want all your functionality in the library as much as you can so my tool should really just call out to something inside my to lib and you can put that in it in a couple of places I added the double underscore main file this morning so I'd know that few people haven't really come across it I think it was added with Python 2.7 obviously Python 3 in that you never used to be able to run packages you could run modules so if you're your library was distributed as a single file you could run that but you it had no way to run run modules now if you if you run my tool Lib with Python - mmm my tool Lib then it will look for that double underscore main file and execute that so the it's not only I suppose you could import it if you really wanted to but it's not designed to be ever imported it so you designed to be run is the entry point into your program so I think I'd be tempted to put my command-line interface in that file or at least some of it and then we've just got some other files that are actually our functionality there's that grey circle is my tool doc viewing util stop py that's it's a reasonable way of structuring a relatively small script and then the setup view I if if nobody if anybody hasn't written one of these before they look a little bit overwhelming they're a huge amount of just data and it's a function call as well it's not a data structure and so they're just a bit scary but it turns out when you write them they're not that scary I recommend kind of but it's a bit of a cargo cult but I recommend finding some command line scripts that that you've installed with PIP where you respect the developer in some way where the script has been well developed I recommend recommend you're going to look at there's there's a whole host of options depending on how you want to install these things so in this case the important lines are really the two up one up from the bottoms is the tell at what package you want to install and you tell it there's a script called my tool and you could install a series of scripts you could have different entry points into your application and they'll just be one-liners calling out to functions in my tool Lib and there's other options with with setup tools as well because this is technically the setup tool setup function you did such a thing called an entry point and where you can where you don't write your script you're right a function is your entry point you tell setup tools what function with what functions you want to be entry points you tell what though you want those scripts to be called and when when somebody pip installs it will install those scripts for you and the nice thing about that is on Windows it will create an executable so to create a binary so it looks much more like a native application to the user and they won't have dot B Y on the end so it is their more native on UNIX as well exit codes there's not a huge amount of exa codes by default Python does some sensible stuff for you if your program just exits because it got to the end of the main function or the end of the script that it was running exits with zero and if it doesn't if you've got an uncaught exception it will exit with one there's not a great amount of information so if if you exited because the user provided bad data you could provide a different exit code if you if you had network problems you could provide a different exit code there's any number of options you can just it's a it's a very small token of information that's easy for somebody else to then understand they don't have to pass any information they're just handed back a number you should always consider that your program is being run by another program because that's going to make it more reusable make it more useful and there's no specific standard for what these numbers should be the new look a new project recommendations they recommend that you set the high bit if it's a serious error I'm not really sure of any projects that are actually doing that a non bsd that have a sysex 'its header file that lists a whole load of what they consider to be standard exit codes so that's probably a good place to look I mean it doesn't hurt that if an if there's a list of exit codes for things in your you have your exit exiting with that condition it doesn't hurt to reuse somebody else's standard so the stuff I haven't had a chance to cover really other command-line frameworks so people have been thinking about this a lot of the stuff here you can see it's kind of boilerplate code and it would be nice for somebody to handle this for you where is configuration stored on Windows versus Mac vs. Linux and cliff and Clint both provide some subset of that I haven't really worked with cliff cliff is designed really for multi entry point projects and it kind of integrates with setup tools and Clint is really a whole bag of stuff it does colorization and progress and command line passing and it'll load configuration from sensible options it's a really cool library it's not fantastically documented something I'm planning to well I have it I'm planning to add some documentation to that at some point let's see whether it's accepted and I haven't got a cross-platform consideration so I just kind of skimmed over that when I was talking about configuration file locations and it's just for lack of time it's like there's a huge amounts of this topic and I'm going to write up some blog posts on it and hopefully that will be useful to some people so I do I have five minutes left I wasn't going to do questions but it's okay if anybody has any questions if they hey um what is what are the creeks to get these command line programs to run fast especially if it's a short-lived command line program ah full I would say minimize your dependencies so with a short-lived program I guess it's not really doing very much so you're looking at interpreter startup time and kind of the time it takes to load in libraries so yeah I would say stick to the smaller none of these are big libraries none of them take a long time to load but yeah I I guess if you're loading a library that has a large data file behind it or something like that but it's going to take some time but I mean the nice thing about Python is it has a really fast start up time anyway so one of the things that makes Python such a good language for writing command-line tools because they often are short-lived and this is syntax error on that file but ignoring that there's a so if you change the behavior when you see a terminal versus when you see a pipe how do you go about testing that because obviously sometimes if you programs that change there even a test environment can be been knowing at times yeah I can see that I guess testing that I would I would mock out their standard out or standard error object with one that lied about whether it's a TTY or not and then you can you could use a you could sort the results in a string io objects and just then just check it against what you're expecting do you have any recommendation for interactive programs on the command line I got these I got this question last time when I say I totally forgot to write the slide up on it so i have used n curses briefly it can be a little bit awkward to get running on on Linux platforms but now I haven't done a huge amount of it so I can't really recommend anything specifically there's the built-in module or CMD which enables you both great stuff like that but it gets kind of awkward once it gets very big yes but yeah that case you don't know any alternative CMD is a very cool module for anybody hasn't used it so it's you essentially for it's for creating your own shell essentially your own dedicated shell so it's you can you allow it when you run your program you then get the blinking cursor and then you can then type type in commands so it works quite well with programs with sub commands for example it's like you don't have to type in to get part of get at each time you just you're within your own contained environment I've used it quite well with ugh past before so it'll only it will call your function you've declared as being one of your commands and then anything else that's on the command line just kind of gets passed in and it works quite well with with our boss but it's you're right that they're kind of difficult to manage and it's just a question of trying to make sure that everything is extracted out into their own functions and it doesn't just become one big mess it kind of provides an event loop for a command line program so cool so awesome that means basically that it has to work right um so about testing are there any specific tools that you would recommend like you know cram for example comes to minds of our other tools that you would suggest for testing the common line parts of application I tend to use unit tests just because because we're dealing with text or at least data and we're not dealing with kind of events and things like that I find that actually unit test tends to have most of the functionality you want I mean obviously there's some add-ons to unit tests as well I haven't used cram sorry I think we're that but but if you are only testing sorry you're gonna mark the sonnet innocent out so we can get colors as well and kind of stuff so I can get colors sure yeah but if you sure really unit testing that and you wanna see the call or sense kind of stuff there's gonna be pain right yeah yeah it's gonna be a pain yeah I mean walking is it we're sorry yeah sorry yeah is he gonna test the actual on the command line he expects works well oh okay thank you I think that's gonna have to be at Fabrice yep I think we're out of time I'm afraid so thank you very much you you

Channel: EuroPython 2014

Views: 121,867

Rating: 4.9034853 out of 5

Keywords: python, euro

Id: gR73nLbbgqY

Length: 41min 25sec (2485 seconds)

Published: Thu Jul 24 2014