Productive pytest with PyCharm

hi everyone and welcome to today's webinar organized by JetBrains I'm Paul Everett pycharm developer advocate and I'll be your host the topic for today's webinar is productive PI test with PyCharm our presenter today is Brian Aachen one of the leading voices for python testing brian is the author of Python testing with PI test a book I bought seven milliseconds after the first beta copies were on sale no kidding I was like okay I got to get this Brian how are sales on the book I'm doing really good actually and I don't have anything to compare it to but I'm happy just as an aside everyone I've ever talked to that wrote a book said I could make more money mowing lawns this was a horrific experience I'll never do this again and then they do it again what was it like right in the book I'm I I didn't have that kind of experience I thought it was incredible at working with an editor and it helped me organize my thoughts about the topic and I think I'm a better expert now than I was before I started writing it so it was worth it yeah if you want to learn something teach it yeah Brian's also very active in Python podcasting with the test encode podcast as well as co-hosting Python bites with Michael Kennedy for those of you going to PyCon Brian I heard you and some sidekick are giving a talk yeah you and me so I think it'll be a lot of fun were you planning to cover um I think maybe we should cover pipe test and punch parent together sounds good all right we are aiming that talk at visual testing with pycharm and the idea is for beginners who were intimidated by testing let's put a pretty face on it and make it accessible as I sound about right yeah all right today's webinar about PI tests is going to be super useful we had a previous webinar with Kenneth lob that covered the basics of PI tests in pycharm today Brian is going to help us acquire some high test Zen kind of looking forward to this alright now that we're all set I'd like to hand the mic over welcome Brian hi let's see show my screen yes I'm so excited to be here so I before we got started I put up the code and github and I've cloned it I've got it already and I'm just gonna open pycharm and when I installed PyCharm this last time it gave me the option to do a command line just make it available on the command line and I set this up so this isn't this pycharm shortcut isn't by default but if you install it right you can get that and if you forget to you can get to it from the help menu oh really cool yep so I just wanted to wanted to start clean to make sure that I didn't have anything that I normally do that I forgot and actually I already thought about something I started in the wrong directory so I'm gonna I've got my project that I put up there I'm going to go ahead and go into the task projects and I did not get my virtual environment ready so oops it's fast though and I usually do this this is something I have a convention I've always using vnb for my virtual environment and I have so many around that I add the prompt um to temp to help me know what project I'm working on at any time I've had source there we go then I can open it oh I need to install everything so I've got everything in in the setup in my setup all the the requirements are there so to install both the project in all my requirements so I'll go and install everything and I install it in editable so that I can modify the code while I'm working on it and open fire now this this beginning bit and just get it set up right pycharm is smart enough to just get most of this stuff right well just double check it and most by most of the stuff I mean just making sure the interpreter is correct that it's pointing to the virtual environment and that we're have high test setup so we see our project and I'm going to full screen this because I don't need to get out of my charm anymore and I've got some notes on what we're going to cover today so let's go ahead and pull that up and and I'm gonna switch it to View mode so we've already covered Who I am there's the github link a little bit why we're doing this webinar and I won't mostly put this stuff there so that when people looking at github later they'll be reminded but setting up looking at the interpreter and the PI test setting I think that's all I have to do to get started so let's go ahead and start that the interpreter is open up the Preferences and there's so many things in here that I just go ahead and search for it yeah Paul and people that are experts at this probably remember we're taught where it's all that inter Ritter project interpreters area and it is looking in my Everett environment so perfect that's just what I wanted the next thing I want to test is the tests there's that integrative tools and it did often this is a I don't know how it picks this but it it saw that I was using PI test so it selected it sometimes it doesn't and it shows unit test so we'll just make sure that it's set and then after those two settings we're good to go there's other since I have my project installed I don't need to worry about paths too much so I've it this is a pretty small project I've just got a command-line interface and the database associated with it and then I've got some tests and before we get into this project too much I've got a couple tests set up that are really basic so one of the reasons why so before I get too much into it do I need to increase the font size isn't any there's this good I think this is okay but let's um let's go ahead and increase it a little bit go to preferences and type font I've got oh good well thank you so what the first things I do is map control plus to increase font so we've got yeah this is one of the reasons why I like PI test is because a simple test is very simple it's just a test that starts with test underscore that's it sire it's a function so to run this run this simple test you can right click on it and hit run PI test and this is um then we'll see the output below we see that it ran it there's a lot going on here and we will cover a lot of it today and I'm going to pull up our where we're at today in the editor in a separate tab so that we can see that at the same time I saw a different webinar with with Paul that he did this and I'd liked it so I will move this over increase hello world test so run different ways so we ran it and that's good we are hello world this little dot means pass it's a PI test is a very not very expressive in that way but you can make it be so one of the things we'll do and we'll get there um I want to run this brand just one test um when we when we look at this there's this little visit control shift I think this means control shift R so let's try that and this one we're rerunning a lot is very handy to just rerun things quickly I mean and I was selecting this one text I can also select these other tests of course but I can go outside of if I'm outside of just in or in the file and hit ctrl shift are now I've run all the tests so I can run them within here I can also go over to my project Explorer and select any file to run and run that and just this ability I may this is this alone is a a great step forward in using PI test because you can do all this in PI test you can select different different sub tests to run but there's a lot of typing and you have to memorize a few things it's not difficult but this is way easier and then when once everything is run you can also select it within the test results so let's say you'll really want to rerun test one you can run it from there and that is really slick and you cannot if there's if you haven't changed anything this a little run tab just reruns things and then all these other goodies are really useful when we have a lot of tests so you can test results you can expand them all collapse them all a lot of this stuff was I'm covered in a previous webinar so I'm not going to get too much into it but this is one of my favorite buttons right here is the auto test and this what this does is it's what it just hangs out and looks at make sure when you change your code you can refine it so if I go over here and cert one equals one and just wait a couple seconds and we run it and how long does it wait well you can configure that so you can set the auto test though there right there so I guess it's the three seconds right now and there we go really handy feature use it all the time especially when I'm really messing with code a lot now what when we select any of these things and test it test in different ways what's happening really is is PyCharm is setting up these run configurations and you can look at what all of the auto derived run configurations just by selecting there and hitting edit configurations and these are the two automatically generated ones that you've got and you'll see that there's a PI test path of what to select there's a place for key words and additional arguments so that's where one of the things I said I'm going to go ahead and select everything and run it again so I can have a run configuration that runs everything but it only shows these three dots if I want to see more clips um I'm gonna add the additional argument so I'm going to add the - B for verbose well before you close that die up that's too late give that yeah if I could point something out the working directory that it chose was the test directory and this is why you had to do the editable install so that your package importing wasn't requiring the Python magic thing to be running in the parent of the directory with the code and so for those of you that wondered why did Brian make me do that step of a setup PI and a pip install - e dot whatever this is part of the reason why yeah there's other ways to get around it so this this even if you edit it in here for PyCharm it automatically adds cool things like adding their the the source routes to Python path so it should have it would have founded it anyway but then that would have failed on Travis or something like that yeah the this is so I end on all these automatic things a lot if you want so I added additional argument here - b-but the it isn't in the other one if I want it for all future ones I can go into the defaults and find my pipe Python test a test and add it there and it'll get copied everywhere else um anymore and we'll touch on keywords a little bit later but I'm going to go ahead and show what - V does dude you just taught me something about PyCharm I don't really never thought to do that alright that's good and so with the - P we get a more verbose output we see exactly which tests for run if I if I really just like that and I want to keep that around all the time the one of the best things to do is to just instead of doing it in I have it's in your configuration you can add that to your PI test any file and Nanni file is just you can put usually one of these I'll put it at the top level of my tests and it's a I forget what format this is but it's it's pretty easy and I'm gonna add ops and all of these are easy to find in the PI test documentation but I also covered in the book of course look for the book and so if I go back to my run configuration and take this out and we're my tests yeah it's still there this time it picked up the - V from the tightest any file and that's a little bit of that we're we're quite just any Wow I even planned to do that that's going seniors testing - oh we didn't do it if I want to do that because that's really cool about hydrant my hello world oh it's in the wrong paint close paint I want on this side and went the notes and this side I want hello world yay so one of the things that high test does is it when there's something wrong so this test will pass and I'm still learning to use ctrl shift are this passes but let's muck it up so let's make it fail by adding like four there and what PI test will do will show you if there's a failing equal assertion it does expected vs. actual and it doesn't know where you're expected in actual is it just picks the left side is the expected one and the right is actual so if you're used to putting the calculated value and your expected value on the other side just know that PI test will reverse those when in this output the one of the nice things about PI test of course is that it'll tell you exactly in a lot of assertions it'll tell you exactly what the difference is like right here at the bottom tells you that the three and the four different PyCharm also will let you and we have this cool click to see the difference thing and it pops up a diff viewer and you can see the difference also and I like that it's neat now this we seen a lot of the this is actually something I just learned this year is this set notation so this is what I've done here is there's put a test within a class and this is just to show I don't usually use classes for testing but it is kind of cool that you can put a bunch of tests in classes and one of the reasons to do that is so that you can run them together so if you have now a couple class of tests that you always run together and they're related you can uh can I run that class yeah I can just run that class of tests that's one of it's really one of the main reasons the other thing is you can scope fixtures to class scope if you feel like it so that you can have setup and teardown at a class level let's speaking of fixtures that's one of the main reasons why I got into pipe tests you can take some questions because we have a good list oh yeah yeah so I have a reminder here that we'll take an interlude so for uh for questions yeah cool Thanks first question from Martin my production code has logging with relative paths which break when I run PI tests from a test directory is there an easy way to disable logging for a PI test run probably but I'm definitely gonna have to look that one up it's not something that I do so the next one's gonna be a lot easier there I think there's a pycharm answer and a PI test answer you've already covered hinted at it and PyCharm is there a way to have testing performed continuously always running whereas you type code and checks near real time yeah well the the we covered the the continuous or this auto test feature but within there is a couple plugins for pi tests that if you're you use using having wanting it to run outside a pycharm of course there's a a pi test watch package that i'll try to look at the URL for but it it it you basically you give it a configuration and you give it a directory and it watches that directory for any changes and reruns the test and then talks but it's not tox-ex Geist I believe is the exodus test Exodus is a plugin that helps with distributed testing you can run and a parallel testing you can run a bunch of tests in parallel but one of the features that has as well is a flag to a watch flag so we'll watch things as well alright next for this sounds like a paid political announcement for you for new Python programmers what is the best place to get started with PI tests yeah well definitely my book but that's one of the reasons why we're doing a lot more visual things like like this this webinar and I'm also planning on trying to put a course together however a lot of the information we've got that I've put together was through my learning on how to use PI test itself and all of those those learnings were captured in a blog series of blog posts at Python testing net that's a that's again another plug for me but the PI test community itself is my test on org and it has there's communities great and they're they're really eager to help new people so reach out to people all right you can do this one fast because we got a lot of questions does pi test do behavior BDD gherkin I believe so yeah there's a there's a plug-in pi tests BDD and I have not used because I don't like to use gherkin and stuff all right this next one you can talk about a little bit I'd like to talk about a little bit as well from a pie chart perspective can options be taken from set up dot CFG as well in my projects I have my dot coverage or C file and I have it at the root and I spent a long time yelling at pycharm because it wouldn't see it and that's because I needed to set my working directory in the run configuration to start at the root there is a ticket about this the PyCharm should walk up the tree like you're supposed to and we but do you want to talk about set up dot CFG and PI test uh not well I believe PI test will pick it up it does the walking of the tree but it does it it that's one of the reasons why to have a PI test any file is it will walk up your test directories to find configurations until it finds a like a PI test any or a setup that CFG or I can't actually remember all of the different things that we'll look for but having having nothing sometimes if you're one of the things I've noticed is if you're playing with just some random tests and you're just putting it in the directory a subdirectory somewhere you'll get weird bizarre results sometimes because it'll walk up past you or what you think is a reasonable working directory looking for configuration file so throwing an empty PI test any file within a directory that you're playing playing with code in is a good good idea let's see let's say we get you back on track we got a whole bunch of questions but we'll save them up for the next interlude okay great um I this before we test this a lot I just this is a stripped-down version of the tasks project that I worked on in the book but it I just wanted to show people that haven't used it I don't know what my tasks are right now that I do webinar well I'm working on that so the tasks this tasks project is just this simple command line it's a to-do list and I built it so that I can keep track of what people are doing at work but if I it doesn't do much it does you can add a task count them list them it's a simple credit application really but this trimmed-down version so that's what we're looking at but we're not I normally don't test through with a user interface I like to test underneath right under the user been a user interface for most of my tests so I've got the command-line interface separated in into into two files and most of my tests will test against this test DB interface or tasks DB so we've got this here one of the things I've set up already is a so I've got this this task object that I'm passing back and forth between the command line interface and the database the this I'm using adders now because I'm kind of in love with it in the book I used named tuples but they worked if from the outside use model they're similar but adders has a whole bunch of cool stuff that you can do with it like one of the things I've got here is this compare equals false what this does is it makes all of the objects that I I want to be able to compare things and have if as long as the summary owner and done flag are the same then two tasks tasks are equal I don't want this identifier to be part of the Equality and that's all you have to do to get that to work so I've got I want to add a test a test the add one and I've got a first attempt here so I've got this is actually pretty good and this will run and hopefully it still passes and it's doing a lot here we've got I guess I didn't need that import right now but I'll use it in a minute the this test will this takes this parameter and this isn't really a parameter this is naming temperature which is a built in a built-in fixture for PI tests and what it does is it gives me it's a figure if this fixture gives me a temporary directory when I'm running it and we can use it to create a temporary file within that directory and I'm passing that so to create an empty database I've got this I gotta have a file I got to give it to this task database and right now I'm making sure that it's equal this isn't really a great style because I I like to have all these certs at the bottom but I want to make sure this is empty here and the actually let's take a look at this path and one of the ways we can do that is to go ahead and run this in the debugger but then we'll fix this a little bit more um and running it in the debugger and then we can just wait till it stops right I didn't really need to do that uh keep going there where now my path is this long temporary path I don't really care about it usually but one of the nice things about this this is for debugging purposes is PI tests will keep this around for a little while and I can't remember what the it's well defined what a little while means but it after your test is completed if if there's a problem halfway through you can go you can go and you know dissect that and look at it and see what's going on and as you notice I guess jumping into the debugger to take a look at it debugging tests there's a real handy feature I use it all the time and run that but I so part of I do like to keep with either there's there's it's known by two names given when then is what behavior different development uses but it's the same as a arranged active third so I have extra C there and other models but I I like to try to push my given up into fixtures as as much as possible because if if there's a failure like let's say creating attack the database doesn't work right and so I'm going to artificially assert right there and rerun the test on the the pie chart marks this is a failure but but I test will also fail it and I don't really want that to be a failure because my adding didn't fail it's the setting up the database so let's move this into I'm going to move this into a fixture and to do that I just put my test fixture and a different function name I want this to be database so now I've got I need to still need to import the tempter take the path assert that and return my database so that you with fixtures you can if you need to do a teardown you do it here but you have to change the return to a yield at that point you just still return your data to to your test but now I don't need within the test I can just say DB I don't need in all of this and I'm just gonna put yeah this all work now to see this a little in a little bit more detail to see what happened I think that's useful to do I'm going to go ahead and run this same configuration with an extra flag I'm going to call that it's set up I think I sometimes I have to look this up but I've done this so many times hopefully I got that right and high test within this with that set up show let's show you exactly what's going on we've got a temperature factory that's actually being created before temperature this temperature uses a module scope session session scope that's what this s is for and then a function scoped after and then then the database this fixture function is called then my way then my test and then we do the tear downs so we don't really have anything to do here but if we had some more code here that would get run afterwards and then we tear down that way very handy and being able to as we can see the I'm I'm already using fixtures that use other fixtures this is just chaining them together will make your life easier the I'm like I'm going to clean this up a little bit more take this run configuration this show out but I made this comment here the when I added something to a database I've got a lot of side effects the account will be increased the I'll be able to get the at the item adder there there's a lot of things I could check but I'm going to know that I'm going to probably write tests to make lists tasks is kind of complicated count early doesn't tell me too much about exactly what's in there it's useful but I will leave it in all I'm gonna take out the list tasks but I don't think that's all those tests somewhere else but I'm making sure that I can get out the thing that I just added I'll just leave that as my test for now it's not a very complete test but it doesn't pretty good I like to keep things the certs really pretty short if there's a whole bunch of them maybe we can just make more tests but one of the things I showed you before is a the failure yeah that happens when I win this failed within the test now if I have this assert fail within the fixture we get a different result so let's go ahead and run that and now it's hi-hi charm still says I had a test failure but I test will tell me that that's an error not a failure and I you can separate those out and I this is huge for me because I'd like to make sure that I chase down actual failures and not things that like if I have a test that if the ad failed I would think there's more something wrong with the ad feature but if it's an error I didn't I know that I didn't even get through my giving condition so that's what I'm like when I'm working with a lot of complicated fixtures that's excellent to do and the other thing now I want to be able to I've got this useful fixture now and I'm going to want to write more tests other places like a test delete our test list in it maybe a different file but I can't see it from this file so I want to share it with other tests I want to take this out completely and I'll grab my imports also and put it in like alpha test I don't have anything there yet I don't need these comments and the return is probably fine and now within my test add I don't need this fixture anymore because it's already in my comp test and you can rerun that again and super-handy I like it yeah I kind of covered a lot of stuff is this a good time to take a break for your questions great time okay um first on this go back to the previous fixtures this way yep I know that you had a yield oh yeah and we had two questions about why do you have to do yield instead of return you don't have to but maybe you want to talk quickly about that and then I'll go on to the list of questions within the you you you have to yield if you have more stuff to do below it right so if I have like if I have to DB disconnect or you know if I often resources have some cleanup that they have to do you'll want to do a yield the resource and then clean up afterwards you know the test frameworks that's like setup vs teardown when you use yield you can do teardown copy yield yeah in the if you do a return then clearly this is just like this is like it won't it won't get that so if you've got something afterwards and when I run this like for instance if I have a connection to know if I'm using long ago as my database instead of tiny I do need to disconnect from it so in that case I will stick a yield in there instead and for those of you that want to see that in action you put a yield with something after it and then run your test with the debugger and like put a breakpoint before and then you'll see that execution will go off into the tests and then it will come back to that print statement on line 11 that brines putting up okay um zooming way out can you go to a test the simplest possible test that you have hello world or something sure so Jacob is asking the questions why is this better than unit test um you know right now so for unit test the the main difference for like this this test class I would have to say unit does not just case and other than that oh the asserts okay so they with with unit tests the assert you can this will still fail and you know if there's a failure of their unit test will fail this as well so what a lot of people say there's a lot of boilerplate with unit tests I don't really see this as a ton of boilerplate it's not terrible well not to get better test results on like the the good to different the yes the assertion failures are important to you to see what's exactly wrong then within unit test you the user there's a whole bunch of helper functions like I don't even remember if he self-assertive all well then there's a hole but there's a whole bunch of them and I don't remember them all that's the mate there's there's that's a decent reason the other reason for simple tests is just that I don't have I almost always get to the point where I'm using fixtures and even for things like temporary directories so I can't use those with unit tests yeah Jacob I'd answer that similarly that your first five minutes of life with unit tests versus PI test might be the same just like in web frameworks it's the remaining five thousand hours of your testing experience where PI test really shines yeah there's there's definitely I mean it's so it's PI test isn't work in progress there's it's still going on I mean it's definitely usable there's tons of people using but there are occasionally you'll get assertions where there's just bizarre bizarre output you have to dig through and you know if you're used to looking at trace packs and it's not terrible look at the firt the first inch and the last inch should give you a clue as to what's going on all right on to the next one we'll try a rapid-fire this a little bit is it a good idea Hans asks is it a good idea to keep your test methods with the original class you were implementing and testing I believe he means the test module in the same directory as the module that you're testing I think I know the answer but I'm gonna let you answer and see if I get it right well my answer is no because there's a couple reasons on the I'm going I'm running the test I like right now it could be for instance if I'm running talks I'm not running this code right next to it I'm running the the code that's running in an install some of the talks will or other continuous integration will install your code and run it they're not from your source code the other bit is I'd like to be able to refactor things a lot and if I want to completely rip out a whole bunch of stuff here in the source code and rewrite it I definitely want my tests separate that's my answer also if you're distributing a package a lot of times people don't want the tests distributed with the code and so they put it in a different directory so that the the wheel doesn't contain the code I do not know the answer to this if you don't we'll let the pycharm team take it what is the best approach to doing PI test testing within a Jupiter notebook ooh I don't have an answer to that all right all right so that one's for you buddy all right this one's an easy easy one but you can just talk a little bit about how it works despite tests integrate into continuous deployment workflows with Jenkins yeah I mean I do it all the time there's a there's a flag that I always forget so I look it up but I can just help to help and grep for J unit something like J in it yeah J unit XML if you it's built into it and if you add that to a path it'll export an XML file with all the test results and that's what you need for Jenkins to you just you know set up your command line to run pi test on something and then set you need to output that XML and then Jenkins can read that all right next up this is one where you can explain kind of the layers of the testing stack for what is pi test better than simple asserts can we do the same or as PI test a framework with many features I don't actually understand the question I do idea sure you can put asserts in your code and just run it under Python you want test Suites you want fixtures you want to test runner and I know in JavaScript these are all different kinds of packages that are meant to be worked together but the test runner is different than the assertion library which is different than the etc etc etc oh yeah like for instance a lot of people have an existing set of unit tests for instance be written in written or set of tests written in unit tests those that's a different assertion library than PI test you can use PI tests to run those you can have pi test just be the runner that is a possibility the yeah and the benefits of keeping the reason why I've gone with PI test is because I can do from everything from unit tests up to huge integrated system tests with them even external equipment and control that just from the same framework as a quick note from Andrey Vlasov ski the pycharm Community Edition team lead pointed out that python 3 7 will have the data class support it's usable in PyCharm EAP now for stuff similar to adders which you talked about at the beginning and Morton who had asked about the diff and you had showed how you can click in pycharm and see a big diff but in PI tests itself you have a diff that's too big for the console display is there like dash dash V or some other flag you can give to tell PI test to be copious yeah so there's a like I actually I've got this I don't have this example here so let's let's make this fail you'll see a and oh no ringer off the I've got it usually says so it says there's a diff now this one's it had it you could see everything but but sometimes it'll say use a use verbose to see more oh I already have verbose setup that's why so if you don't have her bus set it you'll get a little flag that says a add verbose to see more information so this um so there's a lot of X this is more information than you would ski without verbose got it in that file that you have opened right there can you import PI test make a you know a Hello fixture and put it in as an argument for test one sure do we want it to do anything yeah that's fine we just want to show that put your mouse over the argument PI and PI charms whining about shadows the name yeah we know pythons got to do better about that the way to make the pain go away is to move that to a comp test or you can say on line seven fixture name equals hi and then change the function name to kind of high so to do I always forget it's closer right yeah that made that one go away yeah one of the things that is showing here which is fine it says parameter high is not used this often you won't if you have like things that you want to run before here tests they are you're not using the data from it you will have fixtures set in set up that aren't being used in the test that's very common that happens to me all the time let's see let's let you go ahead and continue and come back to these ok I'm like one of the questions sure um I've I've already filled in we're gonna do parameterize testing a bit I've got the other one of the other things already filled in let's drop this down for now um a whole bunch of tests about my task data item data object and these really helped me when I when I switched from switched to using adders is to be able to run tests against against the system to make sure that these worked just identically with with named tuples vs vs adders and these I've got I'm I'm like making sure that it caught these two items for instance these tasks are equal even though they have a different identifier an ID the the unequal let's go ahead this is only the only thing I did is change one element here but that's you know I I might think that that's not really that complete of a test so instead of writing a whole bunch of tests with a lot of different like maybe change the summary change the owner in one I'll just go ahead and do that within one test I can just add an import PI test first and then I can parameterize it to remember to use the British spelling parameterize there's no e here and I'm gonna I'm gonna pass in the T a second task I'm just gonna call it another um and I've it passes in a list I'm gonna go ahead and move this up actually could I have a whole bunch of these tasks and this is so this is identical to the first one I'm going to pass in another two there and T one it's not equal to another and then just do a bunch of these um so then in the second one I'll change changes of true to false and this one I'll just change my name and this one I'll do something gently maybe just sit and the IDS don't matter because they they're not part of the Equality and let's run that oh I failed oh the first one first element failed no because the first element I got with my task test the first one was the exact copy I copied that directly so of course that's equal take that out rerun and now I passed and the parameterization is slick I can do a whole bunch of testing against one to one test and just change the parameters the one of the things I want to point out is the incredible use of IDs because this is not very useful this 0 1 2 I can't I mean you can go back and look it up but there's a better way and the use of IDs is just something you can add and what it does you can either you can specify them there's a few ways to specify them the list in the book but right now what I'd like to use is that you need to have something that takes one of the parameters and returns a string so like a lambda will work and if I take a task and I want to return a task ID that will not work because that returns an int so let's see what that happens we get some horrible expression that is into object has no attribute in code we you know there probably could be a better better error message for that but what's going on is I intentionally did that so that if you run into it you'll see oh my my IDs needs to be an int string so I'll just convert that to string and rerun and I might have yep now yeah IDs on ten eleven twelve that seems weird yeah clearly that's a bug somewhere in my code because it's oh oh the ID can't beat you those three I don't know that seems weird I'm gonna have to debug that so let's not use ID it's not very useful another great one that I learned about that's obvious is just pass in Ripper is it great a great thing to pass in for IDs and now we've got our our tasks those are great IDs and I like it this is good we can we can start we do more though and I'm just gonna add an extra silly test test boo-boo and what if I two parameters two things X&Y and yes we don't need to do anything but I just mark and the drys x if i want to pass them both in i can do it right here in which case the syntax is you've got a list of things so we've got one two three four and we'll see what this does now one one or two got passed in the first time and then three and four got passed in now the this is so that you can pass it in it just you've got a set of things these these always go together we do it this way and you can have more than you get a three or four or five variables whatever they'll get passed in and they get named here you can use them within the test the if we want to do we can also do a similar thing and set it up almost as a matrix and so that we can test all combinations and you can that's done with stacking and so we'll just pass in 3 for X and for y we'll do like four five six and a different name and now I've got a whole bunch of tests because one got passed in we did passed in 1 for X and then all combinations of Y 4 5 6 and then went to 2 for X and then 4 5 6 so we can build up quite a lot of tests really fast this way by the way the pycharm crowds pretty smart they spotted what happened with the 10 11 12 on the previous test because you have one at the end and it's not unique so pi tests appended a character to it to make it unique cool so let's change that somehow good job on that well that's pretty cool that's a cool feature as well but I'm a lambda t as we have a string T ID and don't have enough parentheses yep yes okay yep the ID has to be unique of course all right we've got a bunch more questions although I'm up to show now you know actually answering questions is way more important I think that if there's a bunch of questions and answering all right cool can you go to an assertion somewhere sure like our hello world decision that will actually line 18 so this was pointed out I can't remember who mentioned it maybe Martin that PyCharm has a neat feature I personally get expected and actual backwards all the time so it's like on the equal equal and then do alt enter all to enter mmm which is quick fix in flip flip the evening yep put the left on the right and the right on the left mmm so that's one of those little janitorial things that pycharm does really nicely someone pointed out and I think this might be worth mentioning is in the assertion you if the assertion is cryptic and you want to make it easier to read and this is in your book I believe you could put a comma and then a message right yeah like an F string or something oh yeah sure that'd be a better string to put in there expected if if I Paul no yeah but we permit to do that right we're not to go into it too much but it's a little bit like exception handling in Python if you've got some obscure thing happening 50 miles below you catch the exception and turn it into something a little bit more domain-specific are you going to go over mocking no ok so Kelly asked about mocking if it's that covered in this webinar we could have five webinars on mocking can we talk briefly about PI test the mock and mocker if that's the way you do it or if you just use pythons mocking well I just has a monkey patch that when I okay I mucky paws yeah when I'm using with Isis I often use monkey patch is a great way to just swap out part of your environment so robinson's if one of the things that I've got and I would use it like in the command line interface I had the simplified version in the in the book I have a version that has configuration reads it I can out of a configuration file but within this simplified version I've got down at the bottom or somewhere in here I'm just using yeah here it is I'm pulling out of the home directory I'm just sticking this this database in somebody's home directory um if if I'm with a monkey patch I could change what the home directory putting it down with him test and point it to a temporary directory I use that all the time it's covered pretty well in the book so the answer is get the dang book yeah good question the DB fixture how does it receive the tempter variable Oh temperature is one of the built-ins high-test so if I there's if I'm wherever I'm at I can do potestas fixtures and see what all the fixtures are that are available and it's one of the built-in ones it's documented well online and all of these all the ones that are available here are going to be available for your tests and you just you just put it in there and I test knows where to look it up as an example click on temperature and do command b and b and python pie chart does not know where to go which is a shame because that's one of the things PyCharm is really really good at this whole fixture thing is a magical pie test universe and in the future pycharm if we support pipe that's better we'll do a better job of navigating around and looking up fixtures yeah and the with it but one of the things I did show you was this pie test fixtures it it also showed I didn't have I've got fixtures so I didn't have a docstring available in in my own fix the DB but we've created two fixtures DB and high so far and this fixture shows you exactly where they are so if you you're not sure where they are you can use the dash fixtures and find out where they're all right next up how do you control the order of fixtures execution our class scoped fixtures always run before function scoped fixtures are run before they're needed so the if I've got and that's that's a I know that's a little bit obscure but the but it's to talk about scoping did you talk about scoping yet I have it so we've got within within a fixture we change the name here but we can also add scope and the scope can be we've got class session there's there's let's just write em out session module and class and and function oops completion not working then autonomous so the the session scopes this scope is the like the lifetime of the fixture so you've got if two tests user sessions if any a session is your entire test session so you're not like well that's all the time I'm sitting as my desk but when I run PI test it's it's all the tests that it runs will use one version one instance of a session scope fixture and then and then a module scope will be once per module if it's needed a class will be once its first once per class so if I had a class cup fixture being used in here by either the test or by reference within the by the class it'll get run around this and that will be run before function scopes within that test however as an example if test three used a class scope fixture and test for use just a function scope fixture you're still going to its if you only run test for that class scope fixture isn't going to get run so they if you have dependencies between fixtures they should depend on each other so if I've got and that's um it's it's okay to be completely artificial about that so let's let's just put a couple in so I've got a high test fixture with scope equal session big something and I've got a function scope that out that I always want a function is the default so if you don't write anything it's a function scope and I've got a something else that that always needs big to run before it even if you're not using the data from it just specified it and and then anybody that uses the little test will um let's go ahead and just do that so let's have test one used little and we'll run that and then we can run it with them with our setup show and we see that big gets run before little if there's if there's an order dependency that you need specify it that's what I think that I just learned something that's really Andy related to that you just showed little which doesn't do any computation or anything useful sorry it's purpose isn't to return anything useful it's to do some computation someone was pointing out that we can't there's a decorator that allows you to use a fixture without using its data Raider for that yeah you can right so there's a it's used fixtures yeah it's called but I think it's a marker so it's something like let's try it if on the fly by test mark use I can't remember if there's an underscore or not and it would be something like that but I would I would look I would definitely look it up because that's probably not right okay let's speed through some things what's a way set of fixtures to reuse in other projects and the the the best way I think is to you can package about but there's a lot of ways that people do it my favorite way is to go ahead and just make a package and write a plug-in so if you write a PI Titus I test plug in two with all of your fixtures in it you can just have that be installable anywhere you need them and they're available the it's kind of more more to how to do that is definitely more takes more time than we've got here but I since it's something I encourage people to do it's covered in the book of course I don't know of a better way the other way would be to just have um you know you could share things in comfort test and copy them around but that's not really that great get sub-module or something like that yeah but anything that even if you just write so they they need to be yeah I don't know and then for those of you that use Fink's for those of you that write Sphinx extensions and for the three of us that write tests for those Fink's extensions Sphinx ships with the thing that Brian's describing a set of fixtures that you can use to test your Sphinx extensions so that is an example it's a PI test specific thing - can we do something in Prior tests before running tests like at before in j-unit or you know unit tests unit tests - stuff like that fixtures are the way to do it but is there some other machinery yeahthere's mean there's a lot of hooks that you can then utilize so you just want some code to run before the rest of your tests is that what we're talking about well the there there are some hooks but the definitely the easiest way is a fixture so if you this is a one I'm going to just go ahead and show this because I use it all the time and I didn't think to do this if I've got like a fixture that I want to run all the time I can name it whatever I want and and then I add audio Auto use ah that's what they're looking for and and if auto use is set to true um you will always it will always be run all right we ran that I didn't before isn't being mentioned by test one but is getting run hi everybody it's a function level so I probably don't want it to be run before every tests so the if it's just one time thing then make sure that you're setting the scope to be a section scope something that I wanted to point out about scoping in my Sphinx project I want to be really careful when I rerun the universe and Sphinx because it takes a while and so I use scoping pretty carefully to speed up performance so I can calculate something once and reuse it across a bunch of tests right and for instance when I'm using in my tests here for native business a lot of the same thing yeah but so the the tiny DB is really fast to connect you but I don't really need to create this temporary directory and connect the database all the time for most of my tests I could have set up this as a session scope yep a session scope fixture in which case I have to use I will have to use a the temperature factory instead of temperature because temperature is a function scope I have to use the factory so we would change the temperature use and then and then I'd have another I would probably like tatemae to name this session or something and and then the DB would rely on it DB session and then instead of connected creating it I would go ahead and just do DB session toledo some some way to clean it all out and then yield it a return that would be one way to to make that faster within the tests okay next question is one that for people reading Brian's book you'll probably predict that he's gonna giggle with joy at this question I can fixtures be parameterized for example a database connection fixture this pre-owned rise to connect to three different databases basically the combination of parametrize and fixture yeah there's and that's in influx but definitely yes it's a parameter is in fixture yeah you just um you can add params to it so that's the answer Corrine and then you give it a list of things and so later in your test when you use parametrize like Brian showed before to shove a whole bunch of data into a test that whole bunch of data is also going to be shoved into the fixtures which can pick the pieces that they want and return different fixture data based on the parameters all right yeah yes but the it's it's done a little different so to get that data out so if I had it set up like this I probably wouldn't use an auto use at this point but for for this but the you to get this data out you have to request wrong yeah there's a request object and you return like request not for am tram I think it's that um I'd have to look it up them and you know let's see yeah we don't have to act show it but that's basically in the ballpark and get Brian's book because he covers it pretty well but also that that's the thing that the if you want to it's called fixed repair memorization and look up request program if you google that you can find stuff it's the kind of thing that doesn't fit in my brain but I know where to find it exactly it's one of the reasons why I wrote the book is because there's a whole bunch of this stuff that I can't fit in my brain all right um let's see Cody says five webinars on mocking would be great by the way sign me up yeah mock mocking is one that takes so much Zin oh my gosh I still suck at it alright a question why is import PI tests underlined red up at the top good question that happens to me sometimes as well it's just a case of pycharm getting confused pine test does a lot of magic on a number of things so Ryan for example delete that line the import PI tests and then click on PI tests at PI tests and do alt enter enter into the import and let pie chart do the import for you and did it do an import at the top scroll up this is like it doesn't look like it okay so that's an error on our side the orders it eventually is there a way to use the data from an auto use equals true fixture inside the test yeah but you have to name it oh yeah and I mean that and that's it's not even that uncommon that sometimes you something you want auto done then a lot of tests don't care about the data but some of the tests do you just named name it and it'll you'll be able to get it here's what I can't believe you and I for didn't ever talk about showing this and it's very much worth showing because it's a good mode of testing and I personally look in the wrong place every time testing that something raises an exception oh yeah yeah with something we should show yep um so yeah so this let's make an exception so let's say I give I've got this added new ID I know it's in there so let's um instead of writing a new test I'm just going to extend this one for now if I try to delete it um Phoebe duh what did I have for delete delete delete ID cool this should work this should be fine but I shouldn't be able to do it twice and probably throw an exception and if I really and it did cool I got a key error so if I know that that if I want to write a test to make sure I can't delete things twice I can do by two with by test raises here and I have to import by test again and I will pass but if it right if it raises some other kind of error like a name error it'll fail was it you was it a useful failure gear well I guess I'm not sure if that's really a useful message or not but definitely made it fail okay okay is there a way to control the order of tests and the test yeah the test for instance so if we run all of them they all get run in some order and what is the order there there are ways they're different like for instance there's plugins to do randomization you can randomize them you can but they order them there are different plugins that you can get and to change the order and there's a that's one of the best things to do is to get a plugin that changes the order and look at how we it does it but essentially I don't have the exact details for that by the way in the background the pie to pie farm developers are talking about the red squiggly you got for import pie test do you have more you want to show before taking any more questions we are we got to wrap this up pretty soon we're way over but oh yes yeah there's I I'm glad we got questions I'm going to explore thing one of the things I wanted to keywords are great and one of that's one of the things I wanted to say like for instance I've I have all these tests if I really wanted to just run test foo for instance I could go find it and run it but if I didn't remember where it was for instance I can just run from the top level I can add a keyword and it's probably needs to be a string not sure um and yeah and keywords are awesome and that you can add them with we can do things like and an or not and those are fun um but you know we're just getting started we'll try to get more information to people as questions come in before we wrap up just to show off a little bit of pycharm sexiness you talked about debugging I don't think you actually did it right uh yeah I'm not sure and but it's super easy perfect today I'm not running my tests oh yeah I'm not right in that test okay yep there yeah yeah and that's what my learned from your webinar is I think visual debugging that was a good one this this is a cool feature step into my code and I didn't know that was there and that's awesome it it says basically the step into code but I don't want to step in libraries or anything like that I just want step in my code and I use this all the time so like I just up to here I don't want to step into the raises function so anyway in my PI test flow I'm an embarrassingly wide monitor and I take the run window and put it on the right and I have debug that appears on the bottom just so I know when I've got the two backwards and I find that I spend most of my time running my tests under the debugger because under Python 3 6 pi charms debugger so fast and I can just click a breakpoint and I never flail around wondering what the heck is going on in my tests in the the built-in Python console built-in terminal and the - duze are great but all these things built into pi charm I know they're just I mean they're just extra little things but it makes it so that half the time I never leave my turn yeah window anymore and another thing can you install coverage by T this next one up sure we've talked about testing is something that Python developers know they need to do they don't do it frequently enough it's a little intimidating so pycharm puts a pretty face on testing and once you start doing testing you start thinking about coverage how much of my code has tests well coverage is a tool that is like testing you know you need to do it but you don't and if there's a pretty face where all you have to do is click a little button and it works for you then that makes it easier so in this case we can run the test coverage by clicking yep pretty good scooting let's try clicking up in the toolbar on the icon for coverage and see if that's any different oh because it was still installing all right hip wasn't finished okay then it only ran Fuu but I wanted to run everything okay so this runs it will give you a little panel on the right where you can dive down and see the test coverage it will on the very left in your project tool it will annotate your directory structure with information but most importantly in the middle where Brian has this cursor your lines have color codes in the gutter that tell you information about what lines that were covered by tests and what lines you haven't written test for there's also a feature that you can turn on what's called branch coverage which slows things down a little bit and it will tell you which statements or conditionals were not followed in the wall so decorate the gutters on that nice coverage is a good part of testing good part of high test and PyCharm does a good job of putting that within reach there and coverage by itself is a super cool tool but this this is gutter thing is really nice yeah okay I believe we are way over under reordering tests David says there is a hook you can implement pi tests under collection under modify items you ever heard of that yes yeah this gets a list with you test items which can be reordered but that's it for a thing on the exception handling do I need to have a new width block to test multiple calls with an exception raised by each so there's like nested exceptions and I we're going to put it the the width block will test anything within this block will check for this razor I believe you can have a list of possible polymer actually I'm probably lying there I'm not sure if you can have a list um but if there's if you've got multiple calls that your different parts you really should have them separate though if you really expect different parts to be raising exceptions or if inside of there you want to do some inspection of the exception you can put as exe or something at the end of that context manager and then that will give you access to the exception that was raised yeah okay it doesn't have to be a special name it can be whatever name you want eat that's it for Mike all right this was fun all right you got anything you need to do for wrapping up no all right we will switch back to back to my screen okay thanks Brian for taking the time to talk to us about PI test really we're lucky to get you on this webinar you've done so much on the book on testico podcast and this you know when I bug you to do five more I'm mocking you'll be sick of it but we'll take as much as we can get well we I am thrilled with the the Sheppard's team and the PyCharm team because you've been incredibly responsive when I bring up I've been testing with PI tests with this if I find anything out you guys are usually already working on it so the proactiveness of the team has been great well thanks all right if any of you have any questions later please don't hesitate to reach out to us by email or social media my colleague artist and I do a pretty good job of replying relatively quickly on the Twitter account if you'd like to get more information on pycharm please go to our website at JetBrains comm slash PyCharm we'd love to get your feedback on this webinar so please feel free to contact us on twitter or there's a survey that you're going to get as soon as this thing shuts off please fill it in we actually look at this thing and we will give feedback to Brian about points that he raised but it helps us determine what kind of the topics we should cover and what kinds of things I screwed up as a so please let us know I will also provide some information additional links from the presentation on the blog post when we announce the webinar which we will do in a couple of days all right so popping up Brian got any final words I'm just try to make testing great and it should be fun if you're it's not fun you're doing it right that's a good way to put it and come see us if you're at PyCon you better be in our damn presentation right yeah definitely all right that's all from us today thank you very much for joining us and hope you have a nice day

Channel: JetBrainsTV

Views: 48,078

Rating: 4.823256 out of 5

Keywords: python, pycharm, ide, jetbrains, testing, pytest, tdd, webinar

Id: ixqeebhUa-w

Length: 89min 20sec (5360 seconds)

Published: Mon Feb 26 2018