"How to build a unit testing library from scratch?" - Bhavani Ravi (PyCon AU 2023)

foreign is an independent independent software consultant building python-based back-end devops and Data Systems she also teaches python via online month-long boot camps she's an open source Enthusiast and has contributed to pandas and Apache airflow please put your hands together for povani thank you thank you thanks for such a nice introduction first of all hello everyone hello everyone okay nice good now we are all awake I know we are hungry but just to put things into perspective I'm bhavani Ravi I've been fallen in love by with python seven years back and it's still going on I'm an independent software consultant I get thrown into technical problems and I Rise Up from there that's what I do for living and today we are here to build our own unit testing Library and when I put out this talk and tell to all my friends that the talkers accepted the very first question was why Center why even your own testing Library and I don't have a perfect answer for that I have an answer for that one is it's fun remember when you all first started programming remember that moment when you've fallen in love with that first program that you wrote it gave you that spark and kept you hooked all along mine was printing the first 10 numbers and I changed the program to ten thousand and then to a lakh until the system crashed but there came a point where programs like that got really boring and now I can do this for fun and I hope you can too and there's this Rich environments quote what you don't create you don't understand and I pretty much love this quote and as a result of that I created own flask my own version of flask application and that was almost a year back it's been too long since I've had fun so now I am after Pi test the only problem was I've been using pi test pretty much extensively the last uh eight months and the hundred percent test coverage from the previous talk all of that was so relatable and I love all of these features that by test provides the test Runner parameterized test cases expected exceptions fixtures mocking you name them I use them and show all fans how many of you have used by test okay everybody almost everybody if you have not used it your logic will look something like this and your Associated test case which starts with test underscore followed by you call your logic and test whether that uh that the logic behaves the exact same way and that's a very simple test case this is your parameterized test case where you pass a bunch of data and hope that all the data that you're passing behave the same way as well and you can also expect your functionality to raise a particular exception and a pie test will capture that so I thought okay let's start building it and the first place to start is reverse engineer it and I went and looked into Pi test source code and this was the reaction I well it has about 130k lines of code earliest comment was on 2007 and so far there are 5000 PRS being merged for the project which is a lot more than flask flask was very rather simple what I'm trying to say with all this is It's hard to reverse and generate the next option was chargpt or GitHub co-pilot because we are all relying on that so much but things like this llms really work if there are previous examples available unfortunately even if you Google right now there are not a lot of examples on how do you build a listing library from scratch so again thrown out of all the options here so building my own testing Library reflecting whatever Pi test provides as an interface I'm gonna have all these features and let's see how that goes first starting with the test Runner I'm going to be live coding this and if I make any mistakes please be kind to me and the test Runner this is all our version this is not by this version so it has an interface that looks like this you do PI test followed by the directory that you want to test and if we think about it how would you write a test Runner first problem is finding out all the test functions that are there in all the test files under that for the particular directory and then you have to run them maintain a streak of success and failure and if there is an error you have to report them can we live code it okay I did say we are going to do things from scratch but I kind of lied so we're gonna start with this so here's my test function okay here's my own test module which is where we're gonna fill out things uh the test Runner is the only thing that's going to run everything else is a race an accepted reception parameterize picture which we will come back and fill in later here is my test Runner and some of the complex complexities I have written away using certain functions so I have got all the test files I've got all the test objects here and the only thing that we have to do right now is run the test cases so what do we mean when we run the test case we have a test uh sorry we have a test file with a bunch of functions we have loaded all that function into our module now we have the test function name and the test function object and running would running them would actually mean calling that test function object and if I run this so I'm going to use the own Pi test module and pass all the test cases all this functionality of reading the directory reading all the functions has been already written so if I run this it says success no address has been found whether it's really working or not I might not know so what I'm gonna do is in my case I'm going to make it 10 and see what happens okay there we have it our first assertion error so it's not just a matter of calling the function to run the test case you have to also maintain a streak of all your errors so let's wrap this in a very nice try and accept all right so we're gonna wrap it in a nice except and there we go so we're gonna keep a track of our exception and under here if there are any errors we have to print them so let's print it for wow life coding is hard so here we go I'm kind of cheating right here but it's okay so we are capturing all the errors we are getting a trace back and we're printing them hmm so okay we have something running here and then there is an error under test ad these are the this is a trace bag and where the session error is happening let's also give some more meaningful things so that we know something is running running desk keys test function name all right so we know that it's running certain test cases there are certain errors let's revert the spark that's our test Runner so now we have a test Runner which takes all the text objects runs them marks them success marks them error we're good to go the next thing is the easiest one of all accepted exceptions so looking at this by test.racist we are going to implement the races what can you make out of it the interface races is a context manager and context managers if you have seen it it comes with an entry and an exit block it is initialized with an exception type that you're expecting and when exiting the block you have to ensure that the particular exception that you have mentioned should be raised and if not we are going to raise an assertion error and oops not here yet okay so let's go to our PI test.races own pi test.races and this is my template block so right now to test this particular aspect of it I have a very nicely written test case right here so this is going to address type error if I'm passing anything that's not a integer to my uh ad subtract multiple divide so let's run this now it raises a type error and it's not being captured it's not being handled because we haven't implemented this Rises yet if you've perfectly implemented it then it would just pass through and Mark everything as access because the function is working as expected so at enter we don't have to do anything at Exit we have to check if the first the exception type is none or not if it's none no exception has been raised so that is there and if the exception type is not there then we are raising an assertion error both the cases Association error different logic but how is GitHub copilot doing all that because I have practiced this almost 100 times so far so assertion error raised when the exception type is none or if it's not matched so now let's run this okay that's weird since we have already handled the exception we also have to Market as true that you don't have to do anything just everything is nice and good so now it has ran the test cases it has also ran the test Rises and things are fine and well but is it really working well so instead of this zero division error I am going to mark it as typewriter as well now our function should fail because it's expecting something and resulting in something there we go we have an assertion error my yes raised typewriter has been raised instead of a zero division error so yeah kind of works there let's revert this back and life is all good nice and successful again good the next thing is parameterized test case again this isn't this this might look very difficult but it's also very simple or the drill looking at bytest.mark.paramel trials what can you make out of it what python construct it is a decorator and if you look at it it's not just a normal decorator it's a decorator with arguments oh scary when it's so first you are registering the parameters the second your uh you're registering the function and then you're gonna return a wrapper all of that has to happen and when once you have the function then the function is being called you have to run it n times so that all your subtest also runs and lock the results even if there is failure in either one of them you have to ensure that that is captured and finally you will get there so Pi tested parameterize here is my is going to fill it the keys are look okay first the test case the own tester parameterize looks something like this here I've used Python's eval don't ever use it in production and yeah so to the parameters I am passing like three four Arguments for me to test it and your keys are going to be this your values are going to be this first when registering the arguments I have to put it in a nice format so that I can just pass it as argument so let's make a test args list and here we are going to okay different than the previous one but sure next what we're going to do is we are going to when the function is being called when your test case is being called You're creating subtests so let's create I comma test ARG in numerate of test arguments then we'll print that we are running the subdes and also call the particular function with the test argument so when previously when we were calling the test function it was just calling that one particular test object right now what we are calling is the wrapped decorator so inside the lab tech Creator instead of directly calling the function we are calling the function n number of times based on the parameters that is getting registered so let's run this um yeah so you have running sub uh subtest 0 1 2 depending on the parameters if I go and add more it's going to add more if I change something here that also gets handled because we have taken care of assertion errors globally not you have to do you don't have to do anything particularly inside the parameterization so that is also good great all good now we have a decorator that takes arguments and a function looped over it all good to go the last but the scary part is fixtures this is a scary thing for me very easy to use very nice feature of Pi test but this one suck my brain out because I have to implement it all by myself and fixtures are great because you can do a ton of things from there you can define a whole large data set and have it available and multiple tests can use it or you can do your setup and tear down and there are Pi test comes with enormous amount of features this or to use scoping you can put it in your conf test and the variety no wonder they have 130k lines right they want to make your life easier so it's a decorator again and you can register a function as a fixture and when you are running the test case you have to ensure that you are passing the argument which is a fixture which in turn is a function which is decorated appeared yeah that's how I felt too handling test argument that is a fixture and then you have fixtures that are just data fixtures which is going to return dictionary or a list or a tuple or you have yielding fixtures which is going to yield and then do something later so all of these things are going on let's try to implement this so here is our beautiful fixture and that's it that's all you have to do for fixtures because you register a function as a fixture and your job is done but a lot of things that you have to do is in your test Runner you have to ensure that you are passing the arguments so far when we are calling the test function we are not passing any arguments so first thing that we have to do is to find out what arguments like parameters are there in your test function so this particular construct code variable names will give you all the arguments of a particular function and for args in test args we have to see whether that particular picture is in a fiction mapping but we haven't defined our fiction mapping yet so let's go to that here I have some logic to get all the test functions this does nothing but get all the members of the module finds out all the functions that starts with test underscore this time we are also going to check if members of one which is the object Dot name is a fixture wrapper because we have wrapped it in a fixture then we are going to store it in the fixture app picture mapping so great so it's in the fixture mapping then you call it that is your test argument let's also create a list of course this code is never going to be used or being production so all the solid yagni principles doesn't apply here list all right we are not going to use it test s dot append we are going to call the fixture and get the result put it in a test art list and gonna pass it to desktop list now when I run this it's gonna pass let's uncomment these test cases here I have a fixture that is getting registered and we have a a test con which is going to execute a particular connection which is also going to raise an operational error because I don't even have the database don't it's raising an assertion error which is weird because it's expecting an operational error but it's actually getting an attribute error what do you think is happening here Beyond errors like that has popped up and the problem is because of what's happening in con fixture if we go go here and print out what is actually getting passed when we are running the test case is you can see that it's a generator this is where I said it's an yielding function not a direct data fixture so we have to do something even more crazy uh again back to our test Runner here we have an argument which is the fixture we know that great now we have to get let's get the fixture object and we have to check if the fixture object has the next argument in it so that we know it's a yielding gen it's a generator if it's a generator we are going to call it and then store that value else we're going to call that object directly so that's the logic there are two types of fixtures one is an yielding fixture which is a generator the other one is a direct fixture object which just passes the data and we have handled both the cases now let's run this oh still an assertion error this has also happened quite a bit of time and still legit it's still a generator object now I'm confused any help I didn't call it yes great oh I don't call hmm it's a fixture object we're calling it and then okay this is why I was scared of doing this but okay I'm gonna add more print statements very classic programmer move a picture object what is coming here it's a wrapper right okay I'm gonna pull out another classic programmer move is have your backup code go through that see what it does and just go with it so here is my perfectly running fixture stuff I'm gonna copy paste that all right so we have the fixture value here we are going to call return on it or you're gonna directly call next on it or just gonna directly append it so that's there perfectly out of sync okay much better still a lot of things test arcs to pass all right thank you so yes that is our fixture which is a healing fixture next we call next on it if not if it's a data fixture that will also work so after a very so thank you for that I didn't I forgot to put the thumbs up over here so if you want to go through the same thing you can you can go to my GitHub repository and try it out on your own there is a blog post associated with it with it which needs an update since a lot has changed in the last three months and what can you do to have fun is you can try implementing mocks on top of this or try testing this testing library or testing your own testing library or build your own eggs and have fun as a part of it yeah most importantly don't forgot the first program that you ever wrote which was very fun and kept you hooked so far and if you want to check out my other works here if you want to talk about python devops data data Ops I'm here thank you thank you we have some time for questions they're all too hungry oh just up there so I have a question regarding um once you have the test completed usually you end in a big trouble analyzing the test because usually you just need to look at the assertive statements but when things grow a bunch of things you have hundreds of tests and analyzing that is quite difficult so did you have any strategy once you have everything completed just for example instead of doing the assert racing specific assertions or custom exceptions do you have something in mind quite capture that question exactly what you when you are once the test is completed and the analysis of the test is a bit difficult so do you have any strategy for that all right so assertions I don't know whether this is what you're asking but if you do it with by test this is a pi test version of test case that's here um yes so when you do it with pi test I'm going to change things over here yeah any number would work and I'm going to run it with by test I have printed it pretty badly right because we are building it live but when you do it with whiteest it's gonna show you exactly where that error happens and it also has a very nice Trace back that you can print with more verbose things so it doesn't matter at what level the error occurs you will have a complete race back of things and there are options in pi test to do that I think I when we do this case it's just an assertion error happening at that particular line but if you have it in your logic there also it works so let's say you are expecting five here but instead of add I'm gonna change this to oh let's do something fun 10 always then it's also going to give you that whole Trace back yeah so here it says you know assert on the left you get 10 but on the 5 uh if you're making any particular logical error that also comes as a part of the price back okay thank you very much um no other questions thank you what do you think is the most useful Insight you've got about the internals of Pi test that you learn from this re-implementation is there some part of the internals that you know you wish people better understood oops I'm gonna go back to the slide where I was pretty overwhelmed scared uh it was very hard to even just look at it or understand it because it has built for so many different use cases even something as simple as fixture it behaves completely differently to what we what I have built so I have stopped drawing inspiration from pie test way early in the process and just built up my own version so that's hmm thank you for your talk could you talk us through the entry point of your code how it iterates over the test files finds the test classes finds the test methods and then works out what to run sure so it's pretty but sure all right that's my test Runner and it gets test files and then a bunch of test functions so test files is nothing but reading reading the directory if the directory does not have test in it you skip the directory and if it does have in it check get all the files that starts with tests and ends with pi and then return them so that's all the test files inside any particular directory of course I have done it for only one level and for the test function I'm importing the whole module like the whole file and going through its members using whether it's it's a function and if it if a function starts with test underscore that is a test function and I'm passing them back that's that help thank you okay thank you um thanks pavani for such an engaging talk um it's a token of our appreciation thank you thank you uh one small thing a very thank you to pycon Au they have sponsored me to come all the way from India to present the stock here thank you so much from the bottom of my heart

Channel: PyCon AU

Views: 492

Keywords: BhavaniRavi, pyconau, pyconau_2023

Id: gjUzPGKSuDM

Length: 29min 42sec (1782 seconds)

Published: Thu Aug 24 2023