Test-Driven Development In Python // The power of red-green-refactor

the core idea of test driven development is that you write your tests before you write the actual code today i'm going to show you how that works and what the benefits are but there are also some things you need to watch out for if you're new here you want to become a better software developer gain a deeper understanding of programming in general start now by subscribing and hitting the bell so you don't miss anything in this example we have an employee class that has a name and id and some fields related to payments i used the flow type for payments i think normally in a production environment you wouldn't use this because it's not precise enough you'd probably use decimal or int or something like that so next to pay rates there's hours worked a fixed employer cost whether the employee has a commission what the commission is and how many contracts the employee landed and then there is a compute payout method so you can imagine that you have some kind of hr system that calls this method for each employee and then computes how much the employee should be paid so i haven't implemented compute payout yet we're going to do that later and follow the process of test driven development test driven development or tdd is a term coined in 2002 by kent beck in his book test driven development by example i'll put a link to that book in the description below test driven development consists of five steps the first step is that you write tests when you add a new feature you basically start by writing the tests that only pass if the feature specifications are met and this is actually a very important benefit of test driven development that it forces you to think about the requirements before you start actually building something the second step is actually running those tests and make sure they all fail it's important that these tests fail because you want to check that you're actually adding something new and that the tests are properly testing that part the third step is writing the simplest code so that these tests pass it doesn't have to be perfect in this step you just make sure that it adheres to the specifications you're going to clean it up later the fourth step is to make sure that all tests now pass including any other older tests that you have this ensures that the new feature adheres to the specifications and it doesn't break other things and then the fifth step is refactoring and improving the code while you have the test harness running so you make sure that you always have passing tests while you're doing that so this is the main process of test driven development and you can repeat this cycle of writing new tests making sure they fail writing the code making sure test pass and then refactoring over and over again and if you're working in an agile setting you should do this in small incremental steps and make commits very often the process of writing failing tests first then write the code that makes those tests paths and then refactoring the code to improve it is also called red green refactor so let's apply that to our example and see how it works i've already created a basic structure of the test that we're going to use in this particular example there is a test employee compute payout class that aims to test the compute payout method of the employee class there's already three tests in there one that tests that the payout returns a float one that the payout is correctly computed if there is no commission and no hours worked and one if there is 10 hours worked and no commission there is a setup method that creates a test fixture that's basically the data that the test uses in this case each test needs an employee instance now you could also create this employee instance as part of the test but the setup method is a nice way to make it a little bit more concise next to setup there is also tear down if you need to clean up things after running each test for example maybe a test generated files that you need to delete or there's a database connection that you need to close things like that remember setup is called before every test so in this case we have three tests so this is going to be called three times one time for each test so i'm using the assert methods of the unit test library to check things for example here use assert is instance to check that the result of compute payout is a float i'm using almost equal to compare the payout with a constant the reason i use almost equal is that there are some rounding errors with floats and i don't want to have my test failed because of a rounding error so again in a production environment you'd probably use decimal but for simplicity i'm using float here so the first two tests simply use the employee object that's created by the setup method in the third one we're changing it a bit because we're setting the hours worked and then doing the check so let's add a few more tests here one thing i'd like to test is if the employee receives a commission that this is computed correctly so we're assuming the employee had 10 contracts that landed and 10 hours worked so let's set the contracts landed value to 10 and then we assume that the payout is going to be the commission default is 100 so we're expecting this to be three thousand and let's add another test to make sure that disabling commissions also works as expected this is what we get and then the expected value is 2 000. so now we have a nice little number of tests so this was the first step of test driven development writing the test second step is running the tests so let's run these tests and see what happens so now we see we get five errors all tests fail and this is important to check that we're sure that the tests that we're adding that they're actually testing new functionality now the next step and that's when we're moving to the green part of red green refactor is that we start implementing the method in employee and then you'll see that step by step these tests are going to pass so let's start implementing the compute payout method so i'll remove this not implemented error and then let's start writing the code so the first step let's compute the payout and that's simply the pay rate times the hours worked and for now let's just return the payout so this is not complete because we're not taking care of commissions we're not taking employer cost into account but let's see what happens when we run the test again so now you see that some tests are starting to pass we only have four failures instead of five let's add the employer cost and let's run the test again now we see we only have one failure and of course what's still missing is the whole commission stuff so let's add a line that if there is a commission then we add that commission times the contracts landed and now let's go back to the test and now all tests pass so now we are in the green phase of red green refactor now that the tests pass you have an environment where you're sure that your code adheres to the specifications and you can move around things and change things and make sure that this still works correctly for example what you could do is decide that employer cost is too generic and you want to split that up in a few different variables so let's say we have employer office costs which is 200 per month we have employer 401k costs which is 400 and we have employer sport costs which is another 400 dollar and we're going to remove this and now let's change the compute payout method to take into account these new costs there we go so now we refactor this a little bit and now let's run those tests again to make sure we're still adhering to the specification and we do this is how red green refactor works and to finish this off i have three tips for you on how to write good unit tests and a couple of do's and don'ts but before we dive into that let's first talk about the pros and cons of the test driven development cycle why is test driven development so popular well first and i mentioned this already in the beginning of the video it forces developers to think about requirements first and that means you need to make sure that your feature is defined well and that you know exactly what it needs to do because otherwise you can't write the tests this forces developers to think about their interfaces first as opposed to just starting coding away the second benefit of test driven development is that even though you probably have to write more code it might save you a lot of time overall because you're detecting bugs much earlier and you probably know that if you're debugging your code this might sometimes take a long time before you find a problem and having those tests there is really going to help you find those bugs sooner and that will save you time finally a third benefit is that if you start by writing tests you force yourself to write code that's easily testable and that means you'll be more inclined to use design principles and design patterns to write code that's easy to extend and easy to separate from other parts of the application and that just makes your code in the end much easier to maintain there's also a few things you need to watch out for first is that writing and maintaining lots of tests is going to take time if you're working at a stable company who knows what their product is who knows what their users want and this is not particularly an issue because anyway it's going to help you with debugging faster and keeping the system stable if you're working at a startup though in particular an early stage one then adding all those tests might become a big overhead and it's going to be a waste of your time if any way you're going to completely change your product and throw everything away that you did so you have to find a careful balance there the second thing is that if you've written many passing unit tests you might have created a false sense of security for yourself and you may not put as much effort as is needed in other testing activities such as integration testing or end-to-end testing and finally often the developer that writes the test is also the developer that writes the actual code and that may lead to blind spots so it's always a good idea that especially if you are in a team that you separate the work of writing tests and writing the actual code between different team members because then you have less chance of developing these blind spots another possibility if you don't do that is that developer misinterprets a part of the design and implements the feature in the wrong way but then also test it in the wrong way and then you won't find it out until perhaps it's too late or the client starts complaining that it doesn't work as it's supposed to let's look at a few things you shouldn't do when writing unit tests here i slightly modified the test that we have and introduced a couple of things that i think you shouldn't do the first thing you should watch out for is reusing the same data in multiple tests ideally you want your tests to be completely decoupled completely independent from each other you don't want a change that you did in one test to affect the result of another test because then it becomes extremely hard to debug what i did here is create a single employee instance that's used for testing and then in the various test methods this is the employee instance that i use and that's a bad idea because now changes that i do here in this test have impact on what happens in this test so if i forget to reset one of the values then we have a problem so overall don't use these kind of global instances uh and and reuse them in different tests that's the first tip the second tip is that you don't need to test python built-in functions i added here a test that checks whether employee attributes are as expected so what we expect is that the name is name the id is that pay rate is that etc etc and then i'm comparing these values with the values that are inside my employee to test instance you don't need to do that because this is already defined by the python standard library which normally you can expect won't have that many bugs so you don't need to test that the goal of your test is to test the compute payout method and not python standard library stuff the third tip is if you write a test compare your output with a fixed value what i did here is i created a compute bailout method as a part of the test where i compute what i expect to be the payout and then what i do in some of these tests is that i compare the compute payout method result with calling that test implementation on the same employee the thing is that here i basically made a copy of the compute paired method that's inside the employee class so in this case i'm just comparing that copying method gives me exactly the same results which is weird right so it might be tempting to copy over parts of the implementation to the testing class and then test the implementation against the reference implementation but then you're simply comparing implementations you're not testing the actual code so always compare with fixed values in my startup company we've been adopting a test-driven development approach lately and it's quite pleasant to work in that way so i really recommend you try it writing those tests does take time especially in a startup you have to let go sometimes of trying to reach high coverage percentages it's a careful balance between shipping something fast and buggy but getting useful feedback from your customers quickly versus spending a lot of time on feature and releasing it and then it's very stable but you might not learn as fast as you would have needed the nice thing about test driven development is that you can make that trade-off explicitly because you can see how much coverage you have and how much tests you've written in this video i only touched the surface of software testing there's a lot more to think about like mocking patching different testing frameworks different types of tests etc etc and i'm surely going to do more videos in the future to cover all those things here so that was it for today thank you for watching take care and see you next time [Music]

Channel: ArjanCodes

Views: 30,571

Keywords: test-driven development, test-driven development with python, red green refactor, test driven development, red green refactor technique, software engineering, software development, test driven development example, test driven development in software engineering, test driven development python, software engineering course, software engineering tutorial, unit testing, software engineering for beginners, software engineering full course, tdd, unit testing python, Python unittest

Id: B1j6k2j2eJg

Length: 15min 9sec (909 seconds)

Published: Fri Aug 13 2021

Reddit Comments

[u/arkster]

Your content on python is pretty cool. Thanks.

[u/sangazul]

Love this guy! The code he presents is challenging and elegantly advanced that even novices will learn other things they were not expecting.

[u/bixmix]

There are so many different paradigms for development.

I have found TDD to be most effective for refactors - especially rewriting code in another language while keeping the functionality the same or similar enough.

However, it really does not make sense at all to do TDD when developing something completely new. In this case, TDD actually causes the development time to increase considerably and if the code that's being tested is not actually going to be kept (e.g. the approach was a bad one), then it was just a waste of effort to build the tests first. For new things I generally prototype the code, execute the code to see what happens and then write tests around it. The final piece is to document so that my future self knows what I tried, what didn't work and why I have the current code. At each stage (prototype, execute, test, document), I am asking the question is this what I really want the code to do. Is this really the best way to present the code so I can understand it later and maintain it? And this approach works exceedingly well for new things because what I want is quick feedback loops to know if my approach is a good idea.

I also think language/tooling is important. Python in particular requires more testing on average to show correctness.

[u/doa-doa]

Can you explain why you should int for counting finance in a program? I get why float is inaccurate because it has this weird behavior like for example( 0.1 + 0.2) doesn't produce an accurate 0.3.

But why int and not decimal? Why do you do when you have.... well a decimal number like $ 1.99 ?

[u/irrational_design]

Personally I think BDD is better. It is similar to TDD, but focuses on what is actually important for the program to be usable by the end users instead of focusing on the developer's code, which the end users don't actually care about if it doesn't do what they want it to do.

[u/TentativeOak]

Love your content man. Big fan. I watch them during my morning routine

[u/emmabrenes]

I find your videos so useful that even some topics like the design patterns look easy. Thanks to this I've started re-learning them with a lot more confidence. Keep the great work, Arjan!

[u/mothzilla]

Nice video. TDD seems to assume that tests can be run very fast. In my experience eventually this stops being true so TDD becomes hard to do effectively.

[u/skibizkit]

What was the formatting shortcut used around the 8:30 mark? Created a multi-line assignment.