Unit Testing Best Practices with Roy Osherove

you get really comfortable with it some point you just can't do it without it okay those of you who haven't been here my name is Rajesh Rove I'd love to have you over for a course I'm running during August if you want you just go to this URL why does windows always have this like when you do a presentation I'm thinking my book just came out anyone actually just bought it wow that's cool thank you can I get a copy okay it took me three years to write this book and two kids I started this book book with no kids now I have two and this book took longer to make them both of them and somehow it's much more quiet in fact it took so much lot time to write this book that I actually changed my mind about some things during writing it I had to go back and change them in the actual book I won't tell you what it was though okay I tell you what maybe later this is the second talk in our unit testing series the TDD stuff this is not just about how to write unit tests or how to write them tests first this is how not to write them or how to write them well takes a large portion of the book as well I think it's a really important chapter and I think this is one of the things that most people don't implement there are three pillars to making unit tests good and when I say good I mean three things you trust your tests tests are trustworthy you are able to maintain your code and your tests and it doesn't take up too much time that is you feel that you get more value from the tests and maintaining them does not hurt you and the tests are readable if they're not readable the rest actually falls down as well these three things are connected now to make you test trust trustworthy there are several things that we can go through let's just start with the the simple stuff in my class when I teach test-driven development I say start with a failing test and people say okay let's write a failing test and they write something like this they say they call some with one and two and they expect to get four of course this test will fail and now they have a chance to go in write production code but of course this test is wrong this is an obvious way of writing a test that you cannot trust it's just making really really obvious but I actually saw people do this and this is just lack of experience they said write attested fails but it's not just writing a test that fails the test has to fail but then pass when you change the production code so if you start with a failing test but then you change the production code and the test keeps failing it's probably an indication that the test might be wrong or if you start with a test and you can't get it to fail probably an indication the test is wrong and one of the cool things about TDD is that it helps you test the test just imagine having a bug in your test how long would it take you to find it the interesting thing about it is that you have to see a test fail and then you see it passed that is you see the test in both situations so you know that the test is correct if you write the test after because we're programmers we're not objective about what we write we write tests that are supposed to work we cannot trust ourselves with test-driven you have to write a test it fails so that you will see it in that situation so that you know that it fails when it should and passes when it should that's one of the reasons that I think TDD actually makes for better tests because you will have tests with less bugs you will find the bugs much early when writing the tests as well because no one will write a test for the test right because that tests could have a bug as well and so on recursive descent but these things can actually be much these bugs can actually be much more subtle I want to show you an example of a bug in the code that we wrote earlier now if you weren't at the previous session don't worry it's a very simple piece of code I'm going to explain it we just have a method called some string it takes a string of numbers and it handles 0 numbers and handles one number perfectly well I think we don't have enough tests yet and there are other methods here but I want to let's assume that we have a method here that we wrote I'm not going to write the test-driven just to make a point public string create non string int I int X or is it indexing Y in twice on much better and what this function returns is [Music] what was it is a string format it would be the first number comma the second number so we have 1 and 0 and we have x and y okay so this function takes two numbers creates a string with the comma between them that's all I want forget about why I want it I just want to show that even the simplest code and that has tests those tests can have bugs very very subtle bugs now let's say that we are writing a test for this code and we write a test that looks like this and I'm going to use my naming convention because we're going to cover it anyway let's say that the name of the method under test is create num string now the scenario is two simple numbers so behavior returns string with comma between now this test name may seem a bit long but remember the tests are also API documentation so I don't care that the name is long I care that it's readable I want people to be able to read it like a sentence and yes I have no problem doing this for exam or doing this for example this is not a regular production code this is should be more readable than production code because bugs here and on misunderstandings here will lead to bugs that are much more serious in production code okay now I would say string calc SC equals new string Cal SC create num string and I would send in 1 & 2 and let's create a result and assert and a lot of people do this are equal and I'm expecting what would be that number let's just go here and do this [Music] 1 & 2 and the actual string the result nothing wrong with that right it's perfectly legitimate test right I'm sorry what I just heard him know why not who said that why not okay I duplicated logic but it's very simple logic what's the problem what's the problem I mean seriously this is what people would tell you what is the problem I mean there's obviously not a bug here right so can anyone tell me what the problem is of course duplicating the logic is a problem why is it a problem sorry I'm not testing it and I have to refactor the test as well well I'm not testing it but I am testing if I'm writing it in a test oh yeah okay I'm testing the the logic that I think is correct isn't it what I think it is did you get that I didn't check the end result I because I repeated the same logic in the code inside my test the bug that I may have already created is inside my test as well so I will always get a positive result even though the bug exists because I'm actually expecting the bug to exist here because I recreated it in my tests so recreating the same logic from your production code King your tests brings a high possibility of a real bug even if it's simple code it's much better to use Const s-- or end results for example I would say um 1 & 2 I don't care how you got 1 & 2 I just care that you got it I know this is what the string needs to look like ok now if my code has a bug and I get a different string I will know about it because what I care in unit test is about the current end result so by default unit tests should not have logic and as far as I'm concerned creating or appending strings is just as much logic as loops switch cases and all that stuff now this gets even more serious if you think about people and I've done this a lot of people do this they say okay I want to test this I'm going to create a random number generator and I'm going to take two numbers each time and I'm going to create the string that's supposed to happen from those two numbers and assert that it's equal every time that sounds cool right but remember that you will still have to recreate the same logic in that test just imagine that the two numbers here x and y are created randomly that also means that you every time you run this test you get a different test your testing with different numbers but of course the possibility of the logic itself having a bug is very very large so I wouldn't necessarily say that random numbers are bad inside tests just that they're not really unit tests as far as I'm concerned and write them but only after you write the regular ones like the simple ones when you know the logic is okay and you just want to test a range of values and all that stuff okay very simple stuff let's do this a lot of people stopped trusting the tests because they removed tests or they change existing tests when they shouldn't do that for example someone writes a test the test passes so say okay now let's make this test much more let's say complex so they go to the existing test and they start changing it and of course this is a problem because once we have a test and it passes I don't want you to to remove it or to change it because that's that test should still be there even a year from now as far as I'm concerned to always test that this specific piece of use case still works so think of it about it like tracks and well I come from Israel you just say tracks in the sand but I'll say tracks in the snow for you guys so think about like tracks in the snow test you always want to see all the tracks that you left behind you always want to recreate your steps so that you will not miss even one because even if you miss one there could be a bug there so never change remove tests except these specific cases one of the only times that you remove a test case is when it's invalid if you do test-driven development you start with a failing test you make it pass and then you have a requirement that actually passes you have a requirement now you get a new requirement six months later and you and you write a failing test you make it pass and suddenly as that new test passes some old tests that you don't even remember writing fails and you realize after playing with it for a second that these two tests cannot live together they are opposing tests so they represent conflicting requests conflicting requirements and this is a good indication for you to go and ask the person gave you the requirements which one should go one of them is probably invalid they cannot live together most of the time that's what happens you can change tests but without changing the functionality for example you can change the name of a test to make it more readable you can refactor the test to make it more maintainable but it still needs to do the same thing that it did you don't want to change the functionality because you are actually changing the test itself you're changing the use case being tested the only other times where you are forced to change a test without just wanting to refactor it is when the semantics of using the code have changed so let's take a look at the string calc class let's say that I get a new requirement that says before a crew before working with create calc with calc string and every time you need to call a special init method it's a special requirement if you don't call the init method than you call any other method you're going to get an exception what does that mean about all the other tests that I already have they won't work they will fail by default but they are still important they still test the right things but the semantics of using that class have now changed so I will need to go to all those tests and start checking start making them call the init method at the beginning now this is also a maintainability issue and I will cover that but this is one other case I won't show it to we don't have enough time so only have an hour but this is one case where you need to change tests for most other cases there is no reason there should not be any reason for changing or removing them at all there are code coverage tools out there if you if you want to be able to trust your tests you want to know that they have good code coverage so if you have tests 1200 tests in your code but you're only covering 20% are you going to trust your tests they're all passing does that mean anything to you it means something it doesn't mean you should stop debugging though but if you have 95% code coverage with 1200 tests that means a lot so low code coverage means something means you need to do more code coverage high code coverage also means something but only if the tests are good because I've seen tests to do a lot of code coverage but they don't really check anything they just execute the code so these two things have to live together so I'm assuming those tests are actually testing the right thing and so on so now code coverage actually does matter so there are code coverage tools out there ms test has the built in code coverage and there is n cover there are other tools anyone here use other tools than those two that I just mentioned for code coverage anyone just shout it out I want to hear about other tools no one okay who actually uses one of those two okay not enough people you should start using those either n cover ms test very very popular very simple tools one of them comes built-in one doesn't but what I want to show is a different technique I want to show a technique that I use in test reviews and test reviews is something that I do every time someone checks in coded at our company I mean seriously you cannot check in code without going through a review doesn't have to be by me though but this is one of the practices that we that we implement and that makes sure that every code gets gets their share of respect to to see if there is any what what I call crap edge now tests can be really really crappy if you let them and people need to learn how to write good tests so tests need to be reviewed as well and all the things that I'm going to say here also need to be reviewed but one of the things that I want to make sure instead of just running code coverage is I actually check things manually so I will go into various logic places such as ifs or loops of Const and start changing them just change one at a time run all the tests I'm putting an obvious bug there something should be failing right now if all the tests still pass I know there's a problem now why do I do it manually why not run using code coverage anyone tell me as I drink this the reason is it's it's more intimidating that way well actually it makes me look at the code more because I do code review and and the test review as well I want to see what's being tested how it's being tested the structure of the code as well but also you pay more attention you just don't just let it tool take all the decisions for you you want to see code quality you want to see how things mix together sometimes you see things that even code coverage doesn't catch such as testing things in the wrong way here's an example now I have test for string calc if I go to create num string not create number let's go to some string now there there isn't a whole lot of logic here but the simplest thing I could do is to go inside the if here and change this to true an obvious bug run all the tests if nothing fails oh dear God do you have a problem you have a problem with me that is something fails that's okay I'm going to change it to false and see what happens I'm going to to move this line to be the first line so Dave never happens just to see what happens run all the tests it it does make a difference it makes you pay more attention as the reviewer and I think it's a great technique one other thing is that there used to be a tool for Java that used to do this automatically is to mingle your code and see what happens I don't think there is specific code for that in dotnet I would love to be a to be taught if there is something like that so if anyone knows something either shout it out now which I know you want because you're Norwegian but you can send me an email which will be on the slides you see I didn't even wait for you to shout so avoid test logic I started talking about this earlier there should absolutely be no test logic in your tests if there is an if you're probably testing more than one thing if there is a switch you're obviously testing more than one thing now the reason is because it's very likely that you're going to have a bug there I say 95% even for simple tests why 95% because Murphy's Law says that if it's a really simple test and this is like any say you know what screw that I'll just put in some logic this time 95% that that's going to have a bug okay it just happens I don't know why it happens this is from experience so save you save yourself the trouble and of course if you were doing test-driven development you would have found that out very easily because either the test might have a bug though you would have found it or you would write a much simpler test we test driven you write tests that are much simpler if you write the test later usually write tests to check more things and of course there's the logic that is being recreated from the production code like I showed earlier this is something really important you have to separate the unit tests from the integration tests if you want developers to trust the test results it doesn't come easily to realize this but this happened to me so many times that I realized that that's the only way so just imagine a developer getting the latest version of all the source code from source control and they're right clicking and running all the tests in the solution and some tests pass some tests fail now the developer who wrote the test that failed goes to you and says what do they say come on it's supposed to fail you didn't configure that that's not connected oh that's a problem don't worry about it now it only fails every second time if it's the third time that it fails but that other test passed you're okay if that test passed but it's a full moon there's a problem if you hear that any one of those including the full moon there are applications out there they have to do with astrology I'm not kidding what a full moon is like a leap year yeah you have to take advantage of something like that in software that has to do with something like that these are problems in tests now you solve these problems by writing tests better and isolating them better but regardless of that when you have integration tests that is tests that are not necessarily only running in memory or the touch external resources or that run really slow or that use threads where they're not repeatable then you probably need some configuration to run them and by default you don't have that so you may think that the test is failing for the right reasons it's failing because you didn't configure it but remember that developer who got your source code and started running your test and now it fails you can tell them it's okay they will not trust the test anymore because it fails but you think it's okay so you need to be able to trust your test you want to know that when it fails it should fail and when it doesn't it doesn't so at least at the very least if you have tested need configuration by default put them in a separate project and that project I call the integration tests and all the unit tests go in a different project called dot unit tests in the end and now the project with the unit tests all the tests there should pass by default it's always supposed to be green I get the latest version and I run them and if one fails I know there's a problem there should not be awake a it's supposed to fail this is a unit test because I specifically separated them so now if it fails I now have real value I actually believe the test so it the very least separate those and those that need configuration put them in a separate project and put you know all those explanations of how to make it pass and all that stuff so at least you have that safe Green Zone this is what I call and that that kind of stuff where developer can run them and feel at least partially confident about those specific tests they can believe they can trust those tests that's about trustworthiness what about maintainability one of the things that I dislike doing is to test methods that are not public not because Uncle Bob sets up but because ultimately it's a practice that will deteriorate your test they will make very fragile tests it's not productive now I'll explain how that leads to better design in a second bula how long do we have just to make sure not five minutes right 30 minutes okay you remember if you have five minutes left you have to start dancing go like this because lasts less time he was you're not getting a shirt okay so why should I test public stuff public stuff by default is a contract you say to the outside world this is how I should work this is these are the messages should use if it's private or protected it's changeable because no one should be touching it from the outside so that means that if I write tests by default against private or protected stuff it's more likely much more likely that my tests will fail for the wrong reasons I changed the name of the method I change the parameters I refactor it to multiple different methods and suddenly the tests now need to change because the semantics of that method have changed it's just going to happen much more than with public methods so just for the productivity reason it's not a good idea second it could be a design smell because if you really want to test that method maybe maybe the test is telling you that method really wants to be public maybe it should be static maybe choice should be in a different class I don't think it's a bad thing that the test tells you that something needs to change in your design I just think it should be your choice whether you want to change it or not I don't want it to be a necessity and that's one of the main arguments about the whole idea of design for testability and all that stuff how much do you let your tests change your design or how much do you let your test force your design that's the main argument that I that I hear most of the time and a lot of people and I think I agree a lot of people think that if you have such a smell you probably want to change your design but there are times where you can't and that's where I want you to have the choice that's where things fall apart in the real world you can't always write new code you have legacy code and that legacy code has requirements or you have security requirements you have optimization requirements whatever and you can't always do what the test really really yells at you to do you just can't there are always ways around this the design can always be better and I think it can be made better and refactor to be better but you need tests to make that happen so you can even even do integration tests and all that stuff before you do unit tests and then you can change your design so that's why I specifically wrote sometimes there there isn't any choice about this and from consulting to large companies and small companies I think it's naive to say from now on I do test driven everywhere and all my design is solid design I'm not saying I wouldn't love that to happen I'm saying that in the real world you can only do incremental changes if you're doing a new project I think you can do this kind of stuff if you really knew how to do it but on an existing project it's going to be much more difficult to do and that's where you might want to start with just unit testing or just integration testing and then changing the design and all that stuff incremental work is the key with any with any place where you don't specifically have choices about the design stuff reusing test code is one of the key pillars that make tests maintainable I showed an example of I'll just show you there the code that we wrote in the previous talk the code that I wrote here may be readable but it's hardly maintainable if you'll notice all the tests here can you guys in the back read the code if you can't yell we can't read and we can't read because it's too small not because you can't read okay you can't hear me can't you okay so this is a problem because I'm creating an instance of string calc in all those tests so if I were to go to the constructor of string calc and add a parameter I would have all these tests breaking now there are several ways to fix this what most people do is they use a setup method and this is part of any test framework you have set up set up set up attribute public void setup doesn't really matter what the name is and whatever code you put in the setup method is going to be executed once for each test so if I have five tests here the code in the setup method will be run five times when I run all the tests in the class so you can refactor into say string calc equals new string calc and in the setup initialize it okay so the difference is now I have the string calc as outside the tests and if you think about this like a constructor that runs before each test each test will get a new instance of string calculator which is very good because it means that each one of them does not touch the state of string calc that the others will have so the tests are more isolated this way now setup methods are great for this specific use because in this case this is the class under test so I can just go ahead and remove all the places in the tests where I have SC calc I do this once that's why it's important by the way to start doing this refactoring as soon as you are using this thing more than once do it on the second test and you won't have to go to each line and test and start removing that kind of stuff but what happens when you have another class that's being used but it's not being used by all the tests this is nice because it and it's still kind of readable because we ever reads the test code we'll look at the setup and say okay so you have a string calc for all the tests but if some tests use another object and some of them don't and you initialize it in the setup method you're going to get some garbage that person who reads the test is going to have a problem because they're going to see the setup method and they're going to think that the second object is also being used by all the tests and they're going to start searching where it's used where it's not used and so on and suddenly the tests become very unreadable very confusing and you want to prevent that from happening so the second option if you want to create objects that are not used in all the setup in all the tests is to use simple factory methods so imagine if I was to just refactor this into a extract method here and say create default calc now I actually have a naming convention for that calls make underscore the reason for having a naming convention here is to say this is a method that creates an object so it's like a naming convention because you can have multiple helper methods in your tests so you see all the make methods in the intellisense as well one after the other and so on so it's easier to read the code now I don't have to put this into setup anymore I can just go and do this VAR SC equals make default calc so now I'm I'm not creating it in the setup I may create this directly in all my tests but it's more maintainable because the the call to the constructor happens only in a single method so if I change the constructor I only have to change a single method and all the tests will suddenly just magically work so that's how I fix things by refactoring the code now there is a fine line between readability and refactoring you can quickly get over that line I recommend that you make the tests more readable than maintainable if the choice ever comes across okay for example if if the test is not understandable just because he created a factory method I wouldn't recommend creating the factory method I would recommend taking the head of the maintenance because being able to read the test and understand what it does is I think even more important I mean than the maintainability because if you're not able to read it even if it's maintainable you won't be able to maintain it because you won't understand what it does so one of them has higher higher order it doesn't happen that often I almost never had to deal with this question of how much is too much refactoring mostly what I'm going to show here is perfectly readable so creating objects is one type of helper method now let's imagine that case where I said what happened now you need to create a call an init method on string calc before calling calling it so I can just put that in the make default calc if I wanted to call the init method on it and now all the tests by default call it but maybe only some tests need to call in it in some don't so what would happen then I can't put it in the default calc because then all the tests would be having the same behavior right I would say oh I'm doing a recursive kind of thing here that's nice new string calc so this happens here VAR SC equals make different calc so this is what my test would look like so if I was going to do this as C dot in it let's say it's a method you have to call it before doing any other operations if I do this now all my tests by default to get this behavior I don't want all them to get this behavior so I can have a specific method that I would just change the state of the object to a specific way rido I'm returning the wrong objects there you go wow you guys find bugs you're shy but you're good I like that I like that a lot this is pair programming isn't it what should be avoid I see is an attribute oh it's a field I don't care if it's field right now but this is definitely a test code smell who care it doesn't really have to be field I can just initialize it like this right and of course every every little thing like this helps to create a more confusing code so we just cleaned up the code and you know just because this is a presentation I'm excited I keep screwing up but you guys catch me like that okay what else now I have to make all the tests compiled because you told me to remove the field thank you okay I'm going to do undo now okay and what else I'll just do this s equals make default calc there you go what do you think of that okay now some tests need some conflicts of state and I'm going to continue to try and do this let's say that some need to call in it I would say in this test after I got the instance of string Cal I would maybe have a method that says change state or some form of a specific method now I usually call this method in it but because the example also changes a method calling in it then it would be too weird if I would just go change call in it on SC and calling it is just a logical step initialize and this method would be just a helper method where maybe two or three tests out of ten are using then string calc would would change its state but only for those tests but whoever reads the test will understand that I'm changing the state or specific way but would still be maintainable I would have helper methods for creation and it would have helper methods for changing state of an object now the last case is to have helper methods for assertions how can we you factor this into assertions if you look at all these tests they all look the same well these are lengths without spaces let's look at the some string that we had we have the some string method here we only have to test for it but I want to refactor them just imagine we have more so we have one that creates a call some string with an empty string expect zero we have one that calls some string with a number expects one now I what I can do is I can say let's make a more generic method that takes parameters now if it's if it's more generic and it contains in a certain side it I would probably say assert underscore some string results now I'm going to refactor this so that instead of sending hard-coded values it's going to take parameters I'm going to use resharper here if you use resharper this might be interesting to you I can extract a parameter so I can say make the empty string a parameter collect numbers and make the expected result into a parameter as well call it expected now instead of this test being two lines I can just say assert with an empty string that you get zero so the test becomes one line now this is this makes the test somewhat more not only maintainable a bit more readable but it's I think this actually might make it not as readable as I would like so at the very least I would say that I would call this the expected another not a variable sorry make it a variable here ain't expected so that I know what this is and make this a variable as well string numbers and the reason is I want the reader of the test to understand what's being sent now in c-sharp for o you'll be able to specify the names of the parameters in the method call itself so that you can make the test more readable that way if you were programming in vb.net who programs in VB as well by the way some people here okay I'm a VB dev by the way I mean I started in VB five and six now I do both VB has a cool feature where you can already name your parameters by the way so you can make this test more readable by default already today but in c-sharp you have to do this right now so now I have all these tests calling this specific method I can do the same thing here and if by any chance the way that I assert or I call some string has changed I only need to change one function so now I can have these two tests but there is further refactoring that I can do if you are using n unit or MB unit and unit has a feature that's called row tests anyone here ever use that okay maybe just one person two people a row test provides the next step in this kind of thing let's say that you have the same test and you want to send just different values every time what I can do is instead of making this calling this from other tests I can just say let's call it a test case this is a new attribute in nu 82.5 and in the test case you just specify values of arguments so the first value will be mapped to the first parameter of the method so I'm saying that an empty string I'm expecting zero and here's another test case with just one I'm expecting one and here's another if I really wanted to one and two I'm expecting three and you can give them names and descriptions as well and this way I don't even need this or this because this specifies the same idea take a look at this clean cleanliness and simplicity it's readable it's understandable because it's an attribute you can actually give names to the parameters here so you can make it even more readable and this actually counts as three tests if I run this using either end unit or test-driven net you can see that the third failed to past one failed the one that failed is this one which is obvious because we still don't handle two numbers we only handle zero or one number we worked incremental II I think that's about it for refactoring and reuse I think that's pretty cool now if you're using MS test MS test has a different thing it's more powerful if you're doing a lot of range checking you can kind of use it for the same thing there is in any in the MS test there is a property when you generate a new test when you create a new test class there is a property that's generated as a comment called test context now has anyone here ever used that yeah you up there okay and you've done there and you guys should talk the test contact are you guys from the same company you don't know who I'm talking about though how can you say no the test context is pretty powerful it gives context to the test you can put special attributes on the test that say I want you to get data from a specific data source like Excel or a database and for each role in the data I want you to put it in a property on the test context object so instead of a test having parameters in ms test the test takes the parameters from a specific row on the test context so the test context actually has a data set that it loaded from the database and each time of running a test it will run the same test for each row in that table in the database so it's kind of like that but it's not hard-coded its attributes you can use it with millions of rows the difference is it's less readable you don't know all those values and it's not great for very simple scenarios it's really good for a long ranges of values it's really powerful and if you think about MS test was not really built for unit testing it can do some unit testing it was really built for just testing and it's that it's a great system and that system has range values and all that stuff so you have a lot of power behind you that you probably don't really need most of the time but when you need to it's like taking a shotgun and shooting a fly okay it's very very simple to use once you learn how to use it so take a look at that if you're interested another important thing test isolation I talked before about tests being run and then suddenly failing only failing when running in specific order or only when the moon is full and all that stuff that's usually a great way to find out that your tests are not really isolated from each other isolation just means that this state the universe in which specific tests exists and another test universe are overlapping so when that tech test changes state let's say on the shared database the other test also checks that same database but they're not being cleaned up or reset after before each test and that means that if they run let's say one test inserts around database and one test expect to specific row in the database that would work in a specific order change the order and suddenly you have a problem because the tests are not isolated in fact what you want to have each test do is to set up its own state and to roll back the state that it's set either in the end or to set up the test state at the beginning now well usually both now this only happens when you are either using statics or when using really external stuff when you need to roll back state after each test I think 90% of the time it means that you are using doing an integration test really because you're touching some external state here is an example just like you have a setup method in the setup method this is where a test could create all the state externally that it needs like create rows in the database or whatever but there's also a teardown method a teardown method is known by having an attribute called teardown the name doesn't matter on the method itself now in MS test there's a test initialize and what was test clean up hey you did it congratulations guys - he actually just told me on the right time - they're only ten minutes left okay you're going to get a t-shirt okay so if you have to remove something in the teardown usually means you're doing an integration test you probably want to move it into the different into a different project for example if you're doing database test this is where where you would roll back the database if you are changing files on the file system this is where you would delete all the files that you created so what you want to be able to do is to run the test in any order that you want you can just run any of them and you would still get the same results every time test has to be repeatable in any order and in any amount of tests if they're not repeatable at some specific way you probably have an isolation problem in your tests and isolation problems are really really difficult to solve it really does take a lot of time to find and fix those kind of things avoiding multiple asserts who asked that in the previous session where are you it's not here okay I I try to avoid multiple asserts in tests and the main reason is that having multiple asserts and the first assert fails it throws an exception so the next asserts never run you never know if they passed or failed so only you only get the partial picture now in xunit net it's not a new relatively new test framework you there is special API to run asserts where they won't fail but I don't think the API is that usable or readable I won't recommend using that specific API so I still think if you're doing multiple search most of the time you're testing multiple things the only time you multiple certs are somehow make sense is when you're logically testing the same thing for example multiple properties on the same object you got the right object back so it has all these properties correctly you can also replace that with one simple assert by creating an expected object with all the right properties and then asserting that the real object and the new the the expected one are equal that would be the same logical thing if you can do that or something like that means you're okay but most of the time you don't have to do multiple asserts and if you do just make sure that you know that if you get an exception you only get a partial picture readability I've already talked about the readability of test names I won't repeat that just once you have the name of the method you have the scenario in which you're testing it and you have the expected behavior if you lose just one of those things the reader of your test is going to have a hard time figuring out what it is that you're trying to test or what's supposed to happen now if you're writing the same test with different parameters and not using the test case attributes like I showed then just use the same name and put a number in the end so that at least you have some numbering scheme so someone sees that says okay this is just another angle of that other test they're the same just the parameters are different as long as you're consistent about these things throughout the team then the test will will remain readable that's usually the thing with conventions they have to be consistent if you don't use the conventions consistently you can have a problem okay one last thing I once came upon a test and the name of the test was test something 1 and the test had an assert in the end it looked like this calc bars takes minus 1 returns thousand and three I said wow I totally understand what it does if you send minus one getting back a thousand and three what is a thousand and three what is that magic number and the problem I have with test is they should have no magic number is no magic strings no magic values whatsoever if if there is some number value there and someone has to go and say hmm I wonder what that is it's a magic something okay it's magical you don't know what it does you don't want to know it's important though you want to put them in Const so the test is more readable here's a more readable example of that okay it's a bit more worthy but if the test name sucks at least you have this I get the first result but I send in a negative illegal number this is the scenario the expected result is negative parce return code now if the test name was okay it would somehow being readable but I would still if I was doing the test review for this I would not be accepting these magic numbers and values I would say put a Const there there is no reason to put a string there unless it's the most basic kind of stuff for example an empty string always use the simplest values that you can to send into a value under test into a parent a no method under test just like if you in loops you don't have to declare names for I and J I and J makes sense you know what it is then if you're testing something and using the simplest values don't have to give them constant names but a thousand and three odo you really have to give that a name and minus one probably have to give that a name as well okay last thing I always separate the assert from the action I don't put the action inside the assert method if I want to debug it later just as an example I want to use the result from the action I want to be able to use it later okay in the summary you've seen all these things what I really want to cover is the fact that all these things have to go together you remove one the rest fall behind and suddenly you have all these tests failing and your your cost just went went up a lot went happened to us on a project we lost a lot of weeks on tests we were writing horrible tests at some point we started stopped maintaining them and all that work was just lost and we stopped getting all that feedback the no time for questions because I promised a couple of songs okay for my song I would need a volunteer now thank you so what's your name huh Burda huh huh okay please welcome better I noticed you all speak with a question mark in the end right but I was in Florida and ostilow Lapua I I get it do you know what to do you don't hold anything you just need to press face not yet though and now as I sing the words by the way for helping me you're going to get one of my books ladies and gentleman I just got my books there you go put it away though I need your help though I just want to explain what this song is about and then you can all go to another session um let me see [Music] okay okay [Music] this song is about a team I was in [Music] and we had a developer that wrote really horrible code and he didn't believe in unit tests or whatever and we had to keep you know following his code and debug it and fix it and he worked there for about twenty years so we can get rid of him now I'm not saying 20 years is not a good thing I'm saying that if your hire a developer with 20 years experience make sure they did not repeat the same year 20 times okay [Music] but yet [Music] okay go close your eyes and just a mile his testing is not your style and you tell us how much time you say but then while you're away we debug your code all day to weed out all the bugs stand in hands go all that source no shame all that source code [Music] all that source code all that source know all that source go will you ever change we pretend you've been fired or hopefully retired it's never and then while your way rewrite nests everyday so the source code thank you very much guys um you want to do another song you got a book let's do another song I promised two songs let's do another book not on another book I'm sorry okay let me just switch to the other talk by the way that's my son where is that this is understanding TDD and let me see we should have a song somewhere here okay there you go now this is my book by the way I have already tested you know Nate okay this is a song that I wrote I also wrote the music hey can you move the mouse just so it doesn't interfere okay [Music] this is about a manager I had he didn't like me writing tests what's whatsoever go every build you bring refactoring you me every mark you fake lives like I'll be watching you every stand-up meeting I believe you're cheating tests pains my chest I'll be watching you this is hard Oh can't you see nothing comes for free every test you run is a waste by every iteration 10 years integration now I am and it bothers me you see this is scaring me it would be so strange if my job will change [Music] every bill Duprey every mark you fake every test you make I'll be watching you every build you brick every mark you fake every test you make I'll be watching you thank you guys thank you very much and I'll see you up there I'm doing the beautiful team's talk I think in like three minutes how much time do you have Ulla [Music] [Music] you

Channel: Roy Osherove

Views: 79,574

Rating: 4.8359909 out of 5

Keywords: osherove, tdd

Id: dJUVNFxrK_4

Length: 61min 42sec (3702 seconds)

Published: Thu Sep 27 2012