justforfunc #16: unit testing HTTP servers

hi I'm Francis Cambodian do the papa [Music] welcome to episode number 16 of just a funk today we're going to be covering a topic that even though is super important we had not covered at all during just funk which is testing I just came back from Goregaon I got this amazing t-shirt and during go freak and I actually made a bunch of people that were saying that they loved just the funk thank you so much that was super sweet and most of them they were talking about the topics they would like me to cover and one that appeared quite often what's that thing so that's exactly what we're going to be doing today I thought about getting a piece of code that has had a bit had already been breeding and adding some testing to it but turns out that in order to do that I want to make sure that first we all understand how to write tests and go so these episode is going to be about basics of testing in go specifically you need testing we'll be covering how to write tests what we call sub tests how to organize our test cases and all that stuff and also all the two links running testing which like basically go test but also go coverage and code coverage and not sorry not go coverage but go convey and code coverage and all of these things are actually pretty useful if you're already familiar with these things I would say still watch is episode because you may learn some things at the end but if you do not know these things at all do not worry at all because that's exactly we're going to be covering today that's got some fun let's get started okay so to get started the first thing we're going to do is I'm going to create a new directory that it's going to be completely empty and start writing from there as always with vs code okay so I'm going to create first package thunder go and this is a good but we're going to be testing I'm going to say some we can have function called int that given a bunch of integers returns another integer and let's say that these rather than being implemented with the for-loop it's implemented by using so rather than being implemented with the follow-up I'd say that they've implemented using recursion so if Len of es equals zero return zero otherwise return in vs 1 dot dot plus psi zero this seems to be not very happy with the result up vs 1 and on ok so this seems correct that this should be documented and returns the sum returns the sum of a list of integers and actually even to make it more interesting we're going to do is we're going to have our func int that works a little bit differently so this one is the one that's going to be recursive in this one simply calls the recursive one okay so we have two functions we have in capital I and in lower case I and these are different and do not worry why I make them different it just because later on we'll discuss a little bit I was what we can test with two different approaches but this is because that we're going to be testing do we have any tests so far well no so if we go here and we do go test we'll see this not there sighs cool go test is by the way the way you run tests there's nothing to be run so nothing there cool so let's create some tests now how do we get started well the first thing we need to do is we need to create a file that will contain the code that will test our account so basically our test code will be contained in a file that we'll always finish with underscore test or go so in this case we could call it some underscore test or go now in what package we want to set this up well let's say that we set it in the same package so now that it's whatever we write here in this package in this file will be part of the same package sum which means that we don't need to import anything we don't need to import some in order to use it because we're ready part of that same package so what do we want to do now well in order to run a test we need to write a function that whose name starts with test and then let's say test some or test end and the parameter that is needs to receive is a pointer to testing data and testing the T as you can see here is a package defined in the standard library now that testing that T has two main things that we can do with them which is either failing softly afraid taking very hard if we do not do anything and we did go test you will see that now exists it doesn't say notice files it will say passed and if we add go test - V we will see all the tests that were around in this case there's only one test in that we've defined here in that test passed why because unless we say otherwise test passed by default how do we make them fail well kiddoes fail and we can say tifl sorry see the fail is in order to make them fail with data message so able to sell it felt but there's no message so you know in order to inform why this test fails normally we will do P dot error or RF which is similar to printer print F this test fail because I said so I said so now if we run that you will see that now when it fails to test this Excel because either so we just go there's one more way of failing which is T fatal or fatal F same things as print of in test this test fail and stop running so what is different between this trip well while errors will make the test fail but will mark the test so it fails but we'll continue the execution of the test t-top fatal F or fatal fatal death will stop the execution so if we had T fatal F in these stairs failed because later so the second line will not appear because we do not get there we just stop the execution of the desk completely cool okay so with this now we have all the information we need to write a very basic test I'm going to say that I went to some numbers from one to five for instance so I'm going to go some from one two three four and five that's going to be my son and if the son is different to well two plus three is four five fifteen then TRF sum of one to five should be 15 god s this is not happy because it's calmness some it's called ends there you go so now if I run my tests we get that everything works but if for any reason we added one extra number and we're going to fix it it would fail saying we got 21 but do you expect it to stay cool okay so this is this is pretty good and actually we can already do things like run our tests with coverage which is something that basically is go tests - cover if you want to but most editors also have it integrated with with their own plugins so with this you will see that all of our pet all of our code is being tested about it which is pretty nice what about if we want to have more tests because here we're just testing a very simple case let's say that I want to test also the case where there's no values and the case where we have 1 and minus 1 this should be 0 and this should be also 0 should be 0 0 so some of no numbers should be zero in sum of 1 and minus 1 should be zero and got that and since these are variables that are already defined we need to write that okay so these passes again now when we write this code the first thing they're going to think is well in this case it might be okay because the message that we're repeating the test of everything is not that big which is comparing number but imagine that you have to check more than one thing in that you would be repeating that testing logic over and over monthly most probably copy pasting which is a bad move because I'm sure that at some point you will introduce some bugs which I do all the time so in order to avoid that rather than doing copy pasting we're going to define those test cases as rather than lines of code we're going to define them as pieces of data in a struct actually not in a stroke but in a slice of strokes how does that look like well it's actually pretty simple we're going to do is I'm going to define a variable T T entity stands for table of test or testing table or whatever and that's going to be a slice of strokes and for now this is valid code already and that slice of strokes going to have a couple fields I'm going to say we have numbers which is less of int sum which is an int okay and we're going to have the first one it's going to be one two three four five and results 15 second one is going to be nothing so now and the result is zero and finally we're going to have one and minus one and we also 1 0 cool so now we can get rid of all of this and instead use a for loop here so we're going to say for TC for test case range over TT and I choose these names because it is for easy to remember for me you could choose your own names for those valuables make sure that you know you do not call any of those variables see because these already the testing the T so you will shadow the variable inside of the loop and then where your things would happen and in wooden upper most probably not compiled cool okay so we're going to do is int it's going to be TC dull numbers doubled up this is at some and some of we're going to go TC the numbers should be Q dot some and got some okay so now if we run this test got that as V it also worked let's see if we try to make it fail it felt saying okay some of the empty slide should be one but got zero okay so this worked very well now I'm not completely happy with the way we show the results because when it fails when it's a when it shows for instance some of the empty lives we need to go find what is the death so rather than doing that but we can do is give names to every single test case to make sure that we're documenting better what we're doing if it's something that I do anyway so what I'm going to do is add one named string and this could be 1 2 5 numbers and 1 and minus 1 and now we could use this is a name here so now when everything passes everything looks good but if for any reason this fails we'll get a message saying some of no numbers should be one gutter cool this is pretty simple and and I like it but there's actually an even better way of running tests because right now imagine that you have I don't know a thousand of a thousand of them in here and you want to run a specific one there's actually a cool thing that you can do when you have for instance multiple tests so test food test food let's say that it's going to fail but saying through always fails so that's a different test you could choose which one to run by using go test that's run and you can say int and you can do that V let's fix that so it doesn't fail so you see the only test int was run you can do fruit and you can say that only test flow is 1 and if you pass dot then it's a regular expression that matches everything so both tests in some test will run now what about those test cases we have three here but if we had 1000 I like to be able to choose which ones I want to run specifically and as it is now there's no way of doing that but it is very simple to get to there what we're going to do is we're going to use what we call subtests sub test is basically let me remove this thing because we don't need any more it's something that does t run so when we do killer run we're going to do is create a new test and the primers are going to be we're going to have the name of the test which like you've already have because I will be finding a minute ago and then we're going to pass a function that we take the pointer to T which is exactly the same thing as a test so here I'm going to call it also T so that way of the code inside it's when it's calling theta air F it's not making fail the T here the global T but it's actually calling failing at TDOT run here and here we could now see that fatal F because when VX when we stop this execution we stop only this part of the execution this sub test not the rest so that is cool but what is the advantage from doing this right well now when we do go to test that V you're going to see that not only you run tests in speed you run tests in in this these new your numbers 1 to 5 numbers and 1 and minus 1 which are derived from the string that we chose for the name of the test so now we can run go test - run you can do int slash say one and only the ones that match 1 2 5 and 1 - 1 will be executed the one to test numbers will never be executed but if we say numbers only no numbers will be executed great before we continue with testing something a little bit more interesting I wanted to talk about a little bit of an option we have here which is I said that the package where we're adding our tests is the package sum so we don't need to import anything in order to you the function int because we're writing the package some now there's an option so you can use only the things that are exported so basically we're going to break code that is not going to be part of the same package it's going to be positive of a different package but in go all the files that belong in a same directory should be of the same package right well not exactly because there's an exception here where you can call some underscore test that's name you can gives you a package and now int will be summed up else so there you go and we get going ports to add our import statement which is cool and now this means that our code looks more like what someone using our package would do which you know it's interesting but now that we're talking about this there's an even cooler thing which is in go we have what we call examples as tests so what you can do is I'm actually going to do it in in a whoops I'm going to do it a new file I'm going to call it examples underscore tester go in in gonna call and package some test and I'm going to say example of ends there's going to be a function that we will note that it is part of examples so we can do some ends of let's say one two three four and five s is equal to this sum of one to five is s okay so this is an example of how to use our API it's pretty simple example but this is actually quite useful and the cool thing is that this can be made into a test by simply saying that the output which we expect is 15 so now we can actually run this test look at that and it failed because oh because I'd actually expect that to be sum of 1 to 5 is to state which is good that's exactly what we expected there you go so now we can have an example which is part of our test suite now the cool thing about this is not the fact that there is a test but this is an example and it will appear on the communication so if we run now go dog and we give some port to it and we can go and visit localhost 60 60 go to packages and we should be able to find our just for fun testing some and cool thing is that now you can see that on the function in there's this example right here and that is actually really cool there's an option so you can make it runnable if you go to the combination of the standard library you will see that the examples of runnable in there so you can see how they work and these are exactly what I want to do right because right now you can see Oh front end receives a list of integers of it is an integer we have some implementation but on top of that whoever is doing this documentation can see that this is how you use it all you don't need to finding new slides just as enamors like this which is right now and they're also dead so you know when win okay for the second part of the episode what we're going to do is we're going to test something a little bit more complicated rather than testing just this function some we're going to be testing something that everybody has issues testing which is an HTTP handler I had I wrote this very simple web server let's let's try it so if I to go build web server and I run it in the background then yes I can do curl localhost:8080 double v equals two and it will return nothing I'll because the SH is being special okay for and if I pass three you get six if I pass let's say a it says a is not a number and if I don't pass anything we will say missing value cool so that is our double handler function and we want to test it how do we do that well let me kill the webserver first so what we're going to do is in in that same directory we're going to run a main underscore test ago and these works no matter what package you're using so you can definitely do package main even if it's a package man you can still test it and you should actually so we can do a func test of double humbler given attacking the T and let's say that we're going to pass to so we want to send some somehow want to call double Handler and pass the two so we verified everything for now the problem is that double handle receives a response writer and HTTP requests for the ESP requests it is actually not that hard to do I think that if you let me write under so it adds the things HTTP me request we can do get localhost:8080 and we can do localhost:8080 v now double the ecosphere and not body because I get and that's going to be our request that also eating returns an error that we should handle does it though yes okay so if there is the nail we could not create a request and die here otherwise we're going to do double Handler of something we don't know what yet and request so okay so we need a response rider and a response writer if we go to the documentation here you can see that the response rider is an interface so what is how do we create a response winery's with an interface you cannot you could write a thing that satisfies an interface in order to be able to keep track of whatever is written in there and that'd be fun but there's actually a much better way because there's already something that that's exactly that it's actually in HTTP test dot new recorder a new recorder will return a recorder and that recorder a response recorder is actually something that is also response rather so we can pass it here okay so now we have we're able to call the function so let's see what happens if we run that goat s - double V okay so the test pass is nothing fails but we're not verifying anything either so how do we do it now well the request is useless now we've sent it and we go so we're going to do everything on the response which is rag dot what we can do is rag dot result and that will return a phones now in that response we can do the same check that we will expect that the client will be doing so we can say okay so the response dot status status code should be HTTP status okay so if that is not the case T dot error F expected status okay God responds that status okay so let's run again I see so we were getting a 200 status that's cool and what would the next step let me think so we need to check that the body if we do Reese we want to read the content of the body and check that it is indeed 4 so we're going to do I owe you to read all of the response body we should also do differ response but it'll close no matter what which is not that important because there's a tent but you know out of good form is important to close responses okay I owe I owe you tail with all that will return the content which is a lot a bunch of bytes and an error if the air is a nail then they don't have expected now could not read responds and this is why then we're going to parse the content of that thing so we're going to create a string from B and use that to do a to ISO ASCII to integer that will return an integer which is d for double if the air is done now we're going to say fatal F could not expected an integer god this and I'm doing a person s so the flattest bytes will be transformed to string and finally if P is not equals four then we will say that something fell so in this case we say T fatal F expected double to be for God be cool so there's a bunch of tests in there but it fits all in one screen if we run it it doesn't pass expect an integer God for that doesn't look very good expected an integer and God for why could that be let's sous-vide the Train space just to make sure that there's no extra spaces or line breaks or anything around there okay now it passes good so now that we have this let's try to make it spell so let's say that for instance I pass instead of a passing of 2003 this will fail saying that expected double two before got six that looks pretty good what if instead of passing through a pass three now doesn't pass either okay so this looks pretty good but how much are we testing really here let's see the tube coverage and see how much we're covering actually we're covering only the happy path we're not covering any of the mistakes so what I'd like to do is add more test cases here and as we did before I'm going to create a table with all the test cases I want to check but now it's a little bit more interesting because it turns out that we're going to check a little bit more than that right if we for instance will not pass anything the air will fail because we expected status okay and we got bad requests and also instead of an integer we've got a string same meeting value and that is actually exactly the behavior that we expect so what I'm going to do isn't going to define something that has a name which is a string it has Daud double I don't know how that was working before didn't misspell it twice now okay I said let's ensure that oh it's a CH yeah it doesn't matter because we're still sending it to the student function to the function directly so there is still the question of how we test the routing inside of the server we'll do that in a minute but for now let's concentrate on testing just a specific number okay so we have a name we want to pass the value to double which is an integer we also want to check that the double is this which you know actually let's instead pass a string the value is going to be string and the double is going to be an integer [Music] status it's going to be an integer which is the status of which could be either 204 status okay when everything went fine but also we use the 400 when the ways we expect to get that request and the error is going to be a string which is the error message when the status is not 200 okay so in our first case we're going to have name is double of 2 the value will be 2 and the status will be II that is okay and nine so that would be our first test case and let's see missing typing compasses where is this miss out because that is a slice of Struck's there you go so now what we can do is as we before range over those and we got range over TT so we going to do here it's going to be V equals plus TC dot value so that we're going to be testing we're missing color up cos I plus there you go if that fails we're going to do t dot shadow which it is not good with the follow but I'll move into sub testing a second so we'll take that at that point we'll call this which is perfect then bragged odd result we're going to check okay so rather than checking if it's okay we will do if the status is not what we expect expected status DC that stutters God sprint status code to actually um so that is if we get different status that that's cool if actually let's do it in two different parts which is rather than settling for the status if there's no status we're going to expect status to be okay so if T C dot error is different now we're going to do something in return for now we not doing anything to something otherwise is going to be H status okay status okay thank God some of the status TC dot error ah if it's empty but it's a string so see that error and the fine TC that status yes okay so that is when the error was empty which is we expect to get an actual integer so we check for all of these things we check the result and we finally do T dot double and tip double here and TC double okay now in this is not happy with me to see the double there you go so in the case where it is not empty means that we expect the status code to be for four status code is not bad request not for for for hundreds or a bad request so if it's not that then we're going to do t dot air F expected status bad request god whatever we got starts go and if we we if we read everything from the body so I owe you two dot read all from the body and that we use it as a string actually let's also trim but the trim space of this so that is the message double dot I'm a missing now too many arguments in called over it all could fail of course so if there is no meal you know what we can move these up here and that way we will need to check that again so the trim space of face so if we trim the space around the message we get and we said that the string if that is different to the air we expect then we also fail saying expected message Q dot Q so this is an error and message and return because we're good and now in order to make sure that these actually correct we're going to put all of these inside of teaser run so teach is a name and funky testing the tea and all of these goes inside air F okay so that's a big test but now what we're doing is we're actually testing all of these things so let's say missing value the value is going to be empty so we expect to have an error so double here should be for the errors here should be missing value missing value so if we test this everything passes and if we check the coverage again we should have higher coverage because actually we're now checking for that missing value what about the one where the value is no string so we're going to do is not and not a string not a number so we're going to pass X not a number X so we run it again we pass we have everything is passing which looks pretty good and if we do the coverage you'll see that indeed we're now testing all the things so let's go once more over the code so the way we're testing it is by using table a table of tests with a table of test cases as we did before with some but now in order to call that function we actually need to create a request which is quite simple because we have HDB done memory request and also we need to create a response rather in order to clear a response rather we're going to use the HTTP test package which is HTTP net HTTP HTTP test and that one gives us a new recorder that actually satisfies response writer interface and allows us to then call results to get the response that a time would get and then do a lot of checks in there and then we check if we expect an error we do all of the text on the air to make sure that they're actually make sense and if we even expect an error we verify that the result we're getting is actually the good one cool okay so we're almost done we're going to cover one more thing which is how do we test this part here how do we test the fact that actually not even this part here but this function here right how do we test that whenever we send a request to a slash double actually gets to double Handler correctly let's do that so as I said so far what we've done is testing the endpoint itself HTTP handler but we have not just to drowning and that's why even though at some point I missed I'll double here and I had double or I don't know how to pronounce that the test would still pass so how do we test that how do we test the fact that the request is routed to the good handler well in order to do that what we're going to do first is we're going to have to modify our code a little bit because we're going to do is rather than testing the with the endpoints themselves or the HTV handlers we're going to test the HTTP handler value right which is the one that we pass here normally so we're going to do is we're going to create a new function that's going to create our handler that's going to return an HTTP handler and what this is going to do is going to do first HTTP dot new surf marks our dot handle func double with double Handler and return that and so we're going to do is our handler is going to be the result of calling handler or directly what we can do is simply pass it there so this should do exactly the same thing so everything we've done is we've actually created a function that puts together basically the whole infrastructure for routing a request into the good hammer now if we do run this it should still work exactly as before so let me try they do curl we're doing before double the number zero zero two is four so it works exactly the same as before that is perfect that's exactly what we expected but now the good thing is that we can actually test it a little bit better so how do we test this there's a couple ways of testing this one would be to actually call so to write a function that would the func test routing and to do simply so testing the T let me close a couple things around here so we can read the code better and we could do handler is our H then H dot service VP has that response router and HTTP request so we could do exactly the same thing as before but there's a different way of doing it which is actually quite cool too which is using a different part of HTTP test which is a new server so when you do a new server what you're going to do is you're going to pass a handler so let's call it handler here and as a response you get a server and in that server let's call it SRV you can do two things so close which is something that we're going to do at the end but also it you can do sr v dot URL and when you do a survey that URL what you're going to get is a URL that when you send a request there you will get to the to the server itself so if we do something like let's do s print F of s so we're going to do double V equals two so we're going to put that URL right in there it's going to be our URL so we can do a cheapy Doug get of that that's going to do a response an error if there is no nail T dot fatal F crude nuts and get request and this is why otherwise we're going to check again that the response has a status 200 and that the value is respected so I'm just going to copy this code here and put it here directly this is not as important because actually what we want to do is we want to make sure that we're just calling the right function right so we going to meet also to do this part where we read right here okay expect an integer TC dot double that's going to be for for a nanny for this okay so in this case I'm not using a table testing this time but you should definitely use it but now if it will go to a tree you're going to see that these passes and that's from routing so we want just the routing one it actually passes so what if for instance we had something misspelled here instead of double we had that B or something like that with the previous test but the name of our test already forgot whoops our previous test is called double handler so with our previous test double this would feel pass even though or code is not correct because we're not testing the routing but with our new tests these would fail saying I expected to get okay and instead got 404 page not have not found so the thing is that now we're actually testing a little bit more we're also testing oh if this works oh it doesn't pass because I have misspelled as okay there you go now we're also testing that part where we set up things and that is exactly what we wanted so now we're able to test from one side we're able to tell the endpoints on the other side we wrote it as a round into which is pretty nice we also learn how to use the table or enter testing which is the ones that you're going to see basically everywhere in the Samba library not necessarily kilos run because the wrong was added not that long ago but you will see that the same idea is basically everywhere and I think it could be if you want to have some impact in the NGO project this could be a good way to start find places where the test is written without using sub tests just as a as a table it may be migrated to subject I think that that would be a good way to improve our testing strategy for the go project so maybe a good place to start great so with this we've learned how to do basic unit testing we've seen how to use table table oriented table base testing on people oriented testing to make our tests less repetitive and that all the things that we care about and then we also saw how to use examples which are both part of the commutation in text also finally we covered a little bit on how to test ATP handlers which is interesting because not everybody knows how to do it because it's that way stones writer interface how do you implement it HTTP test is there to help you and then also how to test routing which is also HTTP test but this time new server instead of new recorder I hope you enjoyed this one I'm working on the next episode on how to test something that I have not tested so far which will be locked I've so we'll be talking about how to fake signals how to fake channels how to big time all of these things which are actually really interesting but before we are able to do those things we need to understand how to do the basic testing so this is the end of the episode I hope you enjoyed it let me know what you think about the episode you can do so using twitter i'm francesca on twitter you can also just simply leave a comment down here or you can also use the form that I created form joseph on comm so you can propose some episodes ask questions whatever you feel like doing I might not answer immediately to those but I'm actually reading every single one of them so as always thanks so much for watching please subscribe and see you all in two weeks [Music]

Channel: justforfunc: Programming in Go

Views: 56,203

Keywords: justforfunc, golang, go, testing, http, programming, coding

Id: hVFEV-ieeew

Length: 48min 30sec (2910 seconds)

Published: Mon Jul 24 2017

Reddit Comments

[u/RandNho]

This taught me a lot. Thank you. Maybe also example of testing HTTP client?

[u/klauspost]

Awesome as always! A good introduction to tests. It took me a while to figure out why the /doulbe?v=2worked, but you got the routing test in there in the end. :)

You might consider splitting larger topics into smaller ones. This one for example could easily have been split into "Go tests" and "Go http server tests". It is a bit easier to consume when episodes are below 30 minutes, 20 is the sweet spot for me :)

PS: Link in description for code is b0rked.

[u/Squeakerpants]

This is really great. I'm looking forward to watching the rest of your videos. Thanks for sharing.

[u/kris-nova]

Another wonderful JFF - Thanks for these, they are always fantastic!

[u/artooro]

I like your videos. But my question is does anyone know what code editor u/campoy is using?