On Writing Software Well #5: Testing without test damage or excessive isolation.

hey this is David hi Mary Hanson welcome to episode 5 of unright Excel 4 well today I'm going to show how we do testing in Basecamp we're gonna focus on two specific aspects model testing and controller testing but before we get into that and I'm going to show a specific feature and how we do all the tests around that I just want to set the stage a little bit I've talked a fair bit about testing over the years a couple years back I had a conversation with Kent Beck and Martin Fowler following a conversation or keynote at railsconf the title something along the lines of TDD is dead and that keynote and the following conversation sometimes get misconstrued as though I don't like testing we don't do testing at base camp or other nonsense like that which is yeah nonsense because we do a lot of testing at base camp we write we have tens of thousands of written lines of test code I do not feel comfortable in any way shape or form shipping new features without us having a suitable amount of automated testing where my criticism in the past has come in around TDD and other testing techniques have been what they do to the code itself what happens when you subject model code controller code to stringent requirements offered up by testing such as the idea of unit testing that you're testing a single thing in total isolation and no other side effects and no other interactions are supposed to happen all that stuff is to be stubbed out in the name of speedy tests well I don't think that's a good trade we talk a lot about trade-offs on this channel and the trade-off between having SuperDuper Marika fast tests where you can perhaps run hundreds of tests in just a few seconds versus having to serve as a cadre of mocks and stubs and other forms of dummy methods or dummy objects to be able to do that I don't think it's a good trade I think in the majority of cases you're better off testing the real things interfacing with the real database interfacing with the real generation of HTML and controller these things that are somewhat slow but can be mitigated through a bunch of different factors and can be made fast enough the purpose of developing software is not to have fast tests the purpose of testing new software is to have high confidence that the features that you're shipping are of high quality and that they don't break at least amongst the common paths that programmers are likely to to test down I think there's a whole other episode we could do around QA especially exploratory testing is done by humans humans that were not part of the development process and I see great value in that as well in any case I'll link to a couple of position pieces on testing and TDD in particular in the show notes but with that Set let's get started let's look at how we test a specific feature in Basecamp using concerns and all that other stuff we've been talking about in previous episodes so the feature we're going to look at today is really two features it is the documents feature and it's the aspect of documents that they can be locked assisted only one user in the system can edit them at the time the locking aspects of documents comes to us through a generic consideration that flows through the recordings object recordings is this base object in Basecamp 3 that provides a lot of the shared behavior that multiple types of content in the system can use and documents is one of the types in the system that can use this lockable concern as you can see the lock are the document class here doesn't really have a lot in it right like it's it's mostly a container just for the data not so much about the behavior because a lot of the behavior in the system is held in the recordings class because it's shared by multiple different types of content for example the fact that you can lock a document could then also be a feature that you can lock it to do lists or something else to prevent concurrent access in any case as you can see here we are explaining a couple things the fact that documents have rich and plain text attributes this is something that ties into our content system the fact that we use a rich text editor called tricks to provide uploads and formatting and all these things so that's really sort of inherent to document that these are attributes on the document plies these are not generic things not everything has a content not everything has a title but documents do so this is where we declare that we also declare the fact that there is a default title called untitled then we use this neat little trick of calling super because document is of course an active record base these days if you had started a new application it would have been an application record in rails but we predate down and have an updated since and then we have these three different predicate methods Auto positioning or auto position subscribable and exportable that provides the recording features a way to interrogate the recordable of which document is one and ask whether they're capable of doing this stuff that that they're doing just to set aside this isn't really what we're going to be talking about but just wanted to show that for example the fact that a document is subscribable means that users in the system can subscribe to a document to get comments updates and changes and so on from it and i just want to show this subscribable concern that we have on recordings and i just wanted to show the delegation here the fact that we only subscribe the creator for example to a recording if we say that the recordable is subscribable this is a way of disconnecting the generic concerns of which subscriber was one from the specific content types such as doc and whether a document is able to able to be subscribe below or not we don't have to maintain a master list of types here that can be subscribable we sort of flip that around and let the document answer whether it's subscriber fall or not in any case let's have a look at the lockable concern which is something we're going to test here the actual piece of behavior and it comes to us through the recording as I just said the recording is this base type that provides a ton of behavior to all the different content types that we have in Basecamp whether it's a document or to-do list or file upload or whatever and the concern we're going to talk about is is the lockable concern so that's one out of this long list of all sorts of things that recordings can do they could be moved they could be mentioned they can notify they can be published they can be a bunch of things a bunch of behaviors that are generic applying to multiple recordable types the lockable concerns is really quite simple it just basically maintains a little bit of nicety of dsl around the fact that there's an association here that a recording can have a lock and a lock can be owned by a given user and when you try to lock a recording lock a document then we first check with it it's already locked and whether we can query whether it's marked by the same user so that's basically the sort of the the feature here let me actually show you how that looks in the in the app itself so if we go over here and take a look at the app on the left hand side I have Victor locked in and on the right hand side I have a private session which is basically just a way of getting another cookie domain such that I can be locked in this who different people at the same time this is Cheryl if big trick goes here and says he wants to edit this document well he'll be let in right away because there's really sort of there's no one already editing this he gets he gets access right away if Sheryl on the other hand goes to try to add the document at the same time as pictures editing well she's going to get this the screen this document is locked while Victor Cooper makes edits blah blah do you want to break the log but do you want to - nevermind no bother with it so that's basically the feature we're trying to expose here but before we get to that let's have a quick look at just the simple tests that we do around a document alone not considering the docx or the de la couple concern and you'll notice a bunch of things I'll sort of call out here one is the fact that as a setup method the only thing we do is to set who the current person is I talked about this concept of current Global's in a rep aside you can go back and look at that in the playlist we set that up such that whatever is being done interacted with the system here all those actions would be recorded this me being the creator or editor or whatever and then what you'll notice is that there's no factory methods here we use the stock manila rails idea of fixtures and fixtures is this a wonderful concept of where you basically configure a base set of characters in a world that you can pull upon whenever you want to play a certain part these are sort of just defaults that can be used and also it's the different forms of testing without the sort of particulars that correspond to that specific test for example we have a bucket which is this generic concept that's used for four teams of four projects in this case it's a project an anniversary project that's just a standard project when we need to test something that needs to interact with the project we don't need to first go through the trouble of setting that up we just pluck out a fixture for that and start using it so that's what we're doing in this simple test it basically records a new document onto the bucket which gives us a recording back and that recording has correlation to our recordable which is the docket the document self and we just do a few simple asserts on that that these values we've said go all the way through this next tag test is a little bit more interesting documents have this feature of having their previous versions stick around when you record a new document towards a bucket you'll get a recording but if you update that recording we still keep the old version of the document around I promise a few times that I was going to talk about our unique setup of buckets and recordings in a future episode and and I I swear that episode is going to come soon because I think a lot of the examples that I go through in Basecamp will make more sense once you sort of have that connection but for now just know the fact that recordable such as documents are immutable versus recordings are mutable so recording is sort of the current version and that will point to the most recent immutable document and then it'll still keep a record of the past versions the past recordable versions as we test here in the past so this test is basically just showing that when you when you make an update we can still have access to the past versions of these recordable first we test that the first version of the recorder balls is that latest update that we made new order is what we updated here but then also that the previous document the second version is is what it was up here what curious little funky thing here is we do this travel five seconds why do we travel five seconds we do it because recordable versions are ordered by created at times so if these were created at the same times system would kind of get a bit confused that's thing we're going to happen in real life in real life what is going to happen is you post a document edited at least a second later so this is a this is a fine way of doing that what you will also notice here is that these are not unit tests in the classical pure sense because we are not stopping anything out these things all go through to database when we're recording this new document it goes through the database it creates also the stuff in the database is a bunch of callbacks actually that happens in other concerns and would you select those run and it doesn't really matter because it's really first of all it's fast enough and I think it's better as we talked about in the introduction that you're testing the real thing under you mocking and stopping as little as possible as you can get away with such that your tests remain as close to the metal as close to the to the real thing as I said and as readable as possible that they don't have all this accelerated complexity of of setting things up that aren't necessary for the test itself and that's also the reason why I like this idea of fixtures that you were you have a base world that you can sort of interact with and then you can do specific things like here the new documents that we're recording for the testing the versions feature they're not fixtures because there's something new then something particular to testing this this feature okay so that's documents let's actually just let's run that test let's go here and see the document test you just sort of run that because I want to show one thing is it actually runs pretty quick right like this is not that slow of a test yes not as fast perhaps as it could be these four tests run in two seconds so that's half a second to test when you have hundreds and hundreds of tests that's going to take some time and there are ways you can get down to 100 milliseconds less per test but for me that's not worth the trade-off part of this is becoming coming because we use this thing called the spring preload err which keeps a persistent process for your rails application running in the background but you can tests again so it doesn't have to spin up and evaluate and instantiate an entirely new rails environment with your application loaded every single time we run through tests so they're they're pretty quick but let's go back and look at the concern and testing that concern what we wanted to test was this idea of locking first of all that you'll see that document test just lives over here under test models here's the document test right here and then the lock test lives over under test models recording lock test right so that's where we structure our tests and the lock tests is similar in many ways to document test it sets up first the current person who's going to do all these interactions and then it also refers to fixtures in the same way and it also doesn't care anything about all these other concerns that they that that are present on the recording as you see like we have a lot of different concerns it just doesn't matter they're allowed to run as they as they please and and again this might slow things down a little bit but hey look at this this is a pretty long list of stuff that recording is capable of doing it doesn't matter that all that stuff is there is this accessible and in some cases adds callbacks and so on so forth because we just focus on this one aspect we just focus on the locks when when we want to test locking it's really not that complicated so as you can see what these locking tests are just basically a handful of them and they're trying to test this very simple behavior which really is just a nice wrapping of the DSL around about this axis of the single association of the lock but nonetheless same story straight usage of fixtures straight users of these models straight users just this consideration for the DES lockable concern and for nothing else really and if I go back over and take a step at running nose you'll see that that also actually runs pretty fast again you have four tests that run in a second or half granted this is a relatively fast computer but you're not gonna get that different results these are all currently at least single core tests so you only get to use a single core on them that's changing in rails 6 by the way just have an announcement about that going to do paralyzed execution of of tests not that you've really needed for this case but when you run a larger suit that that can kind of make sense so that's the model we have our document we have our lockable concern that's applied through the recordings to nearing object to documents and we've tested both of those things we found the dev test run pretty quick that they're they're easy to write something else you'll notice here is most of the assertions that we write are extremely simple I basically prefer to use only two different assertions I use assert and I use assert equal and I can make everything pretty much happen within that there are cases where we extract sort of the main domain-specific assertions if they need a lot of setup or complicated things to test but in terms of the stock setup assertions when it comes to model testing at least I prefer to stick it just to these two all right well let's hook them look at the at the controller's how control the testing works how controllers work with this specific feature you can start with the documents controller which is the way we create these document record balls and lists let's just focus on on one method here the the create method as you can see we're recording a new document on the bucket and the bucket is then giving us a recording back that is almost identical to the tests we ran here in the document test that's what's happening in the controller as well and then we redirect to - either in subscriptions or recordable URL this is this concept of sometimes you want to be prodded right after you record a new document s who who should be alerted about this document or should we go straight to the recordable really not relevant for this discussion here so the way the test for this work is it's even more sort of integrated actually let's use that word controllers are even further away from this idea of being a unit test what we just test a controller and nothing else you could just test this is create action and you could stop out everything and you can make that control test run really fast it wouldn't test a whole lot and wouldn't test whether the pieces actually work together but it would run fast well I prefer to have them to be more comprehensive even it then again that means that they're slower so let's look at this notion of the testing the creation of a new document the first thing I test is actually not the create action it's the new action which and the controller side of things doesn't really do much it just assigned this instant variable to a new document but what it does is on the view side of things it renders a ton of things it renders the application layout it renders the specific show template and that stuff involves a bunch of different things that I really want to know whether they're working enough right so before I'm creating the document I'm asserting that the form they were using to create that document also works and that does take some time is I'll show you in a second these tests do run just slower because not only do they render the whole view they also actually compile all the assets that we need to present that views as that we can see whether everything is working or not so we'll compile all the the JavaScript stuff and the CSS and so forth such that we get the entire view this is this is really not lying when I say that there's inheritance from an integration test this is a kind of Benz raishin testing it's stopping just short of using the browser to drive the interaction because using the browser to track the interaction adds sort of an order of magnitude of delay to it I think it's still a great thing to do when you need to test JavaScript behavior that you can't test this way here we basically just dealing with HTTP requests and responses and interrogating these tml that we get back we can do that all programmatically without involving an actual real browser and when you can I think that's a that's a good idea and it's a fair trade-off yes you could have written a real integration test here that ran through a browser but it would be overkill because we're not really in need of testing any JavaScript specific logic in any case here we just assert that we're getting a proper response back that means that 200 and then we have a customer search in here actually as I mentioned before where we test whether the the breadcrumbs are correct the breadcrumbs are just so you can see what it is it's this thing up here and then we do basically the creation we run through the creation we post towards the controller with our full set up or everything we need where this document need to go and with the parent blah blah blah we then followed the redirect because as you will see here the redirect is is this this thing and then in this case we're asserting that we end up on a page that's basically the new document of the new document have an h1 that that says hello world that's it if we have a look I just want to show you that this is definitely a different order of magnitude in terms of speed of running two tests here this is slower but I think it's very worthy trade-off that the documents controller tests a whole stack of integration all the way from the HTTP request to writing to the database and okay we spend seven seconds to run ten different tests fair enough the other thing you'll see here that might stand out to you is if you've read something on testing some people subscribe to the idea that issues should only have one assertion per test I don't ascribe to that idea at all I think you should have all the assertions that you need to feel comfortable about testing one aspect of what you're doing in this case the smallest thing we can test when we're testing a controller as a whole is a whole action and we should test the whole thing so that ends up being three assertions for this specific test and even with the with the model tests we were talking about before in many cases we have multiple assertions most of these said - and this one for example had what is that for I there's no upper ceiling here - how many you should you should have you should be able to have as many assertions that you need to be feel comfortable about having tested that one aspect in depth and it in full all right let's have a look at the dalack's controller the vox controller shows some of the power we have of this recordings recordable pattern the fact that we can implement a generic LOC controller that could have worked not just for documents but could have worked for for to-do lists or any other piece of content in the system and acts on a generic way it shows the power of using sort of concerns through generic objects to apply them to to concrete objects very very nicely and as you can see it's also a pretty simple controller right like the whole interaction we're interested in here is is this notion of being able to show if something is already locked and providing the ability to to force on block is something that's already locked if we go to the tests of that it goes through all these scenarios that we would want to test and in that case first that there will be a lock if we've already locked this by someone else so you can see here we which is lock a recording that we fetched out of the list of features that we have again doesn't really matter what this fixture is we're just we're testing the locking feature on it we assert that that if you then access that document after it's been locked by Jason remember who stacks it's it's David here here we have a slightly different way of setting up the current user a set up method called sign-in asset sets up a few things including current user that we're using for integration tests but you'll just think through when you're reading through this thing this is David accessing a page that was locked by Jason and what it'll say is that this documents locked now if you're accessing your own page it'll say that you're already locking this page and then we test the three directives things were not locked and so forth here you'll also see there's several assertion per protest actually I think the average here is what - I mentioned these features let me just show you that so there are only four different documents that's we use as the basic fixtures and that's really all you need in most cases this is a specific type of type of content so if you don't need that many when we go to the recordings which is basically every piece of content in the entire system has a correspondent recording well we have quite a few more so when we're doing the lookups here as looking for the recording for introduction document there's a there's a few more of those right so we can go back here and see the introduction document it belongs to a pocket recorder blah blah blah blah it's just all set up for us so that we don't have to do that busy work and we don't have to read through that when we're reading through the tests that's basically yeah yep that's a testing setup for one generic feature as apply to a concrete class in Basecamp three

Channel: David Heinemeier Hansson

Views: 12,968

Keywords: Ruby, Rails, TDD, Testing

Id: Tc5z64XIwIY

Length: 27min 21sec (1641 seconds)

Published: Mon Feb 19 2018