The Django Test Driven Development Cookbook - Singapore Djangonauts

are we life okay good okay thanks a lot for coming so last month when we met who was there who was there at the django meetup last month's raise of hands okay so that's about like fifty percent I asked like who who has done test-driven development before and they were like only two out of 30 people who confess that they have done it and I couldn't believe it so that's why I kind of decided that I will punch away punch together some slides and hopefully I can show you why test web development is awesome and why it's worth doing and yeah hopefully next time when we have a django Meetup and somebody asks for texture on the element everybody will raise their hands right so today we were gonna I try to show you the basics I was hoping that I can get into the more advanced stuff but when I realized that I had like 40 slides already and I want to do it like live coding so it's gonna take some time I realized it's not gonna fit into one day so maybe I will do a follow-up talk next months and then we will do the real advanced stuff so we will talk about how to set up your Django project how to test models how to test your admin classes how to test your class-based views and not using functions functions based views anymore but they are even easier to test how to test if a user is authenticated if the view is protected against you know uh none allowed access how to test your forms how the test post requests how to test for 4:04 errors because I like to throw a 4-0 to raise a four zero for exception if a user tries to access an object that he doesn't have access to or that doesn't exist in the database anymore and finally we will talk a little bit about how to use the mock library for example if you want to make requests to a third party API like if you want to get your Twitter timeline or something you cannot really do do that every time you run your tests and probably try to have a block you after a while because you are exceeding your your rate limits for accessing the API right so there's a way to and it will also speed up your tests tremendously if you mark out these requests or third-party api's so I'm a bit nervous because I have basically finished your slides tonight and last night and I have never tried this but I think when I if I had basically only have two options I can show you all the code snippets and then you will probably see it and if you haven't done any Jango before you will not really get what is all about or I can really show it show you how to do it and that's what I what I want to do because I want to show that it's not that painful right usually you will think oh my god I have to write so much code in order to get my tests done and I don't even know how to start so I want to try to show you my way of thinking and I want to try to like think think out loud while I'm doing it so you can see how it grows slowly right so basically we would start with a completely new Django project and the first thing that I always do is I create a virtual environment for my Python project I've already done that so I will not do that again I will install Django into that virtual environment and then when once you've done that you can start a new Django project with this command line tool here and that's what I going to do okay oh wait it's the font big enough everybody can see that right okay hmm I shouldn't copy this okay so I created a completely new Django project and I call the test a tool and it generates some default files for you the next thing that I always do in the very beginning is I create a test settings file basically you put that file right next to the original settings file and you put some stuff into this file so this basically means that instead of a post quest database on MySQL database you want to use an in-memory SQLite database which makes the tests really really fast right because usually you will have hundreds of tests and every single test will destroy the database and create a new database and if you would do that with with the database that needs I Oh operations on the hard drive is going to be really really slow so in memories it's a good way around this and then I like to set this email back-end here through a local memory so that I don't accidentally send real emails has happened to me before to real people while I'm running my tests and usually you run your tests like hundreds or thousands of times so you're gonna send a lot of emails accidentally so this was a painful lesson that I had to learn one day so then you would need to install a few Python libraries first of all PI test then it comes and PI test has lots of plugins and you can just use them by installing them so PI test django is one plug in PI test IP DB is for setting breakpoints into your test and then you will be able to use the IP DB debugger which is nicer than the original one because it has colors and code completion and PI test coverage helps you to generate a coverage report so every time when we run our test that generates a bunch of HTML files and you can see all your code files and how much percent or how many lines of in that file have been hit by your tests it's a kind of very opinionated thing many people say 100% test coverage doesn't really matter doesn't mean that you have good tests and that's true but for me I have realized when I don't have 100% test coverage I get into this situation where I'm like I just fixed this quickly I just apply this I know this anyone and I don't care about the coverage and then this kind of small slop like this small situations where you want to be sloppy they add up over time and then your test coverage goes down and down and down and you end up being at 40 percent or something and then the test your test suit is no longer really useful so I try to keep it always at 100% I cannot even deploy to my production system if it's not 100% so yeah the this keeps keeps you motivated to write more tests my mic is shaking isatis he's chopped oh yeah this is just after you install all this you need to deactivate your virtual environment and reactivate it otherwise it will throw some weird errors and okay so the next thing is I create a PI test any file in the root folder so basically here I'll say new file five tests any put this stuff here inside and you have to tell it where is the test settings file all right and then these are flags that you would usually use when you when you execute the PI test command on the command line right and usually on the command line you start your test by just typing PI dot test and then you can add more flex and I don't want to type all these flags all the time so this basically means add more options to the command line argument so every time I run my tests I want to run coverage report and I want it to be an HTML report you can also output it as a PDF or I don't know what like several options okay so this is the any file sorry at ops at options yep okay so once we've done that we can try it we can try if pilot test works and indeed it tries to execute tests and so PI test is a little bit like nose tests if anyone has used that before it tries to be very smart and recursively go into all your folders and subfolders and search for files that somehow have tests in their name and then it goes into those files and searches for classes that have that begin with tests and it goes into those classes and searches for functions that start with the word test underscore and it tries to execute all this all this code so that's what testing is all about right you try to execute a bunch of functions and in those functions those are your tests you try to execute your actual code and you try to pass in certain arguments like H cases you try to call your your function with zero always very large numbers always negative numbers and see if it crashes somehow mmm so now I told you that I also want to create the coverage report so there's another settings file that we need to create and it's the coverage RC file unfortunately it's a hidden file so I often forget that this file even exists when I start a new project I'm wondering why is my coverage report all messed up and what did I do in my last project and then I remember that oh yeah there's this hidden file here so basically I'm just saying that I don't care in certain files I don't really care about the test coverage right for example I don't test migrations because they are auto-generated anyways and if if the migrations to general migrations have bugs in them unscrewed anyways there is no way I can fix that I don't like to test my settings files I don't want to test the tests obviously that would be overkill I don't test my URLs not PI files and also not the Django managed PI and WSGI files so I just omit those files so that the HTML website that HTML website which we will see later it's not a super huge list where half of the files don't even interest me and because I don't test them it means the coverage will be 0% which will lower my total coverage and I want my total coverage to be 100% so whenever you have a file where you don't care about you have to add it to this list so that it gets removed from the coverage report okay okay so now we are ready and as I explained PI test will be smart about finding files and classes and functions that it wants to execute okay so the first thing that we will do is we will sew each Django application basically has a model right I mean there are very few reusable apps that deal with email sending I don't have models at all but I guess most general applications somehow have a database table a model that they want to use and so let's let's assume we are building some kind of Twitter clone and we will call it birdie so Django admin PI start at birdie but it creates a bunch of files for us and you know Django's I mean even in the official documentation they are very they really encourage you to use testing and they have a huge chapter about testing and they even create a test file for you already to remember to remind you you know you should be writing tests but I don't like to put all my tests into one file so we're gonna delete that file instead we're gonna create a folder called tests and this has to be a Python module so we have to put the underscore underscore any file in here and then we will basically create one tests file for each real code file right so if we have a model spy here we will also create a test underscore models pipe so I'm basically just repeating the name of the original file that I want to test okay right so this is this and oh yeah and on the slide before I said that we gonna install a mixer so you would you would do pip install mixer mixer is a really cool tool for helping you creating test fixtures usually when you run your test let's say you have a certain view that shows your user profile and you want to test that view that view can only be called like by giving in the primary key of that user so the URL would be something like slash profile slash v right that will be the user profile of user of v user in our database so there has to be some thing in our database so that this view can even be called because the first thing that this view is going to try is to fetch that model from the database with the primary key v and if our database is empty then the view will probably throw a for 0 for ever found like page not found error because there is no such user right so we will always have the problem for our test that we need to put some data into our database that has to a certain state so that the test can do its thing and you could obviously just do it by saying let's say we have a post model by always in your tests by doing this post you create a new post and then you say post dot message hello and then post op save you can't fill your database like that right but it's like pretty tedious mixer makes it like this mixer the blend app name is birdie model name is post and then message is hello so it's just one line and if this birdie model has 20 other fields mixer will put random values into each of these fields which is also really cool because sometimes mixer will put in negative numbers very big numbers Unicode names and maybe you have Unicode errors in your code and you didn't know about it and so that means sometimes your test run and run and they all pass but then one day suddenly they fail because mixer accidentally or like by chance put something into your test that you never thought about so it even helps you buy up it helps you over time automatically finding edge cases that you never tested yeah so this makes creating fixtures super super easy all right so and we're I mean I wanna I want to talk about test-driven development that means we are gonna write the test before we even write the code and this is the thing that makes it so hard for beginners because if you're just learning Django how you're supposed to write tests because writing tests means you have to know how the real thing works and looks like otherwise it's going to be very hard to write a test but let's try it anyways basically we know that our Twitter clone app needs a certain model so by writing the test I have to think about my real code so I have to think about the name right and let's call the model post so I have made by writing the test I have made my first conscious decision about my actual implementation I came up with a name okay and this test here has to be here so that PI test can find this class and realize that it's a test class okay and so you can come up with any function name here and it has to start with test underscore and then you describe what you are trying to test and we just want to generate test our model and what we're going to do is very stupid test but you will see why it's what is still useful later we will try to create an instance of burritos and we will make our first assertion so testing is all about creating a certain situation and then called assert statement to see if something is true right so we will assert that object primary key equals one and then we can also give a message so if this test fails this message will be printed and it can help us to understand what we were thinking when we were writing the test so we say this thing should create a post instance for example right so and obviously we need to import mixer so from mixer back end make sure and we need to import okay so this is a pitfall actually this should work all imports are here but PI test so some people are purists they say unique unit tests have to be completely isolated and not talk to any outside system and the database is an outside system so if you want to have super extremely fast unit tests you shouldn't even you shouldn't even use an in-memory database you shouldn't use any database at all you should never save your objects into the database but I don't really agree with that I mean computing power is fast enough so speed is not a concern for me and you you will always get to the point where you have to test the safe function of your object anyways and then you would have to mock out the genuine database I don't know that's kind of weird but PI test by default has this thing that it protects you from actually writing into the database and it will crash here because mixer tries to call the safe function after it has created the object so it will it would it would trigger right and then PI test will say we are not allowed to write unless we put this line here okay but this is why we I mean I will give you the slides after this so this is why it's called a cookbook you're supposed to open this up at home and copy and paste the hell out of it right this is how I do it every day I don't know these things by heart I go into my previous projects and just copy and paste everything so theoretically this test should be nice now so if we run this it will crash and because obviously we don't have the birdie ski bar setup right so first of all we need to tell Jango that this app even exists because otherwise it will not even if we had the model already it will not recognize it but the fact remains we haven't written our birdie model yet so it's still failing so now it's the time that we should go here and we should say class sorry I mean post model not 30 model models Mauro and let's say it has each mess each post has a body and that's of type text fields Oh yeah so now I hope the test should pass and it passes right so we brought some tests and then we let them fail and they kind of guide us into the direction of what we need to do next and we will see this a lot better with later examples one which has views and forms okay so yeah here's the implementation that oh yeah let's have a look at her at the coverage report so every time it runs it generates this HTML cough folder here and you can open an index.html and we can see that in our model spy every line of code has been executed because our test instantiated the model and there's there are no functions or anything that could be called so basically everything has been executed I mean importing this file at this point is probably already enough to execute every line of code so we are hundred percent so that's great okay so let's let's test the model a bit more let's assume so the body is kind of like the twitter message right it's a lot of text and we also want to have an excerpt we want to have like this truncated version of the texts right so we want to write a function that a la that gives us a shortened version of the body and so it's just bigger and again because we have to think about this function before we even use it it makes us basically create our own API before we start doing it so this really helps sometimes ice but when I write a lot of tests I realize the way I had my implementation in mind it's actually completely wrong it's like it will result an unmaintainable code or it will result in code duplication where I could centralize things things a little bit more so this really really helps so we will we want to create a so now again I have to think about how should the function be called right I will call the function gets excerpt so that dictates that my my test function should also be called get test exception okay once again I will need an object so I create one and then I will call the get excerpt function on my object and maybe we want to be a bit more flexible we want to say that we want to get the first five characters from our text all right so maybe on some some point of our website we're gonna say the first we're gonna show the first ten characters or when it's a mobile phone there's very little space we're only gonna show the first five characters so we we already decided that our function is called getting served and it's gonna have one function argument which will be the number of characters that we want to get returned right so and now we need to make an assertion but I have no idea what's the body text so I don't know really what should be the results so why don't we say that the body text for our fixture here should be hello world and then one two three four five years so that means if we get the first five characters from hello world our results should be just hello right and then we will describe that should return first few characters or something so you can see about writing these even this is quite annoying some people don't write these messages I make it a point that every test has to have this message because it also becomes some kind of developer documentation somebody who needs to debug a certain function that he has never seen before I brought this hora I I have to debug my own that I brought two years ago you know I will not remember what I was thinking but when I see this test and then I realized oh it should return the first few characters and maybe has someone else has like changed the function so that it returns the last five characters for example the test will fail the next developer will look into this and realize I should return the first blue character so that's probably what the client wanted in the first place so you write it down in like pros right texts that people can read and understand easily so yeah that makes it easy afterwards when the test fails one day to realize do I need to change the test did the business case change or do I need to change my code because somebody invented the back-right or introduced a bug all right so yeah all right and now we will see how this will guide us to fix the solve the actual problem so we're right this test is gonna say the post object has no attribute get exert because we were trying to call get extract here on a post model and obviously there is no such function so loop crash right so now we will do get exert we will create this like make a very minimalistic version and we will run the test again and now it will say wait excerpts should have two arguments only got one right so we know that we have to we have to allow to fill in the character number of characters that we want to get back right now it's gonna tell us oh this looks good so it's actually calling the function now but the result is incorrect because I was only returning none right so now I have to return the real thing and it's gonna be self and this is cool now I can't I can never remember does it have to be char : or does it have to be : charge to splice the string and if you don't do test-driven development what would you do you would fill in the thing that you think it is then you will restart your web server you will go to the URL and you will look if it looks right right so that takes about 10 to 15 seconds no we will just rerun our tests and okay so that I didn't save the file so okay I my hunch was right it's this way right but if I would have done it this way then again the result would not match my assertion and now it's actually pop this is the cool thing about PI test this is much better than the Jango built-in test client pipe PI test get a test runner I mean PI just gives you a really nice output when when you compare lists or strings or dictionaries it tries to tell you exactly what is wrong what was it expecting what did it get and and if it's a very long string where only a few characters are wrong it even shows you like a pointer to the to those few characters that around so it's really cool so yeah obviously this is the wrong implementation this is the right implementation and now the test pass again and what about our coverage looks good still 100% so you see the test guided us by it's showing us several errors our message towards the real implementation right okay we did this okay so we implemented this extract function on our model and some of you who have who might have used Jango before you probably know that Django admin right so you have this list it's like a database a management tool right it can show you all the posts that are already in your database so and our post object right now only has this body field which can be a lot of text so our list would look pretty ugly if it's thousands of characters per post so it would be cool if we will actually in our list admin only show the excerpt right but we can't do that because the Django admin can only show the real fields unless you provide an extra function then you can show anything so that means we need to write this function for our Django admin as well and that means you need to create a test admin file and our admin is probably called post admin and it's going to have a function that is just called excerpt and once again we need to use mixer and that means we are going to save stuff into the database so we also need to tell PI tests that we want to do this and then we create another post Verde post and there is oh okay so the question is how do you test Django admin classes you can't just instantiate them and call their functions unfortunately that's not possible so it took me a while to figure this out but when you spend that time you learn that there's this call this admin site thing here also we need our models so so for some reason I don't understand why you need to have an admin site instance and only then can you instantiate your own admin class oh don't paste properly okay okay so this probably was with confusing so okay wait okay I'll explain this later so our results going to be post admin excerpt we are calling this function and we are passing in our birdie post and we expect we expect that the results equals hello once again all right okay so how do you get to this without already knowing what you want to do so you know that you want to test your own admin class and you want to test the excerpt function okay so basically we made our first decision here we know our function is going to be called excerpt and we know that's by definition if you if you read the Django documentation about writing admin functions they always have self and an object so we know that we have to pass in an object that means we know we have to create an object here first okay so in this stuff here this is why you go to meet us this is the stuff that you have to Google yourself because you ask yourself how can I actually create an instance of this class that I'm about to write and unfortunately you cannot just do this right it's not like a model our post we can just instantiate like this a form like create form if we will write one later we can just instantiate it like this it doesn't work like that with the admin classes they need the the model that they are about and this admin side here I mean I don't know why I just take it for granted I found it on the internet and I copy and paste this ever since so yeah this should basically be yeah this would be it so let's try and once again it's gonna tell us that we are trying to instantiate a class that doesn't exist right so class post so we have to import our own models import models so this admin is about the post and so this is what I wanted to do the app the Django admin allows us to define which columns should be visible in that list view right and I want I want an excerpt column to be there but this will not work Django will crash and tell you that the post model so the model does not have an excerpt field it only has a body field okay so that's why on our admin we have to define this excerpt function and then when the admin renders the list for each row it will call because we are using it here in the list display it will try to find if there as a function with the same name and then I can return whatever I want from that object so when I want to call get excerpt and the first five characters okay so let's see if the test passes and it does and let's see if the coverage is still hundred percent it is okay okay this is our implementation so next thing fuse so the first you is we want to have a homepage view obviously this view should be accessible by anyone you don't need to be logged in or anything can be anonymous usually if you read the Django Doc's the first thing that you will learn about is they have this self dot client get which allows you to call any of your views but back in the days years ago I learned that my tests get really really slow because if you use this as a Sharma it's almost a little bit like using selenium so it's like really simulating a real request it's going through the whole middlewares and you know your template processors and everything it's going through all your URLs dot PI to find the class data the view that it needs to execute so it's really slow and the good thing about us I think it renders your templates so if your template is missing it will crash the bad thing about it is slow and I care more about speed than knowing that I am missing a template file because I'm obviously testing everything in the browser before I ship it and if I am missing a template file I will I will get a 500 error in the browser and I will see that right so there's another thing that's called a request factory it allows you to instantiate your view classes just like we just instantiated the model or like we instantiate at the admin you can also obviously just instantiate your view classes but the view class the views only really work when you pass in a request so you need to create some kind of fake request object that you can pass into your view and this is what a request factory is good for it's like the name says as a factory that will generate Django request object for you yeah and then we we instantiate our view in the same way like we do it in our URL spy class we use the view name dot s view this turns the view class into a function and then you can use this function and put your request in sight okay so let's try that so new file test views so we have to think about what should be the name of our view class and it's going to be home view and this would be called by anyone anonymous so now we need the requestfactory and we we gonna create a fake request and it's going to be a get request so you could also put post here and you can put some URLs here but it doesn't really matter because we are not using your il spy anyways and if your view code as such does not do any computation over the URL string you don't you can always put in any any URL in here doesn't matter and then we need to import our view which obviously right now doesn't exist and we will we want to get a response from the view so we will try to instantiate our home view like that and now it's a function now we can pass in the request like this so this looks a bit we are took me a while to you know get this into my muscle or my muscle memory yeah but it's like that and then we make assertions so we want to assert that the response status code is 200 and that means is callable by anyone or should be I always start these messages with should be by anyone okay so this is our test very simple test and let's see all will crash because there's no home view so let's create one Django views generic template view so this is basically the smallest possible implementation that you can do in order to create a view in Django you just tell it it's supposed to be a template view and it has this template name I don't even bother to create a template file now because we are not gonna looking at it in the browser so so now the test passes so that means this view can be called by anyone and it will return a 200 success code right yeah so as I said the downside is if you forgot to hook it up in your URL spy you won't know about it if you forgot to create the template you won't know about it and if your template is I don't know using a lots of template tags and whatever they will never be called as well but you should obviously test your template tags as unit tests individually already so this is how we make sure that template stacks work and about the URL spy I mean if you code all this and then you just ship without it even trying in the browser then you're doing it wrong anyways right you always have to look at it in the browser in the end and then you would you will realize that you're totally forgot to hook it up in your URL spy so I think these two downsides here is something that I can live with or I mean if you're really hardcore and you have a very expensive project for a very prestigious client you would probably also run selenium tests and really test your front-end and then you will realize that you forgot your template or you forgot to hook it up in the URL spy okay okay so another thing that we often like to test is on each test is authentication you wanna be sure that certain views like your User Profile cannot be accessed by anyone else right you don't want some hackers go into your site and start changing parts of the URL and they see numbers in the URL they will try to like call every single number and figure out if they can access somebody else's profile and maybe even save data or delete data or whatever so we want to be sure that the views that should be protected are protected and we don't further down the road half a year later somebody some intern comes in makes remove the lock and require decorators like oh what is the sticker that looks ugly just remove it all still everything works everything works but now everybody can access your view and you don't want that but your test will realize that right so the thing is I usually use the method decorate a lock and required to protect my views and that means from that point onwards your request you know we use this request factory to create a fake request your request object needs to have a user attribute like the lock and required decorator is trying to access request user and by default the request factory generates a request object that does not have a user attribute so we need to manually attach a user to the request and we will see how we do that and that means Django has a special class a special model actually but I don't think it has a database table the anonymous user model I have actually never seen that before before I started actually using tester in development so I think most people don't know that that that it exists so what we gonna do is we will create a new view now sorry and it's supposed to be a view that can only be accessed by admin users so so this is going to be called admin view and so first we want to test if an anonymous user cannot access this view yeah so we will create our request and this is the thing that I was mentioning we need to put the user attribute into the until the request object and in this case there's so the question is what should it be sure to be none or something no it has it has to be some smarts so that's why Django offers this anonymous user thing here and this is from Django contract off models there it is so now we have an instance of some kind of user that's not really in our database that's not locked in okay so then we can get our response so that means responds views admin view as view with this request that we just created and then we make an assertion and I happen to know if you try to access a view that's protected with the login require decorator and you are not logged in it will redirect you to the login view right so we could assert we could assert if the response status code is 302 I think it's going to or one I can never remember but I don't like to do this assertion because maybe there's some code in your view that does and some circumstances does a redirect so that does that means maybe a redirect is happening but not because the users are none of us but because some other situation so it's it's more like more strict to test if login is in response on URL because if a redirect happens the response object has a URL attribute and this is the new URL where you are getting redirected to and if usually I mean it happens it depends on your project but by default I think it's account login so the URL will look something like that and then usually next where you came from so this is usually if you try to access a blog view this is what Django will redirect you to and I'm too lazy to write all this so that's why I just check and usually my projects don't have other URLs that also are called locked in I mean if you had that then again this test would not really tell you if you are getting redirect to the login view or to some other view that happens to have locking in the URL as well then you need to be a bit more precise here about the URL okay so enemy the second test tests super user this time we create a user I didn't import mixer No so see we can this is the baked in user model that comes with Django and it lives in the auth app and the app name is off and model name is user so we can also create this model even though we have not written this model ourselves right this is living somewhere inside the Django framework and then as always we creates I get requests and we attach our user that we so this is the user variable here right we attach that to the request and then we try to get a response with this request and this time we assert that the response status code is 200 see where oh yes right this is one of my favourite pitfalls I forget it all the time and it gives you a very weird error message as well which is not very helpful oh yeah and I forgot to to tell PI tests that we're gonna do database access okay so now it's as usual it's telling us that we have not invented this view yet so admin view it's also just going to be a template view template name is going to be birdie ant-man HTML and so how do you protect class-based views it was like this you overwrite the dispatch function and you need to learn by heart what is the function definition and it's like this and then you just return the super call okay and now we can use this lock and require decorator and this is something that I always have to Google I never remember where these things can be imported from so just Google for jingle lock and required and then you go to stack overflow and you'll steal this from someone else and you find these imports so now we can use method decorator whoops decorator with a lock and required function so that means for every request that comes into this view the first I mean every time when when we call our view like this when we pass in the request here right it actually enters the code right here you could set a breakpoint here and this is the first position where by by calling this this line here as view and then request we end up being here in the dispatch function and then we just let Jango do whatever the dispatch function should do based on the template view here so it's calling the dispatch so basically we override the parent classes dispatch function you know sorry and we basically don't do anything special we just call the parent classes dispatch function so we pass on the call to them to the parent class but we decorated the function so that means before this code here is executed this code will be executed first right and this code makes sure that the user has to be logged in so let's see if our test passes passes by the way my my naming was a bit misleading here it sounds like I'm testing for a super user but I'm actually not doing that right now I'm just testing for a normal or CENTAC headed user never met I messed up the function names okay so next thing testing forms let's say we want to have a site where there is a text field and people can enter a message and press the submit button and then a new post is added into our database but your message has to be longer than ten characters we don't want small talk we want real talk okay and yeah so we need a form for this so that means we need to create as usual test form spy our form is going to be called post create form I guess or just post form and in order to test our form we need to import it and usually we will what we will try to create an instance of our form so we will do something like post form instantiate right usually we need to pass in some data into the form so whatever the user has entered oh and we want to make sure if the form is empty is not valid so maybe the first test should be we are passing an empty data into the form right and our assertion will be that form that is valid so this is when you when you know how to use Django and how to work with forms you are very familiar that you always have to call the is valid function and that should be false and it's like should the invalid if no data given so then we will write another test we will instantiate the form again and this time we will pass in some data but less than 10 characters right so once again we have two should be invalid if that if too short and there's nothing that we could test we could test if so the the form after you have called is valid has this errors dictionary here it has a list a dictionary with every field name that has an error and then a list of errors for that field name so we will expect that body the sister field name is in this dictionary there's a key in this dictionary with body because that field has an error okay so this makes sure that our form is invalid but we also want to make sure it's invalid because of that field maybe it has many field and it's invalid because of something else then this test would not be very meaningful right okay and then obviously we want to test if with more than ten characters if it works right so we have to test a positive case as well so now it should be true all right and I guess the forum will try to save data to the database so we need to copy and paste this magic here again and now let's try to test by test aha it says the cannot import forms which is correct because there's no such file forms and probably now we'll say that there's no such class Oh what happened all right I went up one folder yeah it has no post form so post form forms it's gonna be a model form and we need classmate I hear so the model input our own models the model is going to be post I had it's also interesting so this should be enough theoretically no it's not it's telling me that I have to provide a fields or exclude attribute so that this form can be used and actually exclude as an anti-pattern I think they will even remove it in later language ango versions I'm not incorrect so we have to provide the fields and so this form should I mean our model only has one field so it's pretty boring now but this form should have the body field the text field so that the user can enter something huh did not expect this oh yeah okay doesn't need class meter so I tested this just now before I came here and I happen to stumble into that as well and this was a very unused and unhelpful error message to think was to set but if I mean if you use Django every day and you you know how to write your forms you would know that you don't have to use the class meter here right so I'm when I'm doing so much was it was admins and views and forms I have I often forget which ones need class meter and which ones where can I just put it into the object into the class immediate directly right okay so mmm-hmm the model is here okay maybe I'm wrong okay does new classmate oh damn it okay and what else is wrong set object that's weird Fields forms model for foreign performs models okay I guess it's a problem with my test what do you mean oh yeah yeah yeah Wow awesome and I actually when I tried this at home I fell into this pit as well but I couldn't remember just now so I was passing in a bad badly form dictionary right the data you pass into the form has to be a dictionary and I wasn't really providing key value pairs I was just providing the value so that's why it crashed so now it says I was expecting the form to be invalid when the text is too short but my form is always valid at the moment so what that means we need to implement something and Django has this idea of clean body so I can for every field on my model so if there's a field called body I can write a function clean field name and then it will be when you call form is valid it will try to find all these custom clean methods and try to execute them before running the big overall clean method and then finally it will tell me if it's valid or not so what I'm gonna do here is clean data so this is something that you need to learn from the Django documentation it's like convention so if the length of my data is less than less or equal than 5 I will raise a forms validation error message is too short all right huh and all my tests pass and what about coverage everything 100% yes okay um we did all this and we are almost done - two or three more things so I also want to show we did get requests now and now we want to do a post request display so we have a forum we already know that we are able to save stuff into the database we need to create a view that is using that form and so that you know you can go to a URL see that form and press the submit button so there has to be a view for this and and let's say okay this is the first example okay and this view is gonna be called update view let's close everything so as usual I have to think about what is the name of my view it's kind of it's going to be called post update view it should be call 'evil so you have to go to that URL so first you have to make a get request so that you can see the form and then you click the button and you finally make a post request right so we have to test both so we will create a request with requestfactory and it's going to be a get request and our response should be by using our post update you which doesn't exist right now as view and we pass in that request and it's going to be like super lazy implementation anyone can post we don't even need to be logged in and we we expect that the status codes equals 200 should be callable by anyone so let's try this it's obviously gonna fail because we don't have that class post update view and this is supposed to be a forum view actually it oh no it's gonna be a that's gonna be an update you oh yeah okay I remember needs a template name okay so the idea okay I wanted to show how to use your El corpse so the situation is this view is supposed to edit an already existing post so the UI always will be something like post / 5 right then the 5 has the primary key of the post so how do you call a view like that that means we need mixer we need to create a post and here when we call the view we will pass in the primary key as a URL clock and it's going to be the primary key of that object that already exists in our database okay yeah and then let's just write the rest of the tests first as well we want to test the post request and I will just copy and paste it to make it a bit faster so what's happening here since this is an update view we need an object so we create an object and the user is going to type something into the field so this view needs some data and and here is a trick so the primary key is passed in here when we call the dispatch function of the view the data is attached to the post request which kind of makes sense right if you send get a post request they might have get or post data so that's why the data is going to be passed in to the post function of the request factory here oh okay I was and I only call in that yes almost okay sorry so we are passing in the data into the post request we are passing in the primary key into the dispatch function of our view and then we try to call that view and jangle update views they are built in such a way that if the request if the post was successful they're going to redirect you to a success URL so I'm testing for 3-0 to redirect status code here and then there's another handy thing here that I think it's possible since Django 1.8 or so or maybe even 1.7 you can call refresh from DB because when we use mixer up here the body will be some random text mixer will fill in random text because we did not provide a special message here we did not say hello or whatever right so this is going to be some random text but the post data is sometimes that we know and after the post request we want the database object to be updated right so the new text should be int in the inside the database object so we refresh our object and then we compare the body with where's the text that we we expect okay so if I test okay we haven't implemented the update view correctly so it needs a model name and it also needs view class name so this view is supposed to be dealing with post models and it's I sorry I'm I don't mean view class I mean form class and it's supposed to use a special form class that we defined the post form right so it's just these few lines of code Jango will be smart enough to see a URL that has a primary key inside it will then use this primary key and try to get a post object with that primary key if it doesn't exist it will show a 4 0 4 page and if it does exist it will render the post form for us and if you make a post request you press the submit button the your data will be passed into that post form and if the form is valid it will call safe on the form and it will redirect you through the to the success URL oh that means I have to provide success URL as well let's say it should just go back to the home view it doesn't matter for now so I think this implementation should be fine yeah and it works and we are still 100% and almost at the end of the talk template not yet this is what I do it last but not today because I'm not testing the templates so it's not really most interested in security like can users access other users objects by changing the primary key in the URL and certain form validations can user submits half empty forms and stuff like that yeah so force your for our speaking of which they are a little bit trickier to test because Jango raises of 0 for exception and unfortunately it bubbles up all the way into the test so the test will actually crash and and show you the exception and not move on and I mean we want we want this exception to happen that means that the test is successful it shouldn't crash it should be successful so there is a way in your tests to catch exceptions so we will extend our tests a little bit oh yeah so very stupid example we want to make sure that if there's a locked end user trying to use this update form and the users first name is my name then it's not allowed then we will throw a 4 0 for error this should be this shall be the functionality of our view ok so I will create a user and I will set the first name field to Martin I will also create a post because we want to update a post I want I want to try to update a post we create our request factory and I don't even care to put in any data because I know I'll be rejected I will not even get as far as submitting my data I attach this user object myself to the request and now we have this we have this with statements that is part of the Python language and PI test has this raises function and here we can import from Django HTTP import HTTP 4 0 4 so this is the exception name right this is an exception name you could it I mean any kind of exception in person we could probably even do this we expect an any generic exception to happen or I don't know there are so many exceptions so HTTP 4 0 4 is one exception class that's part of the Django framework ok so and then inside of this with statement we are now executing the code of which we expect that it should throw an exception so let's try that and it will say oh I do it important HTTP 4 0 4 huh justice rehab I don't know didn't save I guess so it says test fails because the exception has not been thrown up because we haven't modified our view code so so let's make this view a little bit more secure by over whoops sorry my overriding post function so at some point the dispatch function will figure out that this is a post request and then it will move on it will call the post function and here we are trying to get the user from the request and then we try to access the first name field of the user and if this is Martin we are going to raise an HTTP 4:04 exception here and otherwise we just do whatever the Django framework wants to do with was the parent class update you oh I messed it up where's the rest of syntax error invalid sensors from dangerous it's down here ha what's good to me did you try turning it off and on again okay so I guess I forgot to save again so now actually it should work but we are getting an error here that's because we added some functionality that makes our older tests run compatible because now we are trying to access the user attribute on the request which we have not done before but we have written tests before here where we did not add the user objects to the request so in this case we will just use the anonymous user again and now our test should pass and the coverage should still be 100% all right and this is the last thing Mach actually probably one of the most interesting things so that's assume that we have a payment view where we are going to using where we will be using stripe and on our front end we will attach this stripe dot the check out J ass thing that stripe gives you so you can embed some JavaScript code and your website there will be a Pay Now button people click at the Pay Now button the stripe modal pops up people can fill in their credit card information they perceive when stripe checks the credit card and everything is alright they send us back token and then we will call a post call to our own payment view with that token and then this payment view needs to call on our server needs to call the stripe API with that token and charge some money so this is usually how the flow works and the problem is our server is now going to call it's going to do an HTTP request or HTTP requests to the stripe servers and we and every time we run our tests this will happen and a request if you have slow internet takes 1 or 2 or 3 seconds that will slow down your tests like crazy and probably stripe will knock on your door if if you like some people they have file system Watchers they run their tests every time they save the file like on a separate monitor which is actually very useful so that means you would be calling the stripe API hundred today and then maybe they will lock you right so we don't want to call that API at all for 64 speed reasons and for like rate limits reasons okay and let's also assume that by reading the stripe Docs we know that in pison oh sorry and pison we can call we can do pip install stripe and then we can call stripe charge and we fill in some the token and the amount and currency and stuff like that and then it will give us back this JSON string with ID and the charge ID and then we can save that charge ID into our database and we know B we received the payment right so this is basically the idea I will just copy the whole thing so oh I shouldn't have copy the whole thing only this part and we will need mock so mock is not standard library you need to do pip install mock in order to use this and basically the thing that the only thing that I always use is patch which is part of the mock library and patch can be used as a decorator and I I will explain this after you have seen the implementation that makes things a little bit easier okay so stripe is actually that's actually you can do pip install stripe and use their official API oh yeah and I also finally on the last slide I wanted to show how you can send emails and test this as well so we can do this as well here oh and we also need to import Django shortcuts and redirects okay but this is not important so what's happening we know that somewhere in our code we will call the stripe API okay and we know that this will send an HTTP request to the stripe servers and we don't want this to happen so we basically monkey patch away this module this module that's imported at the top of the file here and we will when the test starts running we will replace that with a so called mock object that's some some weird object that you can do anything with that object you can call functions on it you can call attributes on it you can assign values to it and it just allows anything for it to happen to it it will never say I don't have this function it will record that this function has been called on itself so and this is what patched us we are saying that in the module called Birdie so that is the folder name here birdee we have a file called views dot pi this is our user PI and we are importing stripe inside of that file and we want this to be replaced by a magic mock now so using this patch decorator make sure when we enter this function here we first monkey patch the stripe import and now we can pass in the monkey patched version of our stripe import into our test function so this is the fake stripe version now and then we can use this fake stripe version as a variable inside of our test and we can say that if somebody calls dot charge we set the return value to this this is what we expect what a successful stripe payment should return to us okay so and you know you can come up with anything here you can invent any kind of functions that should be called and come up with return values okay and then as usual we sent a post request this is the token that we got from the JavaScript front-end when stripe validates the credit card and finally our our view should redirect to some success page and it should also send us an email so that we know that we have just made some money and from Jango don't core I think import mail and this is the final thing that you know as we new comer you will ask yourself how do I test it if an email has been sent you can simply call mail dot outbox so every time when in your views when you call send mail mail dot up this mail will sit in your outbox right and we are using the local memory back-end so in our tests we can access mail door outbox and all on our males will be sitting in that outbox and open and when you write more test functions it's like every time you define another test function here the database will get deleted mail outbox will be cleared everything will be like completely clean and fresh and new so sometimes I write very long tests where I'm telling a story like user creates this kind of transaction then he changes some value of the transaction then this happens to seventh because I want to like I need otherwise I would have to setup the fixtures all the time but when it's already like a very obvious way of going through through the same view doing doing things in the same view and changing my objects I like to write it just into one test and reuse the objects that I just created right but if I'm testing completely different functionality where I really want to have a clean database I write another function name and I give it like test anonymous test locked-in test super user or whatever okay so yeah that's this work I see some error message white space at the end okay so let's see generic okay that's obvious here I should just import view and here's well tella and we wrote our Twitter clone with a payment page and hundred percent test coverage and oh yeah one last thing sometimes when your tests suite is very very big it takes a long time to run Maebh I mean for a medium-sized project it can go up to let's say 30 seconds for really big projects that can ten can be ten minutes then you will have continuous integration server every time you push your server will start pulling that branch and will start running the test so that you can go and take a coffee or just continue coding or something else and if it fails it will send you an email there's the usual set up but if you are and obviously that means if you are writing the test for the things you are currently working on you cannot always wait ten minutes for the whole suite to run through so then you have the chance to specifically call a test in a certain file by providing the folder name Birdie tests test view spy and then with this double colon syntax you can provide the class name and the function name so you can if you leave out the function name it will run all functions in that class if you leave out the class name it will run all tests in that file so that you can like be super granular about it or you you leave out that file then it will run all tests inside the Birdie app you might have a lot of apps so that might also make sense and finally if you want to put in breakpoints in to your tests you can use PI test set trace and then when the test hits that point it will just stop and on the console and you can you know inspect your variables and stuff okay so that's what I want to show today actually I went much faster than I thought so there are more interesting things that I can show maybe next month or whenever we don't have any other talks lined up which is testing template X testing Django management commands is interesting because they are also some kind of thing that you can't just instantiate and then call it there because usually the managed by commands they they have command line parameters and they might have X certain exit codes so you can test these things as well Django has some stuff for that testing sessions can be tricky you might have some views that write messages into the Django messages queue and those are usually saved in the session if you don't have a session object on your request object this will fail right usually you can just do request dot session equals empty dictionary because the session object is kind of like a dictionary but in some cases because the real session object is not really a dictionary it has some some more functions and if your view for some reason tries to call one of those functions then it will crash then you need to create some kind of fake session similar to how we create a fake request and I can show how we can do that testing with files the super interesting like how do you test an image upload field right and I can show how to do that and then we can go on how do you test a Django rest framework because the Django rest framework cannot use the normal request factory it has its own API request factory and you cannot just use the dot as view to instantiate your view classes your API views you have to pass in some other stuff so this is something that I can show and then PI tests has another plugin for running tests in parallel if your computer has multiple cores and you have like hundreds of tests you can actually you know make make use of all your course and run tests in parallel which will have cut the testing time in half all right thanks for listening I hope this wasn't too boring and if you have any questions asked now no questions really no questions okay yes we will still be here eating the rest of the pizza and if you have questions just walk up to me

Channel: Engineers.SG

Views: 22,877

Rating: 4.961165 out of 5

Keywords: engineers, singapore

Id: 41ek3VNx_6Q

Length: 80min 29sec (4829 seconds)

Published: Wed Jun 08 2016

Reddit Comments

[u/take_whats_yours]

Is there a fb group for these meetups in the region?