Django & HTMX App - Custom Managers | pytest & factory-boy for unit tests

in this video we're going to extend what we've been building so far and we're going to add some new features to this project with Jango and HDMX so if we go to models.py we have this transaction model here that's the central model of our application and in this video we're going to take the amount for each transaction as well as the type of transaction and we're going to calculate totals based on this data so for example you might have a bunch of transactions that have a type of an income or an expense you might want to Total that up in the database we're going to use jango's orm aggregation Tools in order to do that and the way that we're going to do that in this video is by adding some custom query set methods and then creating a custom model manager on the transaction model and we're also going to write tests for that logic using p test and the factory boy package for generating fake data finally after we've done that we're going to display the results on the UI and that's all going to be integrated with HTM X so let's get started what we're going to do to start with is we're going to create the methods to calculate the totals for each type of transaction so across the whole database for the given user we want to see how much have they earned an income and that might be between two dates we're going to see that later in the series and also how much expenditure has that user spent and again that can be filtered between two dates now what I'm going to do to start with in this video is I'm going to create a file called managers dopy now before we do that actually on the left hand side here's the structure of the project that we have and each video has its own directory now you can get get the code for these videos on GitHub there's a link to this repository below the video we're going to copy video 2 which should be in the repository and we're going to copy that to a new folder that I'm going to call video 3 and then I'm going to take video 3 which is a copy of the previous videos code and that's going to be the starter code for this project so I now have video 3 open in the directory and we have the python virtual environment activated what we're going to do is go to the tracker Jango application and I'm going to create a new file in here and as I said that's going to be called managers. pi and if you're creating custom Jango model managers or quets you can put them in a file called managers. piy or if you prefer query sets. piy and what we're going to do here is we're going to create a custom query set and that's going to have four custom methods that we're going to add to that query set so let's start by importing jango's models from django.db we'll import the models and what I'm going to do here is define a class that I'm going to call transaction query set so that's quite a long name but it's going to be called transaction query set and that's going to inherit from something in jangle and that's the models. query set class now when you override these query sets you can Define methods in the class so I'm going to define a method now called get expenses and that's going to take self as an argument because it's a method in the class and then what we're going to return here is the normal query set but we're going to add the filter statement and we're going to filter by the type of this particular transaction let's go back to models.py and you can see that the type of the transaction is a car field but it's linked to some choices and it's these choices here on lines 19 to 22 so a transaction type can be an income or it can be an expense so for this method get expenses we are going to filter down all of the transactions to only those that have a type of expense so that's a very simple method that we're using on the query set but it's going to allow you to define a nice API around this data and we're also going to copy this and I'm going to paste it below and we're going to Define another method and I'm going to rename this to get income and that's a very simple change we're going to change the filter to filter down to the transactions that have a type of income now these are the base two methods that I'm going to use and we're going to build up on these with a couple of extra methods now what we're going to do is Define a method called get total expenses and again that will take self as a parameter now what we want to do here is we want to filter to the expenses and then we want to aggregate that data and return a total which should be a number so let's start by filtering the query set by calling get expenses so it's self. getet expenses and that's going to filter Down based on this line of code above and then that's going to return to us another query set we can chain additional methods to that and we're going to chain the do aggregate method to this and we're going to Aggregate and that's going to use a models. sum and we're going to get back the total for the expenses now I'm going to move this to a new line here just so that we can see it better so what we're doing is we're aggregating and we want to get back a total so we're giving the field that we're going to get back the name of total and we're calling models. sum now when we want to sum over a particular column in a table in jangle we just refer to the name of the field and the field name is the amount you can find that in models.py and that's a decimal field so we're going to sum the amount and that's going to give us back a dictionary in Python after we aggregate that and we need to index into that dictionary and get the total so let's grab that key of total and if we don't have any objects we want to get back a default value of zero so just to cover what's going on here we have a method called get total expenses and the logic for that is to filter down the transactions to only the expenses and then take all of the rows for those expenses and aggregate to get the total over the amount column now we're going to copy this method and I'm going to define a very similar method below here and again that's going to be called get total income and the method it's going to call is the get income method that we defined on the query set and then we're aggregating again over the amount column and the rest of this logic can remain the same so why are we doing this what we're going to do is save managers. pi and we're going to go back to models.py and I'm going to scroll to the top here and from managers we're going to import that new query set that we defined so that was the transaction query set let's import that and copy the name of that and then we're going to go down to the transaction model and just Bel the fields that we've got defined here I'm going to create a value called objects and that's the default name for the Jango model manager and what we're going to do here is we're going to paste transaction query set and the query set subclasses in Jango they have a method called as manager if we call that method that's going to create an instance of a model manager in jangle and it's going to have all of the custom methods that we defined on this query set available on that manager now if you want to know more about this there's a page in the Jango documentation on managers I leave a link to that below the video and there's a section on the right hand side it's this one here calling Custom query set methods from the manager so if you define a query set and that inherits from models. query set as we're doing here what you can actually do is you can define a manager that returns that query set within its get query set method but the short hand for that if you scroll down to this section is you can call the as manager method and then add that to the model and that gives you access to the model manager with all of the normal methods but also with the custom methods on the query set that you define so there's a couple of different ways of doing this let's now go back to VSS code and I'm going to save models.py we now have a custom manager on the class and we're going to test that custom manager out by expanding the shell here and we're going to run Python manage.py and it's the shell plus command from Django extensions that's going to load up the Django shell with some imports now at the bottom of the Shell let's refer to the transaction model here you can see that that's been imported and it's available to us and what we normally call on these models is methods on the manager so the manager is the doobs property on the model and normally we can call methods such as transaction doobs doall and that's going to return all of the rows in the database but what I'm going to show you now is that you can actually refer to these custom query set methods that we defined so if we go back to managers. piy we defined a method here called get expenses and for some reason the terminal that I'm working on seems to have been killed here but this is the method that was going to define or was going to call it's called get expenses so let's load the terminal again so it's python manage.py shell plus and then we're going to get a reference to the transaction model and that has the manager objects and we're going to call getor expenses now that's not normally defined on ajango model manager it's a custom method but when we call that you can see we get back transactions and these are all expenses because of this filter statement that we've defined in that method what we can also do is is we can get the total expenses so I'm going to tab up to get back to that query and I'm going to replace the method that we're calling with get total expenses and actually that's not returning anything because I forgotten the return statement apologies for that let's go back to the method and we need to return the value here from the aggregate function so let's return that and again we'll go back to the terminal here and if we go to Shell plus we can call transaction doobs Dot and then the method was get total expenses and the number we get back is a decimal instance and it has this value here so this is the total transaction amount for expenses that are in this database table let's now move on and note that you could do the same for get total income and in fact let's quickly do that on the shell here at the bottom if we call the get total income method so I'm going to change the name of the method here you can see we also get back a decimal instance so we can use these in our application to actually get the totals back and aggregate the data that the user has added to this application later on we're going to show this on the page but before we do that it's important to write tests when you're dealing with this kind of logic when you're aggregating over data and you want to make sure what you're returning is accurate so what we're going to do in this video is install a couple of packages first of all we're going to install py test and we're also going to install a plug-in called pest Django and that provides a lot of utilities for writing tests when you're working with Jango and finally we'll install a package called Factory boy and this allows you to easily generate test data for your tests now I've left a link to each of those packages below the video let's go to the terminal and we're going to copy or sorry paste in this pip install command so we're installing py test we're installing py test jangle and Factory boy let's execute that command and what I'm going to do while that's executing is go to the py test documentation so P test makes it easy to write small and readable tests and it can scale to support complex functional testing for for applications and libraries what I'm going to do in this video is start by creating the P test configuration file so let's go back to VSS code and we're going to go to the root of the project next to manage.py and we're going to create a file here called pest. ini and we're going to add some settings here for py test so let's define a block for py test I'm going to add a couple of options here so let's set this to- RP and that's going to allow us to see the output of these tests and print statements on the terminal we're also going to add a reference to the jangle settings module and that's going to tell py test how to load in the jangle settings now at the moment in our project directory we have a settings.py file the project directory here is called Finance project and then it's a settings file so we can reference that with this syntax and of course if you have a dedicated settings file for your tests you can reference that here in the Jango settings module and the final thing I want to add is a python files variable and we're going to set that to tests.py and also testore wildcard dop and finally wildcard tests.py so basically any file that has the word test in it we want P test to find that within this project and find the tests within that file and run those tests now that we've done that we're going to go back to the tracker directory and I'm going to create a new file in here called factories. pi and this is where we're going to import a module called Factory and that's basically a module called Factory boy that's going to allow us to generate fake dat data what I'm going to do is go to the documentation for factory boy and I'll make this text a bit bigger here this gives you the ability to replace fixtures in jangle now fixtures are Json files and these are static and they're hard to maintain as your project changes whereas Factory boy defines a class-based approach so you can build up complex objects and relationships using Factory boy and as your project evolves I think it's easier to use this and change this along with the models in your application so I think this is a slightly better approach than using Json fixtures and jangle for your tests what we're going to do is go to the section on using Factory boy with orms and with d jangle what we can do is inherit from a Jango model Factory class and I'm going to copy the code here for this user factory and we're going to Define that in vs code so let's go back to VSS code and paste this in so we have a user Factory class that's inheriting from Jango model Factory within that we have a meta class and we need to link this to the user and the user in this application is in the tracker. user project so the tracker application and the user model and then we're getting or creating based on the username we can leave that as is and you can Define some defaults for your model here and you can see the default username is John when you actually create a user Factory you can override these defaults if you need to we'll see examples of that later in the video what I'm going to do now is import the models that we have in this application so from tracker. Models we're importing the user model as well as well as the transaction and category model and we can actually set this directly to the user model now and what I'm going to do is replace the username property here with some properties that I've written so Factory comes built in with the faker module in Python and Faker allows you to generate some fake data simply by calling some functions for example here we're passing the first name generator to the faker object that's going to generate an example or a dummy first name and we can do the same with the last name field on the user for the username we can define a factory do sequence and the sequence that we're defining here takes a number in and it's going to create a user based on that number what I'm going to do now is create a couple of extra factories now if you're not interested in the factory stuff you can skip to the next section of the video what we're going to create now is a category Factory and again it's going to inherit from this Jango model Factory class so let's paste that in there and The Meta class should link this Factory to the category model and again D Jango get or create I'm just going to copy that from above and we can paste that in here we're going to set that to the categories name field so Factory boy is going to try and get a category with a given name it generates and if we look at the category model you can see that it has a unique constraint defined so we should only have one category per name the database is not going to allow duplicates so we can set the get or create field to the name field on the model now when Factory boy generates a fake category name here what we want to do is we want want to define a field and I'm going to make this on a new line so we can see it clearly so the name field we want to set to a factory do iterator and we're setting the values of that iterator the sequence that we want to use to this hardcoded list here that contains some default categories such as bills housing and salary and so on so that's our category Factory I'm going to scroll down and we're going to Define another class now and that's going to be called transaction Factory and again it's going to inherit from this class here so I'm going to copy this and we can paste that in here we need to Define that meta class so let's define that and Link this to the transaction model and then we're going to define the fields here for the transaction now the user that's attached to the transaction if we look at models.py and scroll down here that's a foreign key field that means that the transaction has a foreign key reference to a parent user model so in order to replicate that in Factory boy you can create what's called a subf factory we can call Factory Sub Factory and pass in the user Factory that we created above and what that means is that when we create a transaction using Factory boy it's automatically going to create the user that's attached to the transaction by looking at this Factory here and returning an instance based on that so that's the user and we can do something very similar for the category foreign key on the on the transaction model it's going to be a factory do subfactory and this time we're going to link that to the category Factory so what else do we have on our transaction model model we have an amount and I'm going to set a default here of just the number five we have a date and I'm going to set this again to a factory. faker call and let me scroll down here and we'll Define the parameters for this on a new line so I'm going to use a generator in Faker called date between and as the name implies that takes a start date and it also take takes an end date here and it's going to generate a date in between the two so for the start date let's create a datetime object and before we fill that in we're going to scroll to the top and and from the the python date time package we need to import date time so from date time let's import the date time object and we're going to scroll back down here I'm going to set this to a value so I'm going to set the year to 2022 and let's set the month and the day to New Year so the 1st of January 2022 and let's set the end date to datetime do now and we need to convert that from a datetime instance to a date so we're going to call the date function and we also need to convert the start date to a date object so let's call do date on that as well and one final field here I'm going to paste it in so I've pasted this field called type onto the transaction Factory and again we're going to reference Faker here and it's the random element generator what we pass to that is a sequence and the sequence that we're passing is a list comprehension that looks at the transaction type choices so basically that's going to be income and expense and it's going to extract one of those and return a random element for each one so hope that makes sense I'm now going to save factories. pi and we now have a factory for each of the models in this application that allows us to very quickly generate fake data for those models so let's test that out again in the Jango shell I'm going to run Python manage.py and again it's the shell plus command that's going to load up the shell and then what we're going to do is import the factories so at the bottom here from tracker. factories let's import every one of those factories that we defined and then I'm going to instantiate a trans transaction Factory here and when we do that we get back a transaction so what we can do in Factory boy is not only generate a single transaction but we can actually generate a batch of these as well so I can call create uncore batch and I'm going to create a batch of 10 transactions when we execute that we get back 10 transaction objects that are saved to the database and they're saved to the database because we inherit from the jangle model Factory class that will automatically save them unless you tell otherwise and I want to show one last thing we can do with the transaction Factory so I'm going to create a variable called transaction and we're going to instantiate the transaction factory and we're going to pass through a default value for one of the fields so for example on the transaction Factory we're setting by default the amount to a value of five but when we instantiate the factory we can override this by providing an argument and let's say we wanted to set this to 1155 we can do that here and that's going to give us back a transaction object when we look at that object we can look at the amount on that and we can see that we get back the value that we passed in so that allows us to override what's defined by default on the class if you need to do that infrequently in your tests you might want to do that especially when you have relations between one object and another so this is all working now we can exit out of the shell and we're going to start writing some tests so what I'm going to do is go to the tracker application here and I'm actually going to create a new folder that I'm going to call tests and if we scroll down to the applic tests.py file we can actually remove this once that's gone we can go to the tests directory and I'm going to create a new file here and it's going to be called conf test.py what we're going to do in this file is we're going to write what's called a fixture in P test and if you're not sure what that is we did a video on py test recently and that contains some explanation on fixtures it should be appearing on the screen now in this file we're going to import P test and we're going to import the transaction Factory that we defined in factories. pi now what we want to do is WR a fixture that allows us to inject a batch of transactions into a py test function so I'm going to decorate this with pest. fixture and then we can write a python function under that decorator called transactions now this particular function what it's going to do is return a batch of these transactions so we're going to take the transaction Factory that we created and we're going to call a method called create batch on that and that allows you to pass a number into that method and that's the number of the instances that you want to create for the given model in this case for the transaction model we're going to create 20 here so let's pass the number 20 and save conf test. Pi now any fixture that you define in a conf test.py file is going to be available by default without even needing to import it in the test.py files So within the tests directory we're going to create another file and it's going to be called testore models.py and we can now write some py test functions in this file so let's import Pest and we're also going to import the transaction model and I'm going to define a test here called test query set get income method so on the query set we wrote a method called get income if you go to managers. piy remember that was the method that is filtering down the transactions to only those that have a type of income and we now want to write a test for this method so let's go back to our test file so this is going to test the get income method that we defined on the query set what we need to do though is inject the transactions that have been created by that factory so we're injecting something called transactions here and that matches the name of the P test fixture so this transactions fixture here is going to create 20 transactions using the Factory and then by injecting it in as a parameter here we're going to have access to those transactions within this py test function now we need to add a decorator to this function here and that's because by default py test is not going to let us access the jangle database so what we can add here is a py test Mark so it's py test. Mark and there's a mark called D Jango DB now remember we installed the py test Django plugin this is provided by that plugin and if you decorate a function with this it means that your pie test function can access the test database so if you're doing any kind of logic that means that some data is going to get written to the database or read from the database you need to add this py test Mark to your function and that's the case here because these transactions that are created and stored in the database because they're being stored in the database we need to actually use this Mark here to tell p test that that is okay now what we're going to do is we're going to get a query set here by calling transaction doobs and then it's going to be the get income method that we defined that's the custom method on the query set and that's going to filter the transactions and only return the incomes now once we get back that query set we're going to check that it contains more Than Zero Records so let's check if query set. count is greater than zero remember we're getting transactions here and then what we doing in the database is we're getting them by only the transactions that have an income type but remember on the factory if we look at factories. Pi the type is randomly generated from the choices so when we create these 20 transactions we may actually get back 20 that all have expense type and therefore the get income method might return an empty query set now actually that might not be an ideal method for a test so what I'm going to do is actually change this up rather than using random element what I'm going to do is change this up and just like we did above here when we created an iterator for the name we're going to also do that for the type field as well so let's scroll down here and use factory. iterator and I'm going to remove this random element stuff and we're just going to pass in a python list here containing the transaction type choices and that means that the first one that's going to be generated is going to be an income the next one is going to be an expense and it's going to cycle through that process and create one income and one expense and so on and that means if we go back to the test the get income method is guaranteed to return some transactions with the type of income so now we can remove this and I'm going to create an assertion here so the query set. count should be greater than zero and the important test for the get income method is we want to assert that all of the following are true so we're going to create a list comprehension and we're going to say transaction. type is equal to income and that's for each transaction in the quy set so the quy set we're pulling out here gets all of the income values using the custom query set method that we want to test and then what we're doing here is we're asserting that every sing Single transaction that we get back in that query set has a type equal to income so that's the expected behavior of the get income method and what we can also do is copy this method and I'm going to create another one just below and I'm going to change the references to income to get expenses so on the model manager we're going to call get expenses and we're going to assert the same thing except here we're going to change what's expected to expense and that matches the second element here in that tple of choices so let's now test this out what I'm going to do is expand the terminal here and I'm going to clear it out and what I'm going to do is run P test so let's see what happens when we run the P test command and we get some output here and you can see that it's collected two tests which makes sense because we wrote two test functions and these two tests seem to have passed we can see in green at the bottom two tests have passed in 1 11 seconds so that's good we now have working tests for the query set methods that we defined we're now also going to add tests for the total methods so we have two methods in the query set get total expenses and also get total income let's go back to the py test file here and we're going to write two more tests so I'm going to scroll down and we're going to define a function called test query set get total income method and what that's going to take again is the transactions so we're going to create those 20 transactions from the p test fixture we also need to access the database for this so let's copy pest. mark. Jango DB and decorate this test and what we're going to do is get back a value for total income we're going to call transaction doobs doget total income and that's the custom method that should return the total over all income transactions and then let's create an assertion here we're going to assert that total income is equal to a value and I'm going to use the sum function that's the built-in in Python and we're going to look at T do amount here for each T in the transactions that have been passed into the method from the fixture but we're only going to consider the transactions whose type is equal to income so I'm going to get rid of the sidebar here so we can see this as a full line of code what we're doing is we are injecting transactions there should be 20 of those and then we're calling get total income here that should hopefully return a number for the total income over those transactions and then we're checking if that is equal to this statement here now this statement uses the sum function and it uses a generator within that we're looking at each transaction in the generated transactions from the P test fixture but only those that have a type of income and we're summing up the amount for the query set that's returned from this statement here now we can do something very similar if we copy this function again and I'm going to paste that just below here and I'm going to change the references from income to expenses so we have a variable called total income let's change that to Total expenses and call the get total expenses function and then finally we're going to take the total expenses that are returned by your custom method and we're going to check that that's equal to this value here we need to change the type though to expense at the end of this statement so now we have two extra P test functions that are going to test the other two methods that we had in our custom query set let's again run P test on the command line and we're going to see that it's collected four items and all of our tests have passed as as you can see at the bottom now there are some ways that we can probably improve this process what we can do is go back to factories. Pi and rather than hardcoding an amount here of five we could set that to a random number for example so that each transaction has a different value for the amount or we could inject a different value using our fixture so let's bring back the sidebar and we can go to con test.py one way to do this here if we import the random module is to pass an amount that has a random value so for example random. randint between value 0 and 100 we could pass that in randomly for each one of the batch that we create here but we're not going to do that here let's just leave it the way it was our tests are passing and we have some basic tests around this query set functionality that's been written in py test and it uses these Factory boy classes in order to generate test data now the next stage in this application if we go back to managers. Pi is to get the values back from these methods and display them in the user interface so what I'm going to do is just start the jangle development server I'm going to show the user interface at the moment and when we go back to the application we have this UI at the moment we have a transactions list that contains a table of all of our transactions what we want to show above that is the total values so for example we want to sum up all of the income that's been generated we want to sum up all of the expenses and then we also want to take the difference between the two and that's going to give us our net income so we want to show that information just above the table and later we're going to add some date filters to the right hand side with HDMX and that's going to allow us to filter our transactions between two dates and we want those totals that are going to be above this table to update dynamically when we filter between those dates that's all coming later in this series in the next video we'll implement the user interface to show these totals and that's going to be hooked up with HTM X again in this video what we've done is we've created a custom query set object with these custom methods and then we registered that on our model so if we go to models. Pi we created an object property and set that to this statement here and then we wrote some tests around this functionality so we created a p test fixture that will inject some transactions and that's using a factory that we created and then after we've done that we can use the transactions if we go to our test functions here we're injecting that in and then we can Define the logic and use P test to test the actual methods that we've created on that query set so that's all for this video if you've enjoyed the video give it a thumbs up and subscribe to the channel for more and consider buying the channel a coffee if you're finding this content useful it'll encourage me to make more videos and thanks again we'll see you in the next video

Channel: BugBytes

Views: 2,427

Keywords:

Id: n47VFgqG-E8

Length: 31min 16sec (1876 seconds)

Published: Mon May 13 2024