Django ManyToManyFields and Through-Models for many-to-many relationships

in the last video we saw how to link a Django model to another model using a foreign key relationship and that creates apparent child relationship where the parents can have many instances of the child connected to it and the child has a single connection to the pain so it's a one-to-many relationship in this video we're going to look at many to many relationships in Django and in databases and this is when both signs or both models in the relation can be connected to multiple objects on the other side of the relation we're going to see how Django creates Junction tables that link two models together in a flexible many-to-many way and we're also going to see how to use Django's many-to-many managers in order to fetch add and remove these many-to-many objects from a relationship so we'll learn a bit about querying these many-to-many relations and optimizing them and we're also going to look at how we can customize the junction table that Django creates that links the two size of the relationship together and we do this using three models that allow us to Define another Django model that can Define the behavior and of that many-to-many relationship and you can add custom fields and methods to that model just as you normally would with a Django model we'll dive into that in this video so let's get started now we have the restaurant model that we've been working with in this series and you can see below we have for example the rating model here that contains a foreign key to that restaurant so the rating is the child model here and it's applied to a single restaurant whereas a restaurant can have many ratings from many different users so this is a one-to-many relationship and I want to many relationship is expressed in Django with the models.foreign key field but what happens if there's a many-to-many relationship let's go to the Django documentation on many-to-many relationships and we're going to see some examples of these types of relations in this documentation so we have here a class called publication and another class called articun and in this example an article can be published in multiple publication objects and on the other hand a publication can have have multiple articles so this is a many-to-many relationship whichever one of these models we're working with it may have many Associated models of the other kind so it's many to many what can we do in our current application in order to build a many-to-many relationship well an example of this might be a staff member we have here a restaurant model and let's add another model to this application called staff and let's assume that one staff member can work at multiple restaurants and of course a restaurant will require multiple staff members so this could be a many-to-many relationship we want to open the opportunity that a staff member can work for multiple restaurants at the same time so what we're going to do is create a new model here called staff and that's going to inherit from djangles base model class and we're only going to add two fields to the staff model firstly the name of that staff member and we're also going to add another field called restaurants and that's going to be a models. many to many field this is another field type in the Django database model to the models store many to many fields and it's going to be linked to that restaurant model that we have below and because that restaurant is below I'm actually going to cut this out of here and paste it below the restaurant model so that we can add that reference quite easily there so now that we have this new model we need to generate the migration file so that we can add it to the database so let's run the make migrations command and among other changes here we have this one at the bottom where we're creating the model staff and once we've done that what we can do is we can run the migrate command that's going to create that change in the underlying database so if we go to the database browser for SQL Lite here we had 14 tables as you can see before we've added a new Django model so when we refresh this page we expect to see 15 so let's refresh this and we can actually see 16 tables now so we're going to get to why that is but you can see there is a staff table here core underscore staff if we browse this table you can see that there's two fields in this table ID and name now if we go back to the model you can see that we have the name field here and the ID is implicitly added by Django but what about this restaurants here it's a many-to-many field but we don't have a column in the table for restaurants and that makes sense because a column should only store one reference if it's a foreign key or one value if it's a primitive but in this case we could have the case where this staff member works for multiple restaurants so what can we do here to show this flexible minute-to-man relation if we go back to the database structure you can see there's another table that's been created here core staff restaurants and this is what's called a junction table or an association table if we right click that and go to browse that table you can see the columns that we have here we have a staff ID linking this table to the staff member and we also have a restaurant ID that links that to the restaurant so this is how we associate staff and restaurants together in a many-to-many relationship we have this Junction table that contains references or foreign keys to the two parent tables that we're linking together other so let's say we had a staff member called John who worked in five restaurants we would have John's ID here five times and then the associated restaurants that he works with would also be here and they would have their IDs in those rows on the other hand let's say we have a restaurant that's an Italian restaurant in the database and it has 40 members of Staff working for it in this table you would have Forte references for that ID and then 40 individual staff numbers as well so that's how you can link on both sides a many-to-many relationship it creates this Junction table in the database and Django will actually do that implicitly when you use a many-to-many field in your model so we can now create as many links between staff and database and vice versa as we need to in our application now I'm also going to leave a link in the description to this Wikipedia page this is on associative entities which is a term used in relational database Theory and it explains this concept of a many-to-many relationship and how it's joined together in this SQL tables you can read that if you want to know more information about this now let's get back to Django if we go to models.pi if we look at the rating model when we had a foreign key to a single parent restaurant we could access that restaurant directly simply by using the field name which is restaurant and when we access that it gives us back the restaurant model that's associated with that rating and then we can query for other details such as the name or the date opened or any other fields on this model now the question is how do we query for these many-to-many restaurants so if we had a staff member how do we get the number of restaurants that they worked in how do we get the names of those restaurants let's now see how to do that and in order to do it we're going to go to the scripts directory that we've been working with and we're going to go to the ormscript.pi file now the scripts directory contains a single function called run and we're going to fill the body of that function now at the top we're going to bring in the staff model that we've just created as an import from the core dot models module and then we can remove pass statement and what I'm going to do to start with is create a staff member in the database so let's create a variable called staff and we're going to use the staff.objects.create function and we'll give that staff member a name let's just call them John Wick and in fact just because I'm going to run this script multiple times I'm going to change this function from create to the get or create function and then we can unpack the Tuple that's returned from that into two objects firstly the staff member and secondly a Boolean telling us whether or not this was created so once we've created that staff member we can print it out to the terminal and we're also going to get the restaurants fueled on the staff model so let's go back here and we're going to print the type of that field out so it's going to be staff dot restaurants so now that we've added this code let's run Python manage.pi and then the Django extensions run script command we can reference the name of this script which is orm script and then we can run that and see what kind of output we get so we get back the staff object and underneath that we get back the type of of the restaurants field on this staff model and you can see that this object is what's called a many related manager so this is not a normal manager that you get when you call staff.objects.function this is a different type of manager and it's a many related manager and this is used to manage these many-to-many relations in Django now it turns out this manager Works quite similar to the other ones that you get in a normal Django model we can run queries such as staff.restaurants.org and that's going to return to us every restaurant that this staff member works at so I'm going to print this to the terminal as well and when we run this we're going to see that there's no restaurants associated with this staff member it's an empty query set and that's because we've created this member of Staff but we've not associated them with any restaurants in the database so how can we take our new staff member John Wick and add them or associate them with a given restaurant in the database let's go back to the Django documentation this page on many to many relationships and we're going to scroll cool down a little bit on this page and you can see here how they associate the article with a publication and it's using this related manager which has a function called add and you just pass in the publication in this case to that add function and it's going to create that relationship and store it in the database we're going to do the same thing if we go back to vs code with our staff member we're going to add a restaurant to this particular staff member and Associate that staff member with the restaurant so in order to do that what I'm going to do is change the all function to add and we're going to pass in a new restaurant here so let's do this outside of a print statement here we're going to call staff.restaurants.add and here we can pass restaurant.objects.firstin that's the very first row in the restaurant table we're going to pass that in and it's going to add that to the staff members restaurants associations so what I'm going to do is I'm going to remove this print statement and just above this add function I'm going to add another print statement we're going to get all of the restaurants that the staff member is in and then just below that will rerun that query and hopefully we'll see that there's a new restaurant in that particular Association so let's save this file and we're going to rerun the script and as you can see below initially we have an empty query set and that's because the staff member doesn't have any Associated restaurants but when we call staff.restaurants.add so the add function on this related manager that is going to create the association between this staff member and the first restaurant in the database and you see that when we re-query for that staff members restaurants that we have an output below containing a single restaurant model and if we go back to the database we can look at the staff table and we'll see that single member of staff that we have in the database if we go back to the database structure and look at the junction table now you can see the association that's been created here between the staff member with the ID of one and the restaurant with the ID of four so we can see two of the main functions on this many related manager we have the dot all function which will return all of the associated relations and we also have the dot add function for creating a new relation now what I'm going to do is show another few functions on this related manager one of the other ones is the dot count function which we can use to get the number of associated entries and we run that and in this case we get one because there's a single association between this staff member and the restaurants now as well as the add function we can also remove an association and to do that we can use the staff dot restaurants dot remove function and again we pass a model instance in here so we're going to use restaurant.objects.first this is a relationship that already exists in the database and when we run this script we expect the count output now to be zero and that's what it is and if we go back to the database browser for SQL light and refresh we are now down to zero rows in this table so let's go back to vs code we've now seen the add function the all function the count function and the remove function we're now going to see another function called set and we can use this to override any existing relay and completely reset them with whatever we pass in so let's see an example of that now if we use the staff dot restaurants dot set function what I'm going to pass in here is all of the restaurants and then we're going to index in and get the first five of them using that slice in Python so we're going to set this particular staff members restaurant list to the first five restaurants in the database if we save that and rerun this script we now get a count of five printed to the terminal and if we go back to the database browser here and refresh this table you can see we now have five rows in this table so the staff member with id1 now has five different restaurants associated with them in this table so that's all fine but what if you wanted to remove all existing entries from this relationship from this Association what we can do is go back to the Django orm here and we can run the dot clear function and that doesn't need any arguments passed to it so if we call staff.restaurants.clear what the clear function is going to do is remove all of the associations so I'm going to add this to the list of functions that we've seen here and then I'm going to run this script at the bottom and after the clear function is run the count is output and that is zero because we've removed all of those associations and we can see that in the database browser here when we refresh this the table now has no rows so we've seen the main functions that are available on this many to many related manager we have the add function for creating a new association between two objects we also have the all function which will get back all of the associations to that object we have the count function for querying the number of associations we have the remove function for removing an association and then we have the set function which will clear any existing relations and override them with whatever we pass in so we're able to reset the relationships finally here we have the clear function to remove all of the associations from the database for this particular object and there are some other functions as well like the create function and also of course when you query a database in Django when you get all of the objects you can also filter them down using the filter and exclude methods as well as order these objects with the order by Method so a lot of flexibility with the many related manager but it does add some methods such as add and remove that don't normally exist on a Django manager so let's see a quick example of how to filter these objects if we look at the staff dot restaurants related model we can use the dot filter function and what we're doing here when we're filtering is we're going to filter the restaurants by a particular Clause that we're going to pass in now if we go back to the restaurant model here what we're going to do is use the restaurant type and the restaurant type is a car field that refers to these choices above and what we're going to do is get back only the Italian restaurants that a staff member works for so what we can do is go to the orm script and we're going to pass the restaurant type in here and we can refer to the restaurant.type choices and that's the text choices subclass that we have in that model which has a field called Italian we can store the results of this in a variable called Italian and then we can print that to the terminal just below here now if we run this script we're going to get an output of an empty query set because this staff member that we have has no relations in the database at the moment but if we add a random subset of relations just above here with the staff dot restaurants dot set function what I'm going to pass in here is restaurant.objects.filter and in fact let's just get all of the restaurants and slice in and get the first five as we did before that's going to add the first five restaurants that we pull out of the database to that Junction table for this staff member and when we do that when we query just below to get the staff restaurants that are Italian restaurants we should hope to get back the results only that are Italian and that query set and again it's an empty query set if we extend this and set the first 10 maybe we'll get back some output now and indeed we do we get back the first Italian restaurant and a pizzeria as well so just to quickly go over this we have a staff member that we're pulling out of the database we're then setting the related restaurants on line nine here and we're going to set the first 10 restaurants in the database as Associated restaurants to this staff member and then finally below that we're using the many related manager here and we're calling the dot filter function and we're filtering that staff members restaurants to only those whose restaurant type is an Italian restaurant now because this is a many-to-many relationship we can do this on both sides of the relation we're doing it from the side of a staff member here we're using the staff dot restaurants related manager but what if we have a restaurant and we want to find all of the staff members for that restaurant so what I'm going to do is clear out this code here in the Run function and we're going to start from scratch with some new code before we do that let's go over to database browser for SQL Lite and refresh this table you can see that we have 10 rows in this table and these are all for the staff member with the ID of one and we also have a restaurant ID that's the associated restaurants for that staff member we're going to pull out a particular restaurant from this table let's get the one with the idea of name here so let's go back to vs code and I'm going to paste a query in here restaurant.objects.get and we'll pass the primary key of nine we're going to get that from the database and we store it in a variable called restaurant so how do we get all of the staff members associated with this restaurant what we can do is we can take our restaurant and we can use the staff underscore set and this is again our related manager so we can call functions like dot all on that so what I'm going to do is print this to the terminal and then we can see what the output is for this particular query so let's clear the terminal and run the script and you can see we get back a query set containing the staff object and what I'm going to do is go to models.pi I'm going to scroll down to the staff model and I'm going to add a Dunder string method here that's going to return the staff member's name so we can see that a little bit better on the terminal so when we rerun this script you can see the name of the staff member which is John Wick let's head back to our ormscript.pi and I want to highlight how we can go from the other side of the relationship which is a restaurant model and how we can access all of the related staff members using the name of the model which is Staff then the underscore set this is just a Django convention if you go to models.pi you can see we have the staff model that is the model that explicitly defines this many-to-many field so on this staff model we can access dot restaurants which is our related or a many related manager but we don't need to Define this relationship on the restaurant model as well so there's no related field called staff on this but what we do have is the ability to access the name of the model staff then the underscore set in order to get the other side of that relationship and fetch the related objects so I hope that makes sense if you go back to orm script we have the restaurant.staffset.all function and we can use all of the same functions that we used before in this particular manager so for example the add function or the remove function or the clear function or the set function lots of options here this is how we work with many to many fields using these related manage angels in Jungle let's now move on to something different and that's three models now to introduce this problem let's go back to the database browser for SQL Lite we have here this Junction table that links the staff member to the restaurant and this is created implicitly by Django but what happens if you need to store extra information in this table for example right now we only have the foreign keys to the two parent models but what happens if you had to store additional information in this table for example the staff member's salary at the given restaurant or their early wage in order to do that we need to tell Django we don't want it to implicitly create this we need to create a separate model in our models.pi file and then link that using a keyword argument to the many-to-many field so what I'm going to do is go back to models.pi and we're going to go through this many-to-many relationship that we've defined on the staff model and what we're going to do is we're going to add a second argument to this or rather a keyword argument called through and we can give it a name here in this case we're going to give it an name of Staff restaurant and this can be anything you want what we're going to do now is create a model with this name just below the staff model so let's create a class called staff restaurant and that's going to inherit from the models.model class like all other classes have done and the staff restaurant needs to link to the staff member so what we're going to do is add a foreign key here to the staff table and it also needs to link to the restaurant table as well so let's add that second foreign key on this flow model to the restaurant model now the setup at the moment is exactly the same as we have in this table we have two references to the staff and the restaurant table so what we need to do now is add the extra information that we need to add to this model so I'm going to go back to vs code and we're going to add another field here called salary and that's going to be let's just say for this example a float field and we can allow this to be nullable by adding the null equals true argument now we're going to see that it's quite difficult to add a throw table to a many-to-many relationship after the initial Junction table has been created so when we run djangles make migrations command just below here we're going to see that we create the migration file but unfortunately when we run migrate we're going to encounter some issues and you can see below we're getting this value error and that we cannot alter two or from many to many fields or add or remove the through argument to many to many fields so there are different ways of dealing with this but I'm just going to remove entirely the database from this project and rerun the migrations from scratch so what I'm going to do in order to do this is I'm going to start by closing the database browser for SQL Lite and then we're going to delete the sqlite database file from this project and we're also going to go into the corner application and we're going to remove all of these migrations from this project I'm going to delete all of these and I'm also going to remove the pi cache as well once we've done that we're going to regenerate the migrations with the Django's make migrations command and then we can run the migrate command and that's going to run all of these migrations for us finally now that we've recreated the database we need to run this management command called create data this was responsible for populating our database with some test data so let's run the python manage.pi create data and that's going to run through and create all of those objects in the database so we've recreated our table I'm going to load database browser for SQL lightener and you can see the database here if we look at the junction table that we had and before we only had the restaurant ID in the staff ID now you can see we have a salary column as well and that's because what we've done in the models.pi file just to go over this again on our staff model where we Define the many to many field we've added the through keyword argument and we've linked this many to many field with this Junction table called staff restaurant and then we create the model just below here where we have the two foreign keys that link together the relations between those two objects and we have the salary as well so what we've done with the salary is we've defined additional data that we want to capture on this many-to-many relation so for each association between these two objects we're also capturing some salary information in this case and as a page in Django's documentation on this particular issue where you need to Define extra fields on a many-to-many relationship I'll leave a link to this below the video in the description and as it says here for situations where you need to Define extra data Django allows you to specify the model that will be used to govern the many-to-many relationship and that's what we've done here with this staff restaurant model so what I'm going to do is go back to ormscript.pi and we're going to highlight some ways to create these associations where we now need to account for this salary field as well as the two foreign keys so what I'm going to do is remove this code here and we're going to bring back this code where we fetch the staff member with the name John Wick and we're also fetching the very first restaurant from the restaurant table in the database now there are two ways that we can make an association between the staff member and this restaurant firstly we have access to a new way of doing this that we didn't have before if we go to models.pi we have this new model called staff restaurant that's used to govern this many-to-many relationship so I'm going to copy the name of this model and go back to our script and import that at the top of this script and Below we can use this model just like any other Django models for example by calling.objects.create and to the create function we can pass keyword arguments for example we can link the staff member that we've pulled out of the database to this object and we can also link the restaurant as well but we do need to account for that salary field so I'm going to pass a salary in here and let's pass 28 000 into this field and what I'm also going to do is pull out the last object from the restaurant table and we're going to create another Association below so let's copy this statement and paste it below and this time let's change the salary to 24 000 and we're going to change the restaurant name here to restaurant two just to differentiate it from the other restaurant so what we're doing is creating two associations between in the staff member and two different restaurants here let's now run this script and we can see the output if we go to a database browser for SQL Lite we have no rows in the table but if we refresh after running that script we now have two rows and you can see we're storing that additional data for the salary in this table so this is important when you have some sort of connection between objects but we want to store more information than just the references to the parent objects we can use a throw table in order to do that in Django and if we go back to the script here I'm going to remove this code and what I'm going to do is use the staff restaurant.objects.filter function and again we can query this particular model just like we would any normal Django model so I can get back all of these associations where this particular staff member John Wick is involved by passing the staff equals staff to the filter function so let's store the result of this query in a variable code staff restaurants and then we can look over these for s in staff restaurants one I'm going to do is print the salary from this we can just access it as an attribute on that model just like we normally would and when we print this to the terminal just below here you can see we get back the two salaries for the two associations that we have in the database so this is one way of creating these associations that are stored in our through table but what if we wanted to use the same methods that we used earlier in this video for example the dot add function in order to add a new Association what I'm going to do is remove this for Loop that we have here and just at the top I'm going to use the staff dot restaurants.clearfunction that's going to remove all of these associations from this table that involve that staff member with the ID of one so let's go back here and what I'm also going to do is remove the call to adult last and we're going to save this and just run the script so that we can remove those associations if we go back to the database browser you can see we no longer have those rows in the table so let's go back to vs code and we're going to try and take this staff member and add this as one one of the restaurants and we're going to try and do this using the function that we saw before so staff dot restaurants which refers to the many related manager which has that dot add function and if we pass the restaurant in here that worked before but we're going to see an issue now that we have that extra data defined on the throw table if we save this and run the script and then go to the database browser for SQL Lite you can see the output that we're getting here we have the association but the salary is now and that makes sense because we've not told Django anything about a salary here all we're telling it is we want to add this restaurant to the related objects for this staff member so how do we add an Associated salary to this particular query with the many related manager dot add function well to do this we can add a keyword argument here called through defaults and this will be a python dictionary which will map each additional field that we have on Earth through a model staff restaurant to a particular value now the field that were interested in as the salary field that's the extra fuel that we don't have by default when we create an association unlike the foreign keys so let's go back to our script and we're going to add that as a key here and we can map it to any value we want so for example 28 000 if we save that and run the script and go back to the database browser for SQL Lite if we refresh this you can see that that has updated and we now have a salary in this row so let's finish this video with an example of prefetching many to many fields using Django's orm now we saw in the last video that we can prefetch related objects in order to make our database queries more efficient in Django what I'm going to do in this run function is I'm going to set up some test data and then we're going to access that data in our view and we're going to look at Django debug toolbar and then we're going to optimize that query very quickly so that it doesn't send as many queries to the database so what I'm going to do is remove this code above here and we're going to create a set of relations for the staff member to 10 restaurants in the database so what I'm going to do is use the staff dot restaurants dot set function and to that we're going to pass restaurant.objects.org and when we have all of these objects we're going to slice in and only get the first 10 from the database table for the restaurants now because we're using the set function on the many related manager we also need to specify the salary here so what I'm going to do at the top is import the random module from Python and then we're going to pass the through defaults dictionary to the set function just as we did earlier for the add function and I'm going to do this on a new line so that we can see these more clearly the key that we're going to be setting is for the salary for each of these relationships and what we're going to set this to is a random integer and let's set that between 20 000 and 80 000 so we're going to get a random number between those two values and set that as the salary for these Associated restaurants let's run the script at the bottom and we can go back to the database browser and refresh and we see that we now have 10 rows and the table you can see that many of these salaries are the same so if you need to have distinct salaries here this method is not going to work but it's going to be fine for our example let's now go back to vs code and we're going to go to viewers.pi and I'm going to remove all the codes from the previous video from this View and what I'm going to do is go to the top and from the code. models module we're going to import that staff Restaurant Junction model that we created earlier in the video and then within the view function I'm going to create a variable called jobs and that's going to be equal to staffrestaurant.objects.org now what we're going to do is we're going to iterate over each one of these records and we're just going to print some information out for each of these jobs so we're going to print out the restaurant associated with the job and we're going to get the name of that restaurant and as well as the restaurant we're going to also look at the job dot staff property this is the staff member associated with the many-to-many relation and we're also going to get their name and print that to the terminal as well so let's just quickly explain what's going on here we'll get all of the staff restaurant objects that correspond to the rows in this table here and then when we get them we're iterating over each one of them we're printing the restaurant's name and we're also printing the staff member's name to the terminal so let's save this and run Django's development server and we're going to go to the browser and go to this page now I'm on the page now and you can see that nothing is displayed on this page but on the right hand side we have Django debug toolbar now when we click the SQL tab you can see we are executing some similar queries in this output now the reason for this if we go back to vs code is because when we pull out these objects from the staff Restaurant Junction table when we then iterate over them and follow the foreign key to the restaurant and the staff table we are actually issuing a new query for every single one of these operations within the for Loop now the solution to this is to use the prefetch related function that we saw in the last video and we can pass the related objects in here as arguments and on this staff Restaurant Junction table we have foreign keys to the restaurant and also to the staff table so if we pass those in we can go back to the browser now we have 22 queries that were executed before but when we refresh this page you can see that's cut down to three queries that's much more performant because it's doing some of the work in advance so that can result in better performance and your Django applications when you're working with many to many fields so that's all for this video on many to many films and through models the key takeaway from this video If we go back to models.pi is that when we create a many-to-many field in Django we are creating it on one particular model and linking it to another model and we have the option of specifying a through model if we need to capture additional data on that related table and the through model is responsible for governing the many-to-many relation it contains foreign keys to the two parent models and then you can Define any additional information for example the salary here on that particular model and we've seen methods in this video of associating objects together in many to many relationships and we can do that with functions such as the many related managers add function the remove function the clear function the set function and so on so if you have any questions leave them in the comments if you've enjoyed the video please like And subscribe to the channel and we'll see you in the next video

Channel: BugBytes

Views: 11,176

Keywords:

Id: MECLUHlgF2w

Length: 33min 22sec (2002 seconds)

Published: Thu Aug 10 2023