Systematic Bug Squashing - Building SaaS with Python and Django #108

hi welcome to building sas with python and django my name is matt lehman and on this stream we work on a django app working you through the whole process of building an app that is online serving real customers and going through my journey of what it makes takes to build a software as a service tonight we're going to focus on bugs and ux polish and various things that have been popping up over the course of running this school year so i'm running a homeschool application and so it serves um homeschool families and so the school year element is because it's serving home schools but that's what we're going to do we're going to go over some ux stuff so you're free to ask me anything about python django or web development that's what i'm here for and we'll just jump into it i actually didn't have any particular issue in mind so there's a bunch of things that have been popping up we'll stare at one and see if it's a good fit and then if not we'll move on to some other other one so my um so this scheduling bug appeared in the application and this customer that's that's commented on here is my wife who happens to be one of the customers of the site and she was outlining a case that she saw and i think i need to try and replicate the case so there's some logical bug in the system and she was in some scenario trying to check if it um what was going on and this is basically the scenario so she's looking ahead to a future week and had a course that had three different tasks in it and had marked off the first task and so she had a break day on a friday because we were on vacation and looked ahead to the next week and wanted to print off like what are we working on next week when we get back to school after our vacation is over and unfortunately what she recognized after the fact is that the schedule showing instead of the second task so task b i'm just sequencing these abc she instead saw task c so it tells me that something is wrong in my scheduling logic and this is one that i need to investigate and she looks into the future often so i want the future reports to be accurate so there's some kind of edge case or condition that i have failed to consider in the scheduling logic and that's where i want to spend some time digging into that so let's get up the code and figure out see we're going to do two things we'll probably start by trying to recreate the scenario and that's that is crucial if i can't do that then i'm going to have a pretty hard time figuring out what what the bug is so we'll poke around with the user interface locally and do what data would i think replicate the problem and then we can um if i can produce that then we can take that same kind of data and produce it in a unit test and from that unit test we can work backwards and figure out what actually went wrong with the scheduling logic and fix the problem in the scheduling system so let's take a look at what we have get the site going so i'm running from localhost here this is at port 5000 and i have not much data in this application and you can see here we're on a a wednesday and this is the time of this stream and we've got a course here that has three items in it so this might actually the science course might be a perfect example if i can finagle it so we're going to attempt to what can we do to try and synthesize this i think what i want to do is set a break day for thursday and friday and mark this lesson one activity as complete today and then look ahead to next week because she says she was doing a look ahead to the next week and it was essentially this item right here would have listed the wrong activity so that's what we're going to attempt to do so let's go to the school year and mark off the 22nd and the 23rd as a break day so we can come into the app here and oops um set we can add a break for 7 22 20 21 and 7 23 2021 and we don't need a description i think so now we have two days marked off as a break if we return to the week view you can see that this rest of the schedule has been cleared for the two days because that's an important element of this the way my app works is all of the tasks are slotted not for specific days but they're in specific slots and based on the recurrence of whatever course there is so this course science happens to run every day monday tuesday wednesday thursday and friday and it will try and fill up all of the activities for that day and the reason that there's not anything listed on monday and tuesday is the system is designed specifically to advance the stuff so if you happen to miss monday or you have and you happen to miss tuesday rather than forcing a user to go and shuffle things around it has this logic built into it just kind of slide things along so that's why there's nothing listed here so what i'm going to do is mark this lesson one task as complete all right so now we're marked as complete and let's go to the weak view excellent so we've recreated the scenario that's really good news it's not something that we'll have to hunt down it means it's a pretty realistic case it has something to do with break logics logic so the way this works under the covers is um the this is a course in the in the modeling actually let's open up the model diagram because it'll be useful to view so i have a model diagram here and let's jump over and take a look oops let's take a look at um the course where are you okay so over here you can see there's this course model and the course is a summation of tasks it's a container for a set of tasks so the activities that are going to go slot into each day and the way that i display in the app what is going to be on that week is by doing an index into the database so i treat the database table as a an ordered list of tasks they are ordered by the time that they're added that's when the person wants to do them in their schedule and the first thing to do is filter out any tasks that are complete so i take the entire set of completed tasks and that is by having anything with a course work is considered a completed task and i take all of those ids and i say extract that to separate the sets so once i have the set of incomplete tasks then it's a matter of indexing into that remaining list to decide what to display the challenge is that with this system and the reason why the scheduling is so hard and frankly the reason why this is a compelling application like this is the value add for this site is that looking ahead is like the killer feature here the getting this right of knowing what to show and factoring in um weekends or holidays or just different schedules like it doesn't there's nothing that says a class has to run monday through friday so all of these things kind of combine to make this logic actually a pretty gnarly kind of scheduling problem to just to determine what that index is so clearly what's happened is even though i have a lot of this stuff right seemingly i'm missing some logic some kind of corner case here okay so i think what we want to do is now try to replicate this for a unit test and then from there we can figure out what is causing that index to be off why is it being skipped because what's really happening is some kind of off by one error it seems like if it's showing a the third task why is it skipping the second task some kind of logic is factoring in too much into the calculation and it's probably in the end of this it's probably like a scenario where i'm missing any quality check or something like a greater than or equal to kind of thing where i'm right at the boundary condition and i don't have it quite right that's my guess but we won't know until we're able to see it so this view and this schedule we've got to find out i remind myself where that is i think it's in us it's a model method if i recall but once we find that model method that calculates the schedule then we can go add the appropriate test name property test file so i believe this code exists with the student model and yeah this is unfortunate like i'm beginning over time to rethink how i want to put all this together because it's a weird kind of a weird question to ask on this model of saying hey student get me your week schedule given a school year like why is a student expected to know that that's where i put it initially so i probably did it for various reasons but that's where it lives right now so this is the the method that we have to work with it's this particular get week schedule method and it's a pretty involved method so you can see here's the get week schedule this is just kind of the tip of the iceberg we have to get the active courses there's another method involved there's a pretty big loop here there's a thing to build the tasks for the actual context in the template there's some extra logic to loop over the dates it is a large method and again this is if anything like this is the core business value of the application here um it's kind of the beating heart of the app so i got to be careful with this not to screw up the scheduling but that's why we're writing unit tests first so let's go over to the students test models file and look for the get week schedule where those tests exist they start here so actually let's go in the opposite so i'm going now go to the bottom of the file with the bottom of these tests so this is the last test that's calling this directly and it follows my my similar pattern and i know so i know that this is the last test because i can visually see that there is a space on either side of this line which tells me that this is the action that i'm asserting on so this is definitely the thing that i am trying to test i arrange my tests in the arrange act assert pattern the aaa pattern that's arrange act assert for doing test design and here we are so we need to i think what i'll do is i'll say i'll start from this test and i'm going to paste it because just because it's probably it's got a lot of configuration on it i'm guessing a lot of it is stuff that i'll have to do again anyway so we want what do we want we want to [Music] get let's see so the the thing that we're trying to test is the next week um starts with the correct task when the current week has breaks i don't that's kind of lame i don't really like that doc string let's see let's say get weak with breaks because i'm trying to tell a story here that's that's the objective of this test because future me is going to be like what the heck is all there's a bunch of setup that has to go into this and we haven't gone through this to make sure this is actually even necessary yet but um it may be and if so like i'm looking at this previous test and this is an example of a bad test of mine like i don't really understand what i was trying to test here because i didn't leave myself any kind of clue this is something i learned earlier in my project and i really taken to heart the value of writing a good test documentation string so i would recommend that if that's if you're writing your own apps because even future you could be like what are you talking about um so i don't know maybe this is the most accurate accurate statement for now let's just shorten it so we can get out of the too long string problem all right now it's time to modify this test to reflect the data setup that we just did and i think what i want to do is freeze the time look for an example of freeze time and i want to freeze it to um to today like i want to set up basically this exact scenario and i wanted to stay like that as a proof that i've i've captured this this forever um so that you know even if my test suite runs on a sunday and changes then it doesn't it's not going to matter because i've captured the proper conditions so we'll say freeze time and we're going to freeze it to today which is at the time of this recording is 2021 it's july it's the 21st so we'll say this is a wednesday all right so now we also need to import freeze gun so from freeze gun import freeze time okay so now this call to timezone.now.date will be today um and a lot of times i i do like to write my tests so that they are adaptive to the date so that doesn't really matter and i just make everything relative to the date but the dates going to matter in this particular circumstance because we want to specifically cross over to the next weak boundary and it's just going to be easier if we can just freeze everything into place okay so we need what do we need here in this example i have created a school here that starts 30 days ago i don't know that that's even strictly needed anymore let me go see let me let me check on something here so let's go to the student factories student factories and so uses a grade level factory which is from schools okay schools factories which has a school year factory and the school year factory is already picking a start date 38 days in the past and a bunch of days in the future so i think the advantage of freezing the time is that i don't i don't have to do this kind of silliness to say um here's the time range where i want you to start i think this is an old test and that's why that was there so i'm going to try taking this out for now we might find that we need to put it back in later but we'll see so we've got um an enrollment enrollment is the thing that ties the student together to a grade level which is ultimately tied to the school year and you can see it looks like i've extracted that out and this is creating another grade level for reasons that i so i don't think we're gonna need another grade level so let's let's take that out and the reason that they had that is well i don't i don't exactly know they me past me um and there's also i don't think that this is necessary either so this is definitely an old test because i can write grade levels is equal to enrollment grade level to wire the course to the grade level so we do need a course for this and we can let's take out the school year for now i don't think we need that and this is where so what else do we need we need let's let's just write it down because i'm going to be trying to keep my brain straight between data that's needed and then the factories that i need to use so we're going to need a task completed today so that'll be a course work with completion date of today we're going to need a break for tomorrow we're going to need a break for friday both of those things by click completing to the end of the week should be enough to trigger the condition and what we actually want to do is we when we want to get the course work for a week here we don't want to use this week we want to use the week that is next week so to do get next week i'm sorry that i i still need to change my comment color there it's like blue on black it looks pretty rough um so that might be a little hard to read apologies if it is so getting down in here we have a coursework we want um i think we only need a course work which again is the tracking for whether a task is done or not so we need to map it to the student it needs to have that connection and it needs to be connected to the course and we can say that the completed date for this particular coursework should be today so we'll do that and then i don't think we'll need this other course work but we are going to need um my list i left off one we need a task for next week okay i don't think that we're actually going to compare anything with this coursework so it just has to exist there just needs to be an entry and factory boy is going to be creating these in order so this will be the sequence so this will be the completed thing that will go on today and the next thing that we make will be a course task will be for next week um well i guess we actually probably are initially going to need to see two of them because in our example data looking ahead at next week it was skipping one right it's jumping to the the third one so in order to be able to see that we're probably going to need to [Music] hold on to a third task otherwise it might it might actually show us the right thing i don't know that might be another boundary case so we have the task that's completed for the day that's done and then we're going to have a course task factory call which will take um the course is going to be the current course and there's no course work there's no connection to a student yet it's just the fact that this task exists on that course that would be enough to put on the schedule we have two of those the second one is the one that we're going to see so this will be the wrong task all right all right we're getting pretty close here um we need this to be uh next week and the week model is coming from the core schedules file so let's go over there i can't remember if i made it okay so we can give it a day and it will just it'll calculate the week so i set all the week to be like fixed here so that um oh that got crazy uh so that it will always pick the sunday prior to the saturday so sunday was marked as the first day of the week i think and then the saturday was the last pretty sure that was what my customers requested so so when we go down to our test what we can do is pass in a time delta that just in order to get it in the right range we can say date time time delta of days days right yep days equal seven and what that'll do is the the weak constructor will set the first day and last day appropriately but it doesn't really matter so it's just boxing it into the right slot and by putting it ahead seven days i know that we're going to be in next week for sure so let's drop that to do we've got next week we want to change this to next week and now we would let's just for now let's just assert on what the week coursework is we'll start it on an empty dictionary to see i also want to print out the wrong task what i'm hoping to see in the coursework is the wrong task listed on the monday oh i didn't expect it to pass that's not good all right so something happened oh we haven't put in the school breaks yet so both of these course tasks fit on thursday and friday there is no uh task yet or um for next week because they all fit in this current week so let's go to the course models and uh where's the break oh uh i call it school break i think i did okay so we need a couple of school breaks to finish this off so we've got the task for next week and then we need the breaks so um course at school break factory and that's going to take uh i don't remember it's been a while since i worked with some of these models my apologies for not being so quick on the uptake so we're going to go to the school break and it's going to take a start date and an end date okay so the school break factory is actually we can give it one i don't think we even need multiple we can say that the start date is equal to tomorrow so that's going to be today plus time delta sorry date time time delta days equal 1. so that's tomorrow and [Music] then we can say that the end date is going to be equal to [Music] today plus date time time delta days equal two so that should be thursday and friday hopefully everybody follows that if you don't let me know in the comments on the chat and i'll try and get you up to speed on whatever you might be missing so we need to import this factory because we're not using it in this models um so from schools we need the school break factory which i will believe will go here alphabetically and now with any luck now it shouldn't be an empty dictionary anymore oh come on all right let's see what they're putting in the current week let's drop this let's take the current week why is this not what we want so this is showing trying to trim parse like what this is this hash id is the course id i think we have this comment kind of helping us out here there is the date the week first date oh interesting that's that's not right pretty sure first date is to 21. that would be today that does seem right and there is a an object on here so where did our other tasks our other tasks go did i hmm that seems wrong it's got the right course association let's look at the model a second so there's a course here there doesn't need to be a grade level because it's not specific to a grade which only cares about the fact that it's a to a course don't worry about it that's a different kind of feature so we have that proper thing there let's check [Music] let's do homeschool courses models import the course task there should be three of these so let's assert that the course task objects count is equal to three okay so there are three that's good where are they why is it not showing up um that's bizarre so if it's not showing on this week shouldn't it be showing on next week one would think let's try switching it to um let's make sure we can well when we put in next week it was just an empty dictionary right that's why the test passed because we started against an empty dictionary and it said there's nothing so where are these tasks going that's a really good question because this doesn't seem right because it seems to me that there are three tasks and two of them are missing the case of the missing tasks where did you go um what if i take out school break for a minute so no school break and do it on this week hopefully if i understand my own code correctly there will be three days and three tasks to do all right let's maybe i'll test nearest we need to see more i think excuse me the whole structure now what is going on that's not cool i wonder if we've frozen time and i wonder freezing time has something to do with this problem let's test this theory so we just happened to write the freeze time to be the same date as today so we can comment that out without the freeze time now we're doing it against real time and is it going to affect the data well i can tell it did already because it's in utc so it's already looking ahead so i'm doing this late in the evening so it's calculating it for the 22nd still that doesn't show everything like where are those other tasks something is amiss what did i do wrong with these course tasks oh my word i see it and it's embarrassing look what we're calling we're calling get the weak coursework not the schedule so of course it's not showing the tasks of course it's only showing the coursework because we only made one coursework good grief well at least we figured it out so what is the real method that we want to call in this test it's not the get week course work let's go back to the students models and let me wipe the egg off my face um it's get week schedule is what we really want to call i feel pretty dumb we may all make mistakes right so i still want to freeze this thing so i think i don't think that the freeze time was really the source of the problem i think it was just me being daft at 9 40 in the evening um so what we really want to call is the get week schedule where is that first called as a test on line 102 it wrapped on the bottom all right so those tests are actually up there so let's go down to freeze time let's take all this code and delete it i still think it's the right code this we've still written the right test even though we called the wrong method we just but we're close so let's go find the get week schedule um schedule and we'll plop the test down right here that's the last one okay so now we're in the right ballpark unbelievable what does this method actually take let's refactor this let's clean this thing up um to call the right thing so we don't want the week coursework that's dumb we want the week schedule and we don't want to call get week coursework we want to call schedule which takes a school year the current date today and a week to get the week of the schedule for so in order to get next week we want to pass in the um we got to get the school year in there so that's we get that via the grade level which we get through the enrollment and then we want to pass in today and we want to pass in next week or you know what let's start with this week let's see what it looks like this week and and assure ourselves that we actually have the right data this time now i would expect to see a single course with all of the tasks for the week with one of them marked as completed okay much more reasonable um oh i appreciate that did you birth the mechs that's that's a curious handle name you've got there i'm glad my stuff is able to help you out and you can see clearly even even someone who does this a lot can make silly mistakes all right what do we have we have um the way this is laid out is we've got the week that is that we're looking over and it has the days listed it's a list of objects um and you can see monday and tuesday have nothing on them because we've already passed them at this point in time that's when we passed in today what is today it always wants a frame of reference for its time calculation so those are ignored essentially and then we get to the the wednesday this is the the the day that has a completed task um and then we get to the thursday has a task on the schedule and the friday has a task on the schedule okay that makes sense let's add the school break back in and then we're gonna come working up to where we need to be so now we've got the school break in we'll run the same exact test and nope i need to run it in the same way so we see the verbose form and now what do we see we see monday and tuesday still skipped that's good wednesday has the coursework that is also good um thursday hmm what why are they saying no school break that's not right school break factory you know we forgot the school um right yeah you need a school so i just picked so here's what happened by doing this i did make a break but i made it for some other random school because there was no school provided so it slipped that in there all right so let's um since we're now using school year and i will need to use in a couple spots let's go ahead and pull it out of the enrollment like it was earlier i probably should have just kept it [Music] so we've got a school year and this will let us to allow us to trim all this stuff off and that will clean up that line reduce those three lines down to one which is good and then we can add the extra school year line here and say it's equal to the school year so this is good why i'm stepping up to this because i didn't have my test set up correct and again i need to see with the verbose format okay cool so monday and tuesday have no activity and no school breaks that's that's what we want to see wednesday has a completed coursework which is also what we want to see the thursday here has a school break and the friday has a school break and then just for future reference as we get a little farther along um here's these the description text of the course task and we're going to need that in a second because when it when the diff is going to print the course task it's only going to identify it by the description which is why i'm printing out this wrong task thing i don't and so that um that's gonna go we'll see that in just a second so let's switch it to it's still gonna be today still from the frame of reference of today but now we want to look ahead to next week and again i'm called i'm so used to i so here's what's happening when i keep messing up my test invocation i'm literally hitting space bar followed by the t character so two keystrokes which is great on most the time when you want test to be really fast but by default pi test doesn't give all the verbose context so my keyboard shortcut does not give me a verbose test output so that's why i need to do an ex command in my editor here so if you look if you're you may not be familiar with vim but vim has an um basically a command mode that you can kind of switch into ex mode and so you can execute commands and that's what i'm pointing at trying to highlight here in the bottom left corner so i'm saying my test plugin is saying test the nearest thing and pass at these command line options in this case it's dash vv which is a pi test way of saying very verbose output which was hinted at in the pi test output from the previous command run where it shows hey you're not we're not showing this all of you all of this here for you because it could be too much but if you want to put it in verbose mode so my way to do that from my editor is to call it with this test nearest command so that's that's what's going on and why i keep i keep getting frustrated myself because i keep getting jump on the gun and just doing my normal shortcut so running it with the full output mode which you can see this is a lot more stuff now we are now looking ahead to next week and here we can finally we finally have proven that this test is not what it should be so it says maybe recognize middle citformer this is just junk test text that gets dumped into this task creation for the description and here's that wrong task that we printed out all right so what do we need to assert on what is what is going to make this test pass what we need to assert on is that we can go into the weak schedule we can go into the courses we can go into the list and go into that first that's so that's going to be a dictionary it's got days as some keys that's another list so there's there's quite a structure we have to navigate through and eventually we're going to want to say that the task in there is the right task um hey quatlum welcome back good to see you around um thank you good to be back from um vacation wet rested so um we want to assert what did i just say we want to assert on so the thing it said it is a well i'm already blanking out on the structure it's hard to keep that kind of structure in your head um all right so it's a dictionary first is keys of courses we've got the key of courses and we want the first thing okay now let's go ahead and run our test again so we're kind of we're digging into that structure and we're getting closer so there's you don't see the courses key anymore you don't see that list anymore we're getting down to it so now we want to go into days and we want to index into the first day that monday there so next is days and then again into that first list that's going to be the monday and then um i believe the last thing is task but let's just see it one more time just to make sure we're really zoning in on it what we've got and so here's that one item this is just monday and it says operation us short alone is the task that's not the task we want so what we want to do is we want to say task is the piece of data that we want to compare against and now we want to see that that is equal to task this i think is finally the test so we've got actual stuff coming back is i'm still printing it out is this final white country summer law view these are fun to say and the thing that we want is provide left hit him um that's not very nice is it it's funny what random strings can pull together um okay final cleanup we don't care about this current week that's not really what we're after and we don't care we know that there are three tasks that's we've proven that and we don't even need the printout anymore and i guess we don't need to even have well let's leave the wrong task until we fix the logic and then once we've proven that it's no longer the wrong task that it's just task um we can delete this and actually let me put a to-do in there so i remember remove wrong task so before we commit that code we should hopefully get that right okay so that seemed like i mean i just spent the last i don't know 30 minutes 40 minutes digging into this and and some of that time i've recognized that as as as being a streamer and trying to communicate you to you my thought process i'm definitely not going as fast as i could be if i was just doing this on my own i hope that's clear i mean i could still make the same kind of goofy mistakes that i made getting there um but i i have noticed that my stream time there's a factor reduction so even if you factored it down and let's say you're a competent developer and you did that in 10 15 minutes why why would you want to do this um i think in my opinion like having reproduction steps makes it much easier to solve your future problems by having a very clear case an example that we can work from now what we can do if we need to is is we're going to inspect the code here in a minute and see the scheduling logic and see where this breaks down and where that calculation is happening but it also gives us the power to step through and like really dig into what was the context at the time of of the issue so i think it's it's useful for like the actual problem solving aspect of this but it is also useful um as a regression test like if i ever change the logic in the future like there's reasons that i i think some of you may have not have joined the stream yet but i've shown the code that calculates this and it's not simple code um it's it's complex code does a lot of things because it's factoring in a bunch of different weird edge cases and by having these tests and documenting them well it proves what those edge cases are so that if future me is ever like well i can simplify this then i run the test suite and prove to myself either i actually did simplify it or i forgot about some certain kind of edge case so that's why i do this kind of stuff um i'm pretty confident in this test now and i think we can go look at um the actual code that is calculating this and drill into it and see what's going on my theory again coming back to like and what is even the problem so a bunch of you and i've noticed that the number is uptick here so let me revisit the problem one last time um in case that you weren't clear from the unit test we've got this scenario where there's a couple of break days here and there is a completed task here on the wednesday and looking ahead to the next week um these are numbered lesson one so there's lesson one two and three um it jumps one it's an off by one error most likely and so it's going to lessen three when it really should be going to lesson two and that's um that's no good that's that's wrong so um we have to figure out what is it about that next week calculation that's getting that off by one error that's that's the core of what we're trying to solve here tonight okay so let's go over to the model stuff that was going to this i'm going to try and see if i can even put some break points in there and get right to where i think the problem is going to be and then we'll take a look at the calculation the calculation is based off an index into a database table and then counting the number of tasks in order that are need to appear there and i think that that calculation is off somewhere so let's open up the students models file again look at the right method this time and we're going to skip through a lot of this because i know just from my own personal knowledge where this starts to matter and i believe at least i think i know oh i'm pretty sure all right i'm lying i'm pretending to know it's been a long time since i look at this code ah here it is okay so we've got this private method that's building the course tasks that go into this schedule and it has to do all this calculation it's based off of it it's getting a bunch of information all at once and some of this is denormalized for performance reasons so that's why it's getting all these parameters which is why this is not straightforward and when you get down to it the critical element here is this task index calculation so if we were to um let's say let's print just let's just print the task index so we can see what it is because this is almost certainly going to be the code that is executing and so let's try running this test again and we should see a number come out the number is one so the number on the face value that that seems like you is that should be the right number well clearly it shouldn't because it's wrong so talking out loud here um when it's calculating that index it's trying to it is i believe trying to factor in what tasks have already happened so this this real next index number really should be it should be zero right because there's there's only one task that's been done and that's going to be filtered out when with this exclude right here you can see it right in this call so it gets the task for this court course and then excludes all of the completed ids so this is a query set and that that query sets then gets excluded and you can see how this logic happens it then um doesn't uh a slice into the query set based on the task index so the answer for what should monday be should not be a one it should be a zero so the question is why why is this coming back as one and we can get in here and look at this course task index stuff and i have this is probably my third or fourth round of doing battle with this method this is a hard method to get right for some reason um so my last attempt was not correct i guess and well let's leave that why not we're not going to keep it um and let's you know let's switch these to be side by side it's a little tight on the code reading i know there's a resize so there's a resize um vertically i know to go up and down but i don't know if there's like a horizontal resizing bim i'm sure there is i just can't think of what it is so we'll we'll stare at this a little bit non-optimally but it's better than nothing i think you know what no it's not let's get rid of this i can still with my command at least test the file so we can we can get the the output fairly quickly okay so see what i mean this is this is where i've done battle with this method before it's not simple um all right where to start goodness gracious uh wowzers um okay so it's it has claimed it's one but the the index is based off of these two values of unfinished tasks this week so that should be that should be zero because monday and tuesday are cleared out there's nothing unfinished on wednesday and thursday and friday are a break so that should be a one so we should be able like test out each of these numbers and maybe maybe that's where it's going wrong unfinished tasks this week um we'll say unfinished let's see what it says unfinished is one there we go i'm getting right down to it and okay another thing to think about here i am doing print debugging right i'm using print i think there's no shame in that and and here's why it's actually sometimes even beneficial with some debugging sometimes you want to step through a live cycle and we may have actually get to that but sometimes you're like you don't necessarily know where the problem is and it's faster to kind of jump around and put in reasonable guesses and run it quickly and be wrong and try again um i kind of view as this like uh like shooting in the dark like i've seen this analogy being used in code and other times but think about like the the way modern militaries with machine guns have learned to operate at night what they do is they have uh tracer bullets um which are uh bullets that glow they're phosphoric or whatever and so you see a little bit of light and so as they're firing they just kind of see oh i'm way off target so they'll like move over rapidly right and so i'm viewing these print statements as kind of like tracer bullets i'm just going to rapid fire running the test over again that's the equivalent to my machine gun here and seeing how far my tracer bullet is off versus spending a lot of time calculating and trying to figure out exactly where i am so and just by doing this this minimal analysis we've pretty quickly gotten it down to the index is off by one and it's coming from this unfinished count and why is that unfinished count off as the thing that we don't know yet so i think don't let anybody shame you into thinking that print debugging is a bad thing i wouldn't use it all the time i do still reach for regular traditional debug step debugging sometimes but there's tons of value into rapidly iterating on on your thinking on stuff the the other flow so this is definitely the scenario that we're going to be in is it's not going to be setting this to zero so we're actually getting this unfinished count the tasks this week by getting the task count in the range so i think i'm starting to see the problem already um this well maybe i'm not maybe this is factoring in break days and i'm just not thinking about that yet or we need to get dig into that method that very well could be so i you don't know you'll all know my project's domain but calculating the range of dates when a a student can be doing activities is again kind of challenging because it has like there could be the weekend there could be holidays in between there could be days that the course is scheduled to run for so if you make a course that only runs on monday tuesday wednesday and friday you have to make sure you skip thursday so all of so you need the the course itself you need to have a bunch of pieces in here and so we're going to have to dig in i think the next place that we're going to is getting into this get task count in range viewing the data that's coming into this test um whatever's today what is the last school day and then figuring out why did this come back as one did that make sense like maybe it does make sense and then the calculation logic for this helper method is actually off um that's that's a possibility or or the other option is that this get task count and range function is not counting properly i think that's less likely based on its scope and function i think it's more likely that this code right here is just not figuring everything out properly but until we dig into that we won't know so let's go over to the get task and range function let's send the schools models get task count in range and you can see this is why i think it's like it's unlikely for this thing to be incorrect because this is much much shorter than the others it's pretty tight and yeah i just think there's less room for this to be wrong so print we'll say this is the start and we'll do the start date and we'll do the same thing for the end get the end date so we're going to see those two things and then we will i mean we know that based off of what we've seen this is this return gets assigned to that unfinished number this task count is getting incremented somewhere um so let's let's run this first let's see what dates we're dealing with so there are two times this is being called oh that's i think this is the one we actually care about though unfinished of one yeah all right so there are two dates that it's considering let's see when is it actually um is date to check here when is it actually getting bumped is what i want to know hmm oh interesting so it's counting okay maybe this makes sense it's counting the current day the day that we had a completed task as um a the task count in this range which made sense right because we passed in the start date was today the 21st so i think this code is doing what it says it's supposed to do get the task count for a course and factor in any breaks be inclusive of the start and the end so if you passed in the start date it's doing what it says on the tin you told you asked me how many tasks should be done in this range today counts so i think what that means is that my other code is doing this wrong it's it's not factoring in the the coursework that was finished today too and so that's why it's getting that number off and if it subtracted out that coursework that was completed today then the resulting index number would actually be 0 and not 1. hopefully some of that made sense i know that's a lot of data um and i'm i'm still like wrapping around my brain to see if if i'm even convinced of this but i i'm feeling more and more comfortable that this method is right so that gets us back to over here and unfinished tasks this week during the active part of the week tasks roll forward each day this means the unfinished work is everything from today to the end of the last school day that makes sense but where it fails is like the task today was done right so it's not doing and do we have that course or shoot man so what goes in here we have the course but we don't have the task ids and this is being this calculation down here is being naive by including today today may or may not be done and i think that's the root of the problem like okay let's let's see if we went back here this is going to be wrong either way i don't know that's a good question i don't think so i think it will be right um so let's go into this day and let's mark this as actually incomplete so now we're back to this is still to be done and if we go ahead then it's it's correct here because it literally it's still so that would be an unfinished task so that means when we do the query and we're filtering out the the exclude the completed coursework that that task won't be filtered in so by indexing to one you get to the proper second task so i can see how that logic worked if it wasn't complete okay so my thesis then is that when when we calculate this unfinished tasks the to do here is to subtract out the actually completed tasks from this number so this is like the hypothetically what could be completed what could be unfinished but we need to [Music] get the the real unfinished which may mean that asking this count using this count range method might not be the best right way to do this we might need to look at the actual tasks that were for that schedule for that week and compare them to the list and do like a real count based off of the actual week available it's probably smarter in fact i'm gonna have to think through that okay i where am i thinking like do i want to how far do i want to take this let's think on this a little bit more and i'll give it another five minutes because this has been a tricky problem my brain is kind of getting fried and if i haven't solved it by then then we can i'll work it offline and we'll you'll at least have seen the the process like this was not a this was not at any point hopefully you'll agree with me a a real guessing game like we had to we had i guess i suppose we had to guess a little bit to kind of get to the code that was in question but the process was guided by a test pretty much the whole time and making real educated decisions based on observed data points so we try to be fairly scientific about this in in getting to where at least where the problem is doesn't mean we have the solution yet or i know roughly what the solution is but the best way to implement the solution is kind of the next phase of this this thought process um and how i want to best approach this i'm just looking at this code right here this logic make sure i understand why this this is happening this was a special case logic for accounting for the weekend and there was so my spouse has already encountered some problem with the system already where she would look at the sk the future schedule on saturday or sunday and stuff that was done not done from the previous week but she wasn't going to go back and do it because school was over for the week was being counted and affecting the future week and so that's what that edge case is for so when today is less than or equal to the last day of the week means is a translation for this is really the the week is still ongoing in fact maybe that's a this is a good thing for me to like i don't want to have to think about this every time i look at this code and as i've noted i have to come into this code often so um let's let's give this variable let's give it a name is current week active and that is equal to this today less than or equal to the last school day this week and that seems like a kind of a stupid thing to say but if i'm haven't if i'm here having to explain it and having to walk through that cycle again and stare at this code it's telling me that my code is not being descriptive enough so now when i can look at this code yes there's a conditional here and there's a comparison but really what i can just see is like if is current week active that i think reads clearer to me um then than trying to read this conditional yeah that's a decent decent enough name so this calculation i i have faith that this calculation itself is correct because i put so much thought into it so we still need to get the unfinished tasks for this week that that name that name is the right thing but how it is being calculated is the wrong thing and what does that mean is is the unfortunate part we don't actually know what the tasks are until after this point because look we're we're calculating the index here we're like trying to predict the future and calculating the index is going to give me a slice of tasks for this particular course and only at that point could we know is this done or not but the the index calculation is dependent on knowing what's done or not so it's kind of a pickle and i guess i know that some of you are not necessarily native english speakers so a pickle is like a problem quandary sorry some english phrase so is there a way that we can solve this this is okay i think it has to be solvable because if we think about this is asking about doing an index into a future week right so that implies to me that the data for the current week may already be available like this get course task index function um build course tasks this called all the time it is um shoot all right there goes that theory i was thinking it might have been a thing where i only call that task index calculation for future courses in which case i could have kind of punted on this problem but i don't think that's the case all right so well it is yeah so this chunk of code only runs on future stuff because there isn't this um guard clause when this is the current week or the past we don't look we don't try and do a future calculation so i could take advantage of that knowledge that basically everything below line 223 here is considered future index calculation um probably figure that out all right sadly i don't think this is going to come up i don't think i'm going to be able to devise a quick answer here so i think what i'm going to do is let my brain have a rest call it a night and leave you all with this like cliffhanger how is this going to resolve um i will have a good think on this and try and figure out the solution and either come back next week with what did i devise as a solution or we can work through it again where fresher mind fresh perspective um yeah because i don't think i'm gonna have too much more success so anyway i want to thank you for joining in on the stream tonight hopefully you saw my bug resolving process and how i kind of systematically debug bugs and that was illuminating to you thank you for joining in and have a great evening take care

Channel: Matt Layman

Views: 209

Rating: 5 out of 5

Keywords: twitch, python, django, saas, bugs

Id: fmXW4T6YZkE

Length: 79min 30sec (4770 seconds)

Published: Thu Jul 22 2021