Planning a Network Caching Strategy (VLOG)

welcome back to another one my vlogs in this vlog I'm going to talk about caching strategy what it is and how to plan a caching strategy for your applications so as most of you know I'm building a course currently on clean architecture and dynamic feature modules and I've decided to add a network layer to the application because hey let's let's face it all applications pretty much have a network layer it's very rare to find a real application that doesn't access the network and the cache so first of all real quick what exactly is a caching strategy a caching strategy refers to the plan that you have in place as a developer of how the data is gonna come from the network gets stored locally on the device and gets served to the user likewise how the how the data is going to get stored on the device or created on the device and then sent out to the network so that it can be accessed from other devices let's quickly take a look at the app that I'm building for my next course in this next project that I've been working on and I'm gonna outline kind of what the requirements are for this caching strategy because I think that's a really good place to start you really need to understand what your app is going to do what the purpose is why the data would get sent to the network why it would get sent to the cache before you start planning out your strategy so as most you know it's a simple notetaking application I can create a new notes just you know some random title here click OK that will then insert the note into the cache and it will also insert the note into the network into firestore so I can obviously update that note like if I clicked on it I can make some changes to it click the check mark that would also update it in the cache and updated on firestore so that's kind of the first requirement the first requirement is inserts so I have kind of a list that I'll keep track of the things that we need to do number one is inserts when a new piece of data is inserted into the cache that also needs to get sent to the network to firestore and number two is updates so if a note is updated in the cache it needs to also get sent to firestore to reflect that update so once again like I just showed you a few seconds ago if I was to click on one of these notes and just like make some kind of a random change to it click the check mark that should get applied not only to the cache the local cache but also send out the firestore on the network now we come to the third point the third point is going to be deletes so this one gets a little tricky this is def where the strategy requires a lot more thinking so suppose I wanted to delete a note now one of the ways that I can delete it out and this application is simply by swiping this out so when I delete that note I get a little notification here this is pending and I have the ability to undo it if that transaction is complete I want to not only delete it from the cache I want to delete it from firestore from the network so why does it get tricky when it comes to deletes well number one is I have the ability to undo it so like I said if I was to swipe this out and I was to click undo that has to get re inserted into the local database and also reinsert it into fire store so that's kind of the tricky thing number one the number one tricky thing I guess you would say so what else makes delete operations tricky I'm going to bring another device on the screen here to illustrate what I'm talking about another reason why the elites are tricky so suppose we have a second phone and this is true for most applications you should be able to open an app on any phone or even open an app in some cases on a web browser apps like Trello Instagram Facebook Twitter all of these apps if you were to open it in the web browser and make a change submit something to the to the network no matter what app you open that on and whatever device the data is going to be the same so you want the behavior to be the same in this application so over here on the Left I have the app running on my real device if I was to click on a note say just this first one up here notice that the content says neato mosquito I'm going to add some exclamation marks to this and I'm going to click Save and then go back so I expect that when the app launches on the emulator I should see the exact same content let's go ahead and launch that on the other device I'm going to click the app and it's going to load up notice it shows a splash screen for a couple seconds and then it actually takes you to the list of notes so if I was to click on this we expect it to say the same thing which it does so we know that that's working correctly so what I did here was actually when the app starts up so let me just close it and reopen it when the app starts up it actually does a data syncing process so that's what's happening when the splash screen is in view as soon as the app starts it looks into firestore and compares it with the notes that are in the local cache and what does it do well it does a couple things number one is it looks in firestore and it finds any notes that have a date newer than the ones that are in the cache so it searches all the notes in the cache it says if the day is older than what it finds in the network then it updates it from the network and likewise if it finds a note that is newer in the cache it then updates the network so you get kind of the best of both worlds there the tricky thing here is deletes so let's take a look at the demo here again so what happens if I delete a note on my real device does that get affected or is that reflected in the other application on the emulator so I'll launch the app again on the emulator and we expect to see that the note was deleted so we should not see it in the list and there we go the note is definitely not there so coming back to our list that I was kind of keeping track of the operations with we have inserts updates deletes and then the fourth point here is data syncing I know I kind of skipped over it and I gave you the demo before giving you the point but I wanted to make sure I put it up on the screen here so inserts are pretty straightforward you just literally if it inserts into the cache you insert it into the network also updates are exactly the same you update it you update the network deletes are a little trickier but generally if you delete it from the cache you then delete it from the network I'm going to talk more about deletes in just a second here and then there's data syncing so the reason you have to have data syncing is you have to if you open it on another device or a web browser or something you need to make sure that the data in that cache is the same as the data in the network and therefore is the same as the data on all the other devices now let's briefly come back to deletes because there's a couple more fine points that I want to point out about it so how does this work you know how if a note is deleted from one device how does the other device know that it was deleted there's a couple ways you could go about this the two kind of major ways that come to mind probably for you right away is number one just you know query every single note that exists in firestore and if the cache has one that the network doesn't you should delete it from the cache there's a couple holes in that logic so let me just point those out number one of course is performance if you're constantly if every time the app launches you're querying every single note from firestore every single note from the cache as you get more and more data that's going to get exponentially slower so if you ever can help you if you can ever help it and not have to query like every piece of data that's generally what you want to do you want to you want to optimize that if you can the second sort of big flaw with that type of logic is what happens if you delete it a note but somewhere down the future you wanted to like you were like aw crap I shouldn't have deleted that it had like some really important information there's no way to look up that like history I guess you could say and you could consider like Gmail or any kind of email provider as an example suppose you delete an email that deleted email usually goes into some kind of trashcan or archive for at least you know a few months that way if you have that oh crap moment you can go back and look through the archives and find that deleted content and restore it so if you're keeping only one list obviously that's not possible once it's deleted it's gone forever so the second way that you can do this that I see so we just looked at number one which was keep everything in one list in the network the second way that I see which I think is a better way is to keep a secondary list called in like some kind of a deleted section so if you take a look at firestore let me just show you my structure so the structure goes like this there's a notes collection and there is a deletes collection then inside of that is a user ID so if you had a lot of users imagine this would be like a ton of users and then they have all of the notes here that they have inside of their notes collection likewise deletes is exactly the same you have deletes you would have the user all of the other users in the app or that are using the app would be right here then they have a notes collection and this is all the deleted notes just to give you kind of more of a visual let me pull up a diagram so here's what the notes node would look like in firestore you have notes then organized by user ID and then notes and then all of their notes in a list likewise you would have the deletes note so this is literally pretty much exactly the same other than there is a deletes collection at the head of the the head of the whole hierarchy instead of a notes collection so basically we're saving two lists we're saving a list of notes and we're saving a list of deleted notes so I'm gonna bring the app on the screen here I'm also gonna fire store in the background so you can see exactly what happens in the database when a delete happens so over here I have the phone I'm just gonna scroll down and delete kind of a random note but I'm gonna make sure I have the deletes collection selected so I mean deletes I have the user ID and then there is the collection of all the deleted notes all the notes that have ever been deleted so watch what happens when I delete one of these I'm just gonna swipe this out we're gonna watch fire store notice that in note was added to the deletes collection likewise if I was looking at the notes collection I would see a note was deleted so a note is removed from notes collection and a note is added to the deleted collection now if I was to launch that app on another device which I'll do right now let me just launch it on my real device it's now doing the data syncing process and now notice that that note is no longer in the app so what happens here is it launches and it looks inside of the deleted notes collection if there are any notes in that deleted notes collection that exists in the local cache and the database cache those are deleted and like I said I think this is a good way to do it because it's optimizing for speed you know you don't have to look through the entire network to find which notes need to be deleted you can just query that deleted note see if any of those are in the cache if they are delete them it's definitely a faster operation and you have the ability to go back if you wanted to add some kind of feature in your app later on to look at like archived notes or archived content you could add that very easily and just allow them to browse stuff that they've deleted in the past you could also you know on top of that if you didn't want to keep a bunch of stuff on your server for too long you could write like a cloud function or if you were using an actual server you could write some kind of a function that would run in the background that maybe every three months or something it would go through deleted notes and then just delete them just gonna clean them up so one last thing before we go I'm sure some of you are wondering why I'm doing the syncing only when the app starts so for example if I click the app it starts it goes to a splash screen it does that data syncing process and then it goes into the note list so why am i doing it only when the app starts why aren't I kind of listening in real time and to be honest the only reason I'm not listening in real time is because it's just more work in a real application if you were launching you would probably have like a website you would have an app maybe it runs on tablets all of these kind of things so you definitely would want it to be listening in real time because if you had it open on your tablet you had to open your phone you change something on the tablet you would want it to show on the phone immediately likewise if you had some kind of a web component if you changed it on the web browser you would want to see that update in the app immediately this app is it's a learning project so I didn't implement this functionality I didn't really think that you need because we don't have a web component theoretically the user using this should only ever be using it on one device maybe two if and if there's two they're almost never gonna have you know one phone open changing a note and then another phone open changing a note that's just not going to happen so I think in this scenario we can get away with it but definitely if you were building a real project you would have a website you would have other devices that it's running on you would definitely want to be listening in real time all right so to kind of leave you with some parting words or some parting advice when you're as you're trying to plan this network caching strategy you always I guess you want to look at the edge cases you want to try and think of well first the easy stuff you want to think of okay what happens when I insert obviously you're inserting what happens when I update obviously you want to update the leading is deleting so those things are pretty straightforward you know insert update delete whatever querying data obviously query the data but when it comes to the edge cases we're talking about multiple devices or like a web component so what you want to handle the sinking of the data how are you gonna handle the sinking of the data if something is inserted how is that going to reflect the data on another device if something is updated same thing other device if it's deleted what does the other device do you have to think of the other device how does the other device always reflect the correct data state or data that's gonna be it for this video remember that the project that I showed you in this video is going to be part of a course on my website on clean architecture and dynamic feature modules where I'm going to build this notetaking application where you can create notes update notes delete them and also yeah query them I think I said query already and all of this is going to be syncing to a cache and then also syncing to the network on firestar we're gonna start with clean architecture and then we're gonna write tests we're gonna make sure that everything is working correctly I want to show you why cleaner architecture is so powerful why everybody loves clean architecture and really comes down to testing and organization those are the two main main reasons and then we're going to move into dynamic feature modules which is like further abstraction further levels of organization which I will talk more about that in other videos because this is already too long now before you go do not forget to like the video do not forget to tell YouTube that you like this video because if you don't you too will not recommend my videos to other people again thank you very much for watching and I'll see you in the next one [Music]

Channel: CodingWithMitch

Views: 8,379

Rating: 4.9764242 out of 5

Keywords: codingwithmitch, codingwithmitch vlog, programmer, android programming, android engineer, learn android app development, dynamic feature modules, room persistence library, android caching data example, android caching tutorial, how to cache data in android, how to cache data in android studio

Id: 4u7ajxGxo1k

Length: 13min 47sec (827 seconds)

Published: Mon Apr 27 2020