Pagination from Remote API & Local Cache Using Paging3, Compose and Clean Architecture

hey guys um welcome back to a new video in this video I will show you how you can build a clean architecture app with pagination and caching so on the one hand we will have a remote data source on API from where we will pull the data we will pull the data in a cache so we can access that even if our app is offline and we Implement all that in combination with pagination so that we only show let's say 20 items at once and to handle all that efficiently we will use the paging 3 library from Google and directly show you what we'll build we will build a little beer app so an app that displays information about specific beers as you can see the API for that is completely free and authentication free as well so we don't even need an API key you don't need to register for that you can directly pull the data in here we will put it in a local cache and yeah you don't see the pagination here but there is active pagination because it just loads so so fast because the response is pretty simple and you'll also notice that if I turn on airplane mode to yeah make sure I don't have internet connection and I relaunch this app and then we will still be able to access the data we won't be able to access the images unless we set up some kind of caching mechanism and you also already saw that there was an error message that we could not resolve the host because we are not connected to the internet so this is really something we now saved in a local room database and before we get started with the video a quick little reminder that there is an ongoing Easter sale active for all my premium courses till April 12th you can get 30 discount on all of my paid courses I really never had a bigger discount before and to apply it you can simply use the coupon code Easter 30 during checkout and does Discount not only accounts for the single courses but also for the already discounted course bundles so if you've been thinking to get one of these for a while then there is no better chance than now so do check them out by clicking the first link in this video's description and now let's get started with the video all right to get started let's quickly go through the initial setup so I recommend to check this video's description and clone the initial code for this product from my GitHub repository since there is quite a bit of setup for dependencies we will use on the one hand here kotlin capped and Dagger Hill plugin so we will be using dagger Hilt in form of a dependency injection framework so we will use that we will also use coil compose to remotely load these images to last cycle utility dependencies for a jetpack compose to get view models and safely collect flows and Dagger Hill dependency is nothing new the paging dependency which handles the pagination retrofit of course to be able to pull the data from the API and finally room to be able to also save the data locally so we can also show it in offline mode in terms of composing cotton version I am using 1.4.0 for compose and 1.8.10 for kotlin and right away uh while we will stick to clean arcade sexual guidelines here I won't explain every single bit about that so if you have no idea about clean architecture and your whole purpose is to learn that then I recommend to watch another video of mine for example the clean architecture note app or the stock market app for example literally I really want to focus on implementing this paging mechanism in combination with a cache into an existing clean architecture setup so the API we will use is this one this Punk API and it's yeah super easy to use as I said no API key no other form of authentication we can directly call the endpoints here and we actually only need a single endpoint which simply gives us the list of beers so let's go to Android studio and the first thing I want to do is I want to create a data package in here and then inside of that a remote package for our remote data source so in there we'll Implement our API interface with retrofit and first of all our beer dto so that is the data transfer object and this is basically the object that represents our Network model so we'll then take the Json code we get from the API and put it or parse it into this kotlin data class don't ask again add and in here let's think about the fields we want to have for specific beer on the one hand it contains an ID in form of actually an integer it contains a name it contains a tagline which is simply take a look here the tagline is this little gray text here you know you shouldn't it contains a description it contains a first brood date which is also in form of a string here and it contains an image URL which is a nullable string because I think some beers might not have an image and then we can go ahead and create our beer API inside the remote package in which we Define the different endpoints we have we need to access our API we only have one which is a get endpoint so we want to get beers that is also how the endpoint is called and then we'll have a suspend function called get beers and that will take two query parameters which we annotated with query on the one hand in the page which is simply an integer so since this API of course needs to support pagination if we want to make a pagination tutorial here then we need to pass some form of page key to that and in this case it's simply a normal integer so we want to get the first page then it would give us the first try any beers if we pass two for the page then it will give us the next 20 beers so the beers from number 21 to 40 and so on another parameter we need is the per page which I rather want to call page count which yeah does exactly that so that defines how many beers we want to get per page and the result of this call is simply a list of beer dtos I also like to have a companion object in here in where we Define our base URL so constant value base URL which is simply https API dot punkapi.com V2 slash that is our base URL and then the beers endpoint will then return our beers next up I'd like to create our domain package so domain in our package and in there we are just going to have the beard class so just our domain level class which honestly won't differ much from our dto and our entity for our database later but in a larger app where we would have more Fields um there would of course be certain differences which is why we want to have that common class for overhaul project which in this case is just called bi so first we have an ID we have a name we have a tagline we have the first brute date which is a string we have a description and we have an image URL like this and now last but not least let's Implement our database entity which again will look very similar let's actually just copy this beer dto class go to data create a local package for our local data source and in there I will simply paste this beer dto we of course want to rename this pressing shift F6 and rename this to beer entity so an entity is something that corresponds to a database Model A dto is something that corresponds to a network model that comes from Json and the model without anything is simply the domain model which we use across our project after mapping our data related models through that now I think we also need to know that has been properly renamed I just like to rename this first brute with a capital B which is more kotlin stylish like this and image URL as well we then want to annotate this with entities to tell our room library that this is a table in order database and the ID needs to be annotated with primary key and then finally we can have our mappers package with the single beer mappers file in there in which we're going to map our beer dto to be your entity to directly save that in a database so here we can return a beer entity the ID is simply the ID name is name tagline is tagline description is description first brute and of course the brood like this is first Boot and image URL is image URL and then when we read something from our local database since that will serve as the single source of Truth which means we should always just display data directly from our cache and never directly from the network this just really helps us to illuminate bugs because we know that data is only coming from a single source so we only need to look at that single Source when we're looking for a buck so then we will have a beer entity mapper to a beer model like this and we can actually just copy paste this beer like this first brute is first brood and image URL is image URL and we need to make this one nullable so let's go into our beer model make this nullable and there we go so now we can get to the exciting part and that is getting our API response putting that in the database and then properly paginating that and normally if I Implement pagination just from an API I personally prefer to just have my own solution because you can just write a simple class for that and it will do everything you need for pagination without needing a library for that but if you need to paginate from both the API and the cache then Things become really complex if you want to solve that with your own solution in this case I already prefer to use the paging 3 library from Google because that just works so well together with room and the thing so the the central control unit kind of which controls this paging logic and puts our loaded items from the remote API into our local database and then forwards just the page that we want to load and that thing is called a remote mediator and we need to create that for our beers so in our remote package we'll have a beer remote mediator will be a normal class and this will now get access to both our data sources so on the one hand to our beard database which we by the way don't have yet let's Implement our database first before we can do this so in local let's have our beard down first of all our data access object but we Define the queries we want to use to access our database this is a Dao but this will be super simple here we're just going to have an upsert function suspend function up search all where we can pass a list of beer entities beer entity and it will simply insert this all and yeah update them if they already exist we then want to have a function to get the paging source of this beer down this is specific to the paging 3 library and the paging Source will just help the library to very easily handle the paging mechanism from our local database so here we want to return such a paging source and again it's just very cool that room works with this so well that we don't need any further setup to get this paging Source because roon will just generate all the code that's needed to return this so here first of all the key we need to pass to this paging source is our page key so what kind of key we have to load the next page in our case that's just an integer since we say we have a page one page two page three and so on and the value so what we get for each page is in this case just a beer entity because each page just consists of a bunch of beer entities and the query for this would be select everything from beer entity so the cool thing here is you could also for example Implement a search mechanism and Page these results so that you could adjust this query to only return certain results that match a query which you could pass here as a parameter or you could page your beer entities in Reverse so we could sort them descending and much more so you're really flexible with this approach and last but not least you're going to have the option to clear our cache our database which we can do with the suspend function clear all and we annotate this with a query delete from beer entity so this will simply delete all beers we can only have in our local database so far for the Dao let's Implement our database before we can finally get to the page you make a mechanism that will be called beer database make it a class an abstract class this will be a room database and in here we will have an abstract valid for our Dao which is a bit out we then just need to annotate this class with database from room we need to pass the entities we have in our table which in this case is just beer entity double colon class and we want to set the version to Simply one we need to increase this version as soon as we make a change in our database and we want to implement some kind of migration strategy which we really don't need here so next we're going to go back to our beer remote mediator and here we can now pass in our beer ADB which is our beer database since we now have that that is our one data source our local one but we also need our remote data source which is our beer API so let's also pass that in here then this remote media needs to inherit from remote mediator that is a class from the paging library and this now again takes in our keys and form of integers and the values and form of beer entities call The Constructor on that and then we need to implement a function pressing Ctrl I which is called load and here we get a bunch of Errors we can hit Alt Enter to opt into this experimental dump API which you always need for Android Let's uh simply do this and then the errors will go away and now let's take a look what this load function does and what all these parameters mean here on the one hand this load function will be called whenever there's some form of loading in regards to pagination so there could be different load tabs which we have here and a low tab could for example be refresh that means we're currently loading because the whole list is going to be refreshed for example commonly implemented with a swipe to refresh Behavior another load tab would be that we're currently appending new items so when we already loaded the first 20 entries for example and we then scroll down to load the next 20 then we are effectively appending these next training entries to the previous ones and that would also take a little moment because we need to make the next API call and to just distinguish between these different types of loading we have this low tab parameter and this paging State just refers to the current kind of paging config so for for example to what kind of page size we're using what kind of page we're currently at and also to things like the items of the current page and in here we now need to return a mediator result so now we need to tell the paging Library how we load items from our API and then in which case we consider this loading a success in which case is an error and to do that this is very similar to just implementing a simple retrofit request we return a try and catch block onto catch i o exceptions on the one hand well we would like to return a mediator result that error since this would be unsuccessful of course we can pass our exception and we also would like to catch HTTP exceptions so if the API returns some kind of error code we'd also like to return a media result that error with our exception and the exciting part now happens in this try block where we always first of all need to get the corresponding load key so that is the the key for the current page basically we want to load so the key is in the end in our case just the current page so one two three and so on and since this load function gets called multiple times every time we want to load a page we need to get the current load key so the current page we want to load first of all in this load function we can get this from our load chart so when this load type we can hit Alt Enter to add the remaining branches when the load type is refresh and then we now want to return the page we want to load next and since if we refresh then we usually just want to start from the very first page again because of that we can simply return one which is the initial page for prepending so if we want to prepend items in our list that is something we don't support in our app because we only append at the end of the list so in this case we can simply directly return a mediator result dot success and here we need to pass whether the end of the pagination has been reached or not in this case we say true and append is the interesting case and we need a little bit of logic because we now need to calculate the next patreon to alert first of all to get a reference to the last item in our current list which is equal to our load state or paging State I mean last item or now and if this last item is null so if there is no last item we'll return one so we simply again say okay if there is no last item then it must be the first call to append so we just return the very first page however otherwise we need to calculate the next page we want to load and for that we can use the last item dot ID by the way only in this case because the ideas are just ascending so the first beer will have ID 1 the second beer ID 2 and so on if that is handled differently by your API you need your own kind of logic here to calculate the next page in our case that's pretty simple though since we can simply divide the ID by conf state config.page size and we then add one on top of that because if our page test is 20 for example and our last item id would be let's say 80 then we would divide 80 by 20 which is 4 so the current page is 4 and since we want to refer to the next page we add one on top of that and now if we take a look this load key is nothing else than integer so this is the next page load So Below this we can now make our API call without beers is beer API dot get beers and we pass our load key for the page and the page count is simply our state config.page size so now we should have gotten our list of beers that is coming from the API the next step is to take this list of beers and insert it into our local database into our cache and for that one to say brdb dot with transaction and here we want to check if the loads type is equal to load type dot refresh so if we are completely refreshing the list then we want to make sure that we clear our whole cache because let's say we already loaded um the values into our cash till page full and then we refresh the whole list so we just load the first page again the first 20 entries then we want to make sure that our cache is empty again to insert these new first 20 entries but it should not contain all the older entries up until page four so in this case we say beer Dao beerdb dot beer or Dao dot clear all that is why we have this function and after that we can say we have our beer entities so in all cases no matter what the load type is we can say the entities are or beer beers list so the dtos and we map them to beer entities and then we can say beerdb dot Dao dot up sort all and we pass our beer entities the reason we use with transaction is because we execute multiple SQL quotes here one after another and we only want to execute them all if they all succeed because let's say we would clear them all and that would succeed it will clear our database but then this call would fail for whatever reason and even though this would fail we would still have an empty database normally but this with transaction makes sure that we actually have a real database transaction so that either all codes succeed or none will succeed at all or it will at least not change anything and yes after that all we need to do here is we need to return a mediatorresult.success and we now need to determine if we reach the end of pagination or not how do we do determine that in our case we can check that by checking if our beers list is empty because our API works the way that if we pass for example page 100 000 which definitely does not exist then it will simply return an empty list so as long as there are elements our API returns for an incremented page we know that there is also a next page but as soon as there is an empty list the API returns we know okay this must be the end of pagination so please stop paginating here so now we've finished our data layer and the paging logic let's now see how we can take this and integrate it into our UI layer for that I want to go to a root package create a presentation package and in there we're going to have our beer view model will be a super simple view model make that inherit from your model and the way paging works is that we want to inject um something in the Constructor here so we say inject annotate this class with Hillview model and in this Constructor we need a so-called pager actually we don't need to make this a proud vowel we can just say pager and that is a pager that again uses a key and a value in this case beer entity and the job of this pager is to just provide a flow for the loaded entries and then to load the next ones so just provide something we can easily observe in the UI to get access to the newly loaded items and here this is a little downside of the paging library at least in terms of or if you at least if we want to implement that for clean architecture that we need to kind of couple our presentation layer here to our data layer so to our beer entities but I don't think there is a better way to to handle that because we can't map the beer entities to our domain model in the data layer because then otherwise this paging Source would not return something valid because this needs to work with beer entities and there is no way to map such a paging source to let's say a paging source of type beer so we need to expose this paging Source later on directly to the page that we use here so it also needs to contain these beer entities this might work if you implement your custom paging stores but I think it's just too easy to to return that in the beer Dao itself but we can however map the resulting flow this pager returns inside of this view model so we can say well beer paging flow is equal to pager dot flow a DOT map so we want to map this this returns paging data as you can see uh paging data of beer entities and we can then however map this paging data again two beers and then we simply say we cache this in viewmodel scope and now we have a resulting flow of type paging data of type beer which is our domain model so that is now working fine and this flow will trigger a new emission of paging data every time we scroll and we want to trigger loading the next page so I would say we now get into implementing our beer screen so the screen of a simple list in this case in our presentation package computer screen select file and make that a composable this will take a list of beers which is of type lazy paging items that comes from the paging library and it's just the way it uses to communicate with our lazy column we will Implement to easily implement this paging mechanism there it's of type beer and in here we can now join our screen first of all if we would like to show errors in form of a toast just to show you how you could show errors in this case it would be rather annoying to show all errors since this could also be just that the API could not be reached which is totally a valid point in an offline first app but just to show you how this would work we could get the context here could have a launch effect block and depending on beers that load state so whenever the load State changes we can then check if the beer's load state that refresh is actually a load State DOT error that's the case so if we are currently refreshing and an error happens then we can show a toast toast not make text pass in our context we can then say we have an error and the error is beers.loadstage dot refresh Dot error Dot message so that is just the throwable message and for some reason we get an error here because it wants us to cast this to load State error even though we should already know that this is a load State error but seems like this is a bit too complex for smart casting so let's just do this and then we don't have any issue anymore we can say toast uh no not make text toast dot length alone and show this and besides of error handling we want to now Implement our lazy column that actually displays our appears to be able to display a beer reverse of only the beater item so the composable which displays a beer let's quickly Implement that in presentation called beer item select file make that a composable like this we can add a beer that we want to display and a modifier like this and since I got the feedback that I should show the preview a little bit more in my tutorials to see what I'm building let's also add a preview beer item preview and in here we can have our compose paging three caching theme and put in a beer item in there we can then construct sample beer onto display just a beer ID could be one just one name could be beer tagline this is a cool beer first brood let's say something like this the date doesn't even exist yet whatever the description could be this is a description for a beer new line This is the next line just for preview and then our image URL is just now here because that won't show up in the preview anyways if we then go to split and let's actually first add a modifier off fill Max width and then we can rebuild this here to co preview right now we of course won't see anything but from now our changes will reflect in the preview so you can also see what we're building so as a little reference first of all um the final card will look like this so we'll have a card then a row with an image and a column for all these titles so using a card where the modifier is a modifier um just the one we passed by the way and the elevation is something like 4 DP import DP then and there we will have a row where we also assign a modifier fill Max width and padding of 16 DP and the first thing inside of this row will be our async image which comes from coil to load an image using a URL the model is nothing else than the URL so beer dot image URL content description is add a new just the name of the beer for example and the modifier will be modifier dot weight one F you will see why and we want to hard code the height to let's say 150 DP to not make this too large and even though no image is showing of course because we don't have a value URL this already looks a little bit like the card I showed you in my preview then below this async image I want to have a spacer with a height of oh no actually a width since we are in a row a width of 16 DP and now next is that spacer we're going to use a column for all of our texts and this column should have a modifier of weight 3F and what we accomplish with that is that this column here that contains our texts contains uh 75 percent of the width of our card while this one here the the image contains 25 of the width of our card that is what we achieved with giving this a weight of 3F and the image a weight of one F and then inside of this column we're just going to have a bunch of texts on the one hand we have a text that says beer.name this will have a style of material theme the typography H6 and we're going to give it a modifier of modified fiber.filmax web and I can already see where this is going where our text is going to be displayed for the title we can then use spacer give it a height of ATP and below this ATP spacing we're going to have another text for the tagline this will be beard tagline we won't assign a style here but rather a font style which is italic to make it a little bit Yeah I think that look looks cool for something like kind of a quote and the color will be light gray and you can see where this is going looks cool we can duplicate our spacer paste it below and then we'll have another text this time for the description this should not be italic and also not be light gray but this should show this description then another spacer and another text for when the beer was first The Brute so let's say um first brewed in and then we say beer that first brute you can see how this will look like then and want to make sure that the text align is end here so it sticks to the end oops not that end this text align dot end and we're going to decrease the font size to something small like 8 SP import SP and we can also make sure that our content is centered in the column for example by adding vertical arrangement of Center and then this will hopefully look a bit better um of course if we also assign a height to that row which is intrinsic size dot Max and we then say this column has a full Max height modifier this should fix the yeah the alignment you can see now it's actually centered since we have a fixed height for our row so intrinsic size of Max will simply assign the height for that row of the highest child of that row so since that is in this case here this async image which has a height of run 150 DP this will also assign this height to our row here so now going back to our beer screen we want to implement the list um first of all I want to wrap this into a box because we also want to show a loading spinner if we are initially loading so modifier fill Max size and import this modifier and if our beers that load state that refresh is load state DOT loading so if we are loading we want to show our loading spinner as a Sam so we're going to show a circular progress indicator where we say we want to align that in the center of our box however if not so if we're not loading in all other cases want to show our lazy cut for this later column we're going to assign a modifier of modifier.fill Max size we want to say vertical Arrangement is Arrangement data spaced by 16 DP to give each item or two to have 16 DP of space between each item and the horizontal alignment is Center horizontally in here we want to have an items block and now since we reach the paging library for compose there is the specific items import that uses lazy paging items and that is the one we want to use here um just without a key so let's use that pass in our beers get a reference to each view in the list and then we can show our beer item where we pass our beer like this and a modifier of modifier fill Max width we got an error here because the beer could be known theoretically so I want to make a null check before and only done show our beer item and now we took care of a loading and displaying our beers but this loading state is only referring to refreshing so if we completely reload the list then we want to show this uh Central loading indicator on our screen but we are loading more often than this initial load we're also loading if we scroll down to the list and then we want to load more entries to appendix to the list so that is also loading indicator we want to show so in case that happens we want to add another item block just a single item so if the load State the beers load State append this time because when we're appending then this low state will be changed if that is loading then we want to add a circular price indicator as a last item of our list but only then and that's not really everything we need to implement paging in our UI in composed products the last thing that's remaining for this video is to call the screen in our activity and then Implement di with beggar Hilt to provide in the end just this pager to a review model so let's go ahead in our root package presentation actually no in main activity I want to annotate this with Android entry point since we want to inject our review model in this and to make that work we need Android entry point when using dagger hild and inside of the surface the we will now put in our beer screen how do we now get this list of beers which is a lazy paging items list well for that we need our viewmodel which is Hillview model view model a view model to be specific and in this remodel we get our beers list which is viewmodel dot beer paging flow and for that we can say collect as lazy paging items that is also function coming from the paging Library so we can directly pass it to our view screen now and I think that's it yes and yeah that is how we get the items from the viewmodel but what you don't know yet is how we create this pager and provide it to the viewmodel for that I want to create DI package in the root package and Implement our app module this is an object annotate it with module and install in this will be a Singleton component and to be able to create this pager we need to create a remote mediator so or be a remote mediator and to be able to create a review remote mediator we need our beer database and beer API so all these are things we now need to provide in this module we use provides Singleton have a function to provide our beer database we need the context for that which we annotate with application context context like this and this will be true in our beer database and here we can then return room dot database Builder context is context the class is beer database double colon class at Java and the name is just beers.db without call.build and now Diego knows how to construct our database next up it should also know how to create our API provide beer API in here we can then return retrofit dot Builder we pass on our base URL which we get from our beer API dot base URL we add a converter Factory for automatic Json parsing which is Moshi converter Factory dot create and then we call that build and Dot create and our errors are gone next up the last thing we need is a function to provide oops to provide our beer pager provide beer pager which we will then finally inject in the viewmodel and this will need our brdb and our beer API and we'll then return a pager of type int and beer entity so how do we now construct this pager we want to return a normal picture just using its Constructor and this will now take a paging config it will take a remote mediator because we are paginating with a local cache as well if you're just paginating from the API and you're using this Library you don't need remote mediator and it needs a paging Source Factory so how we can create this paging source which comes from our Dao so for the config there's just paging config and we pass a page size of 20. so we load training items at once then the remote mediator will be a beer remote mediator where we pass our brdb and our beer API and finally our paging Source Factory here we now need to return the paging source of type int and beer entity and that is exactly the one we return here in our dial so you can see that all comes from room it will be generated code and we can then call this here so beer DB Dao paging source and we also need to go up here and I'll enter to add this experimental dump API and there we go we have everything at least in regards to dependencies the last thing to make the decorated work is we need an application class very simple just beer app in our root package in her from application and annotate it with field Android up we can then go to our manifest add this as a name beer app and we should be good I think this should be everything we need I would say we just launched this on my device and then take a look if we can see all of our items Oh and before we actually do this I forgot to add Internet permission ah we love to forget these things so this will probably crash um so let's simply go to manifest user permission internet and while we're already add this let's also go to our remote mediator and add a little bit of custom delay before making the API call so we can just see that the loading is working so let's say two seconds we then relaunch this take a look and yes we see the loading takes two seconds then the BLS is showing that it's cool we could add a little bit of padding by the way um but let's scroll down to the bottom then we are not seeing now we are seeing this loading indicator to load new beers so that is looking quite cool um if we now turn off internet by going into airplane mode for example and we're launching this app we should still be able to see some beers you can see we get an error that is working fine and yeah we see our locally saved beers we should also be able to see these if we go into our app inspection tab here select the process okay it automatically selects it and here in our beer entity table we should see all these beers that are now in our local cache with all the values so very very cool and this is how you can Implement paging 3 into a clean architecture setup together with a remote API and a local cache if you enjoyed this video then I would love to hear about that and also if you have any type of feedback just let me know that down below and I would be happy to read that apart from that thanks for watching this far I wish you an amazing rest of your week and I'll see you back in the next video bye bye foreign

Channel: Philipp Lackner

Views: 15,520

Keywords:

Id: AasI-0IRXUM

Length: 40min 30sec (2430 seconds)

Published: Wed Apr 12 2023