Master the Back-End Pagination With Node.js

in this video we're going to build a pagination system with node.js and the focus is actually in node.js so if you want to know how to do it in node.js you're in the right place no matter if you want to use it with react or normal javascript or no javascript at all or with sql or nosql databases well i got you covered after watching this video you're going to be able to build any pagination system that you can imagine whether you want to use it with react angular or vanilla javascript or no javascript at all on the front end or for the database you want to use mysql postgres or mongodb it doesn't matter you're going to be able to implement a pagination system anywhere even if you want to do it in java c plus or python it doesn't matter either because you're going to be able to do it everywhere since you're going to understand the fundamentals of a pagination system all right here we can see what we're going to build and for starts we first implement this in node.js and went on to javascript and no database at all and then i'm going to show you a couple more ways to do it i'm going to show you how to do it with react how to do it with sql databases like mysql and i'm also going to show you how to do it without any javascript on the front end whatsoever actually we're not going to code along all these solutions we're just going to go through the source code and discuss how things work also you're going to have access to the whole source code on your local computer we'll talk more about how we're going to do that as we go along the course let's imagine let's imagine that you have gotten a project from someone maybe someone from your company one of your colleagues one of your friends even from a freelance website like upwork it doesn't matter and your job is to implement a pagination system so your employer will just send you an email with your details and that's it and i'm trying to make the situation as real as possible all right so here we go so this is the email that john has sent us i have this project that i want you to set up a pagination system on it the project source code is at this address all right so first i want to do is open up this link and follow along with me if you can all you need to have is node.js installed that's all even if we're going to use multiple databases but it's not mandatory to have those databases installed so just node.js will do the job and even if you don't have node.js installed just go to nodejs.org website oops and then here download the version that's right for you the lts version and then just install it like any other software it's really that simple [Music] all right so the link was let's take another look yeah github.com slash agile 181 h1 slash node paginate and well it's a github repository so i'm going to clone it and if you don't have git installed or you're not familiar with git just download the zip file all right so i'm going to copy this remote url then i'm going to head over to my terminal and i want to navigate to my documents folder and here i want to clone this repository so it gets cloned and then i'll paste the wheel did i just copy it once done i'm gonna open up the project files in my code editor visual city codes all right so cool let's go back to john's email and see what else he had told us so the pagination system should only show 12 records on each page and i have already implemented the front-end logic you just worry about the back-end all right fair enough do all your changes in server slash routes slash articles route.js file and the client's js src index.js file alright that's all that's all we know so yeah let's see what we can do all right so let's now go over the source code and find out how this project is structured and how are we supposed to set up this pagination system and also notice that i change my code editor theme just to make it easier for you to see the codes all right so what's the first thing you do after you clone repository well you start to read the readme file right that's the first thing you want to do so i'm gonna open up this readme readme.md file and start reading it so this application will show a list of articles to the user all right then you're installing the dependencies and for this we have to run npm run ads from the root directory so let's actually do this pretty quick let's first install the dependencies so i'm opening up my terminal and first we have to navigate so it's cd node paginate yeah and then npm run at while it's doing its job let's get back to the readme file and then yeah here's an important part how to run the app you can start the app by running this command from the root directory and pm start running it you will be prompted to specify what kind of database you want to use you have three options the mongodb mysql or no database at all so choose one of them all right so now let's actually run this command and see what this means so now the dependencies have been installed successfully now i'm going to run npm starts now we've got a question back what database you want to use yeah this is this one so we have three options my scale mongodb and none i have neither one of these databases running right now so i'm gonna just choose none all right so the server has successfully started you can now open the app in the browser by going to localhost colon 4080 so let's open the app up localhost colon 4080. all right so we got nothing back that's well let's get back to reading file and see what else we need to do you also might want to add some data into the database oh yeah this is probably what we need to do that you want to use you can either do that manually or well of course not manually i have no idea how the database is structured or you can just run the c.js file by running this command from the root directory that's better so let's run npm run c from the root directory and see what this will do vm run seat [Music] let's get back to reading file then you will be prompted to choose what database you want to use i successfully if you want to see data to be saved in you have these options mysql mongodb and node database if you choose no database the data will be saved in this file if you want to choose mongodb make sure you have it running the same for mysql if you want to choose mysql make sure you have a server running and also you need to create a db call to this all right so let's see what we got back [Music] yeah this question what did you want to use well none all right nowadays will be used data will be saved in this file a hundred records has been saved to the data.json file let's open this file up and see what we have so it's server database and then data.json yeah it looks like git is ignoring it which is expected uh we got back a lot of data here and so let's see what we have inside of this file well it's a json file and then it starts from here we have this articles collection and then and which is an array that starts from here all the way to all the way to the ends all right so this whole data is just an array with these objects so this is the first object and it starts from here all the way to here you can see that visual city code is nicely highlighting this opening and closing brackets which is really nice and yeah we have another object here and looks like we have a hundred items of these objects so here it told us a hundred recurse has been saved to this file so yeah we can assume that we have a hundred one of these objects here and each one has a title yeah title and this is the title which is gibberish then a body which is kind of long starts all the way to ends here and then an author image right so yeah we have 100 objects here which which are saved in an array let's now rerun the app again and see what this time we will get in the browser so i'm gonna run npm start again and for database i'm gonna choose none and let's now go ahead and refresh all right so this time we got some data back looks like that this is coming from the state of the json file which is actually yeah it's come from that file we have a hundred one of these records which is a huge file i can scroll all the way to the ends yeah this is what we have right now and apparently we're supposed to set up a pagination system on this because well obviously getting 100 records back at a single refresh is not really a good user experience all right let's now go through the code a little bit more now that we have something on the screen all right so we started by running this hours it's this npm start command let's see what this does it's all it does is that it will start the node server from this file server slash server.js so let's open this file up there we go all right so this is kind of like the parent file everything we have is linked to this file so it's great to start reading codes from this file to understand how this whole app is working right so here we are including some modules express path body parser and course and then this read line so let's search up the documentation to see what this red line is so i'm going to search for node.js read line and i'm going to open this it's a native module to the node.js itself so we don't have to install anything to be able to make use of it here's the documentation for this module it said it provides an interface for reading data from a readable stream well all it does is that it allows us to read from the command line that question we got back with database you want to use this module has been used to read whatever we specify the command line so here we can see that we are creating an interface here and then we can ask a question so what do you want to your question here and then the answer will be passed to the callback function and we can make use of it alright so this is the red line let's move on we're initializing express our public path the ports and here this is the crit interface that we just saw so and we have this code here let's see what this does ask on the command line for the database type if not specified so yeah this is the question that we got back and it first checks to see if we have process.m.database if not move on and if we actually sorry if we don't have it then run this code and if we have it just move on all right so all it does is that it sets this to the process that m the database if we don't have this environment variable and this is an iffy format if you're familiar with it we're defining a function here and then we're immediately calling it after it's defined we're passing this string to it and this string will be passed as a question and then returning a promise this arnold question will ask this question and then the inputs will be resolved as a promise and then this await keyword will return whatever we specify back to us it will then be saved to the processor and the database all right so all it does basically is this question we got if i do it again npm sort here what database you want to use this is the question and whatever we specify right here will be saved to this variable all right let's move on and then we are here requiring this database it's kind of interesting we're going to get back here soon let's move on this is our routes apparently this is a route route which sends the index.html file from the public folder so public index.html file if we hit the slash routes uh or if we hit this route it will send us back this html file and then we have our api routes we will look at this later too and then well our 404 page and this message that server has been started all right so this database looks to be kind of complicated uh yeah we're just requiring this database let's see what we have there so dot database here's the database folder and we're not specifying anything after it so the index.js file is being included so let's open that up so we are defining a db object in this file and then we're exporting it let's see what this db is the db object will have different methods depending upon what type of database we want to use note that those methods will have the same name and the same return type and parameters all right so here it's saying what are those methods as of now this object will have these methods on it all right so this is just commenting here if you are not familiar with it visual city code is doing this highlighting so it's nothing related to code itself it's just comments so we have these two methods on this object db.countdocuments which accepts a string as its first parameter and its only parameter which is either the collection or the table name and then it returns a promise which contain number of how many documents we have so yeah then we have this method here db.find which accepts three parameters string objects which is the first one is collection name then our query contains what should be found passing an empty object means to find all then the options how to find those records storing sorting limiting and skipping so this is our options and it returns a promise which will result with an array with all the records that it has found it right so yeah this is the object and here we're doing this check let's see what this does so we're checking to see if process.m.database is one of these three then is not excuse me if it's not we are saying if it's equals to minus one so if whatever we specified is not one of these then just log in error and exits uh yeah so this is when we are here and we specify something that's neither one of these for example this yeah we're getting back this error please specify the right database to use for the app which is this one and then we're exiting out of the app right let's move on then here we're doing a check if it's mongodb require this and pass this db object to it if it's mysql again and the same goes if we choose none so let's go over one of them to see what exactly is happening so we chose the non for the database so let's see what's happening requiring this file or folder what is it yeah it's a file no dash db.js so let's open it and what this does so it returns a function and then we're calling that function again and we're passing this db object to it so if we look a module that exports a function which accepts the db objects and then let's take another look so this is the whole file it adds two methods on this object db.counts documents and db.find this is actually all i need to know that yeah this object will have two methods on it a find and count documents with finds we can find the documents that we need and with this count documents we can count how many documents we have in a specific collection all right so let's close this db file and yeah this is all we need to know about database for now to be able to understand how this app is working and to start doing our changes so i'm going to close this database folder and let's move back to the server.js file so this is our database then we i think we had something else to discover what was it kind of it got lost oh yeah this api routes let's also see this one you're requiring this file which returns a function and then we're calling it and passing the app variable to it so routes slash articles routes so routes articles routes article routes excuse me and yeah if you recall this is the file that we need to work on right the audio changes in routes slash article routes which is this one and client.js client.js.json we will get to it later so this is the route that will return the articles so we're requiring our db objects from the database folder and then we're using that db.finds method here to find all the articles passing an empty object i kind of forgot what this does so let's get back to it where was it yeah here this db.finds we're just passing the first two which is the collection name and the object which is our query yeah passing an empty object means to find all so this empty object that we're passing means to find all the articles and then we're sending back the articles data to the client site whatever the client is all right so so are we done with the server folder we took a look at this database routes server.js file and our config which just includes the confidence variables we're going to need so if you i don't know whether you're using it but let's just forget about it we'll get back to it if we need it and that's actually all about the server so this is the file that we need to work on and yeah that's pretty much it all right so that's it about the server folder let's move on to this public folder let's see what we have here we have a couple of svg icons we have our styles apparently yeah this is all our styles our html file let's see what this does it's our heads and link nothing too complicated we're linking to this style.css file and remember that when you go to the slash route here we are in the slash route here we are uh it went all right we have to restart our server so when we go to slash routes we are sending this file back to the user back to the clients which is which has this style.css file and this app.js file so this is basically just some styling we don't have to concern ourselves with this file so let's just close it this app.js file which is kind of interesting oh oh we have some crazy code here what is all of this so apparently it's a webpack file i mean this file is being created by webpack so we should have webpack somewhere in our project there it is i think this client.js file has our webpack yeah webpack configuration file let's open this up and see what we have inside of this file so recurring path then our entry is src slash index.js well so source index.js file let's open it up we'll get back we'll get back to it in a minute and then for the output we were outputting this app.js file right here as we saw and yeah that's pretty much that's all this webpack is doing our entry and then export we don't have any babble or anything like that set up on the swept pack all right so nothing too crazy for the webpack so let's close this file and go to the index.js file all right so let's see what we have here let's actually get back to the readme file and see we had something else here we're on the seed file and then yeah this front end logic so let's start reading it the front end is written in vanilla javascript es6 and css all right so we don't have anything installed for the clients it's just windows javascript in es6 so all of this files is written by john apparently so let's continue on no framework and library has been used right in the front we just have webpack which just bundles all the codes into a file in the public folder we just saw it and to change the javascript front end code you need to first start the webpack to be able to apply and see the changes all right so how we can do that to do so excuse me from the client.js folder run npm run start so if you want to do changes in this client.js file in order to see the changes and apply the changes to get back the right app.js file we first have to set up babel excuse me webpack so let's do it here i'm going to duplicate this tab yeah we have to navigate to the client.js folder and from here let's take a look at the package.json file for a minute our scripts when we have the start script which all it does is it setups starts the webpack server apparently well not a webpack server all right let's actually just run it npm start so it's watching the files if i do a save here compiling and yeah so it just starts from here and it outputs it to file it to the app.js file right so yeah that's pretty much it so we have this class here and then we are initializing this class and calling the render function on it so this is our class let me close up this site navigation all right so this render function is here and all it does is that it calls this dot fetch which is this one we'll get back to it let's actually get to it right now so our effect fetch function accepts a callback which is this one and then it sends a request to our api i also pass this page equals one and then whatever it gets back it will save it to this dot articles here all right so what is this requested yes i'm kind of curious because you're not installing anything uh let's take a look at it if you're using visual studio codes you can just click option click on whatever function or class or object you want to see the definition just option click on it and it will redirect you to the definition so an option click and there we go if you don't have access to this functionality just go there manually for example here request is here so import request from lib and request.js so just go there manually or if you have access to this option click it's pretty nice right so let's see what this file is this the object that we'll use for our requests so it's a class which only has a method on it which is called gets which accepts a parameter a url that we want to send requests to and then it returns a promise which will be resolved with the response property that we'll have after the external ajax request has been successful so if you're familiar with axes and packages like that it's pretty much the same we use this get method on it and it results with the request response all right so yeah this this is just a get request and it returns a promise so we're making use of this await and whatever it returns we're saving it we're using es6 destructoring which is an es6 functionality and then we're saving it here all right all right once we've called the this.fetch function we will have this our articles available on this property and we're initializing another function constructor here with the articles data and then we're calling the render function on it so let's see what this articles is so i'm going to option click on it there we go so let's see where this file is in case you don't have access to that option click functionality it's in source components and then articles.js let's scroll all the way up so this is the article's components when we initialize and call the render function on it it will put this html into the dom right so apparently all it does when we're re-initializing it and passing that data to it it will put this html into the dom based on what we have here in the comments so yeah let's that's all we need to know let's just close this file all we need to know now is that when we are doing this we are getting that html to the dom so if you recall in the app index.html file we are just including a div with an id of root and then that file is responsible to put the articles dom elements in here so forget about it right now and yeah that's actually pretty much it for the client.js file and i'm wondering where should we use the pagination all right so let's take a look at the file structure here we had in the client.js file folder we had this src then inside we have this components and lib we took we took a look at this requested js file which is an object that we'll use for our future requests right now we only have this gets method here but i can imagine in the future if you wanted to add post request patch request delete request we just add them right here so that's it for the request object then this is this is our library so let's go to components our articles which is including this article and this article is apparently only one of those articles so this is the whole article i mean we had a list right so this is our whole list and then it's making use yeah this is kind of confusing let's let's go yeah this is where we have it we have a pagination file here so we have to make use of this file if you wanted to because if you remember here in email uh here where was it i have already implemented the frontend logic you just worry about backend so apparently we don't need to do any changes in the front end but we just have to figure out how to fire up this pagination because we're including it here and it's not being used so let's go to this pagination file i'm gonna option click or just go to this file it doesn't matter and all right so this is the file that we need to work on so let's dig a little deeper into this file this is the pagination components when we initialize and call the render function on it it will put this html into the dom so apparently just like the articles components when we're initializing it and calling the render function on it it puts something into the dom in this case it's this one so we need to make use of it let's see how we can do that we are let's close this file that we're not going to need them so we're including it here let's initialize it right here after this articles class so just like articles so new pagination oops pagination and we also need to call the render function on it right but we have to pass some data to it let's see what we have to pass to it so again option click this is the constructor oh we have to pass a lot of data to it so let's see what we have data total pages all of these variables that i have no idea what they are and let's actually put something for them let's pass something because i want to pass an object because it was an object right this data we need to pass an object so i'm going to copy all of these variables i'm going to press commands and then double click on them to select them all and then command c to copy them and right here i'm going to paste all of those so let's pass something for them i have no idea what they are i wish we had typescript it says it's any so we i don't know if we need to pass an object to them or a string or a number so let's see what we have here so i'm going to select this and hit command d to see where it's used well here so i can assume actually it's a number yes yeah so we have this i it's a for loop and we're checking if it's below this so it's a number so i'm going to pass five for it i think the current page i think it's that's a number two yeah so let's check command d we're checking if it's triple equals to i what is i yeah i is a number so what i equals one so i'm going to pass y for it too oh sorry five total results again i think all of these are numbers right total results it's like a string right see where else we have it only here so it doesn't matter if you pass a string or number so total results almost past 1 100 showing from and showing until let's do a check there too yeah if we had typescript we didn't have to do all of these it's just like total results can be either a string or a number so i'm just past 20 showing until oh geez that's that's the same here so that's number i can assume just like before so we're going to pass 40. all right so we're initializing the pagination here and then we're passing this object here well i guess actually this object needs to come from the server but for now let's just put this and then we're calling the render function on it so let's check let's see if we have our server running so yeah we have the client.js and let's also start the note server simpium start i'm going to use none and let's go ahead and refresh all right so let's scroll all the way down i'm going gonna hit the end key on my keyboard to go all the way to the end of the file and yeah there we have it we have our pagination obviously it's not working or it's not showing the right page we're just passing this data but as you can see here it's this 20 what was this 20 so this is showing from this 20 and then 4 which doesn't make sense to have 4 here this is showing until and what is this 100 total results right and then total pages five so let's change it to eight just for fun actually just 12. so what's gonna happen the end key all right so we got 12 pages so yeah this is total pages current page five and yeah we are on fi under five page so that's it just make it to uh like 90. it doesn't matter what we passed here they are just pseudo values right we're going to change them later all right so we successfully fired up our pagination let's see what's going to happen when we click on one of them obviously it's not oh it it scrolled us all the way back to the top interesting but nothing is happening right so we are on page five it just gets us to the top of the page let's see what's what is that code that's doing it so let's get back to the pagination file all right we should have an onclick event handler somewhere so fetch and render let's close this and let's close this function our render function is here all right i can see that we are making use of this el file actually in a lot of places so let's see what this el is some option click on it where is it it's in lib slash el.js so let's scroll all the way up this class is responsible for creating domains oh so yeah that's all we need to know this class we're using it to create dom elements and we're using a pattern here this design pattern is called the builder pattern where we're where we can actually do this so if you can if you see here we are actually yeah we are initializing the class and then calling dot class name and then dots and other methods and then dots oh here's on click event handler who quite found it by chance but so yeah this is we use this class to build or create dom elements right and then at the end we call this build methods yeah we're connect this is the builder pattern if you're not familiar with it and all it does is that it returns this this dot el this class which is uh no not this this this dot el which is a documents which is a dom element to your document.create elements all right so yeah that's pretty much all we need to know and then on this on click event handler which we are quite interested in because you saw that whenever we clicked on one of the links apparently this code kicks in and it got us all the way back to the top of the page so this is the on click and all it does is that it sets whatever function we pass through that on click on the this on the click event handler on the dom elements so yeah that's pretty much it so we're calling it and then it does a check here and yeah this is apparently this at fetch and render so let's see where this function is it's right here so yeah here's the window.scroll we saw it and then we were sending a get request to the server we're passing a page to it and then here we are re-rendering pagination and articles all right so yeah this is the code and now what we need to do actually all we need to do is back in this file articles routes we need to pass the right data back to the client something like pat pagination data and this object here will have all these properties on it just to demonstrate i'm not going to run this code right now so all we need to do is to pass right objects here all right so let's get started on working on this project all right now enough of john john didn't wrote this code i did it for the purpose of teaching you about how to do pagination on any system and what you saw just now me doing a code walkthrough over an unknown source code to me is what you will be doing a lot in your career as a software engineer so this is actually kind of like the basics i mean you get an email or something that says your objectives and some general idea about what you need to do and then you need to look on the source code and try to understand how the system the application works then you do your changes you push your commits into a branch then the viewer whoever he or she is will review your code and boom approved so this was kind of like a real world example and it's fine if you don't understand some parts of the source code for example you might have no idea what's going on in this file that's fine you should just understand as much so that you can start working on the projects you can definitely go ahead and try to read the code of the other parts as well if you wanted to but it's not really necessary to really understand everything in a project to be able to work on it so don't freak out when you get a crazy big source code and you're supposed to work on it just take your time to get a general overview of the source code and most importantly the parts that you need to work on in our case we know that we just need to work on these two files but in most cases no one will tell you on what files you need to work on but that's fine too i mean you just need to figure it out yourself in our case it was definitely clear that yeah here's our get handler for the articles and then here's where we need to set up the pagination right but yeah in a lot of cases you need to figure this out yourself no one will tell you only that you need to work only on these files [Music] or for example if you change the file that you are not supposed to change well the viewer and github pull request will just tell you that please don't change these files and don't do your changes here and you just change it to make everyone happy it's just a matter of seconds to do so alright i could just as easily not do any of these i could just not give you this code and say that hey let's just start to build a pagination with actually no front end and probably if you are kind of feeling confused about the source codes you wouldn't be in that case but if you don't challenge yourself you're not going to be a better software engineer you need to challenge yourself you need to do the things that you are not comfortable with you need to make mistakes you need to experiment and that's the only way you're going to get better and every single video on this channel is aiming toward making you a better software engineer so if we say yeah let's build a pagination system that doesn't just mean that yeah a simple pagination that you actually might imagine all right so in case you skipped the code walkthrough we actually did nothing except talking about the source code we did some changes but actually let me undo this so and then here all we did in the client-side source index.js file here we just make use of this pagination that has already been included so we just actually may also remove this line so we did nothing yeah so if you skipped the code walkthrough we did nothing all right so now let's start to understand the pagination logic all right so this is our server and this is the client the server can be on whatever we want it doesn't matter either it's node.js java c plus python or whatever and also the clients can be on anything react ios angular android etc so we have this bunch of data and also just to make it clear each one of these circles represents a record maybe you want to send a list of posts to the user or a list of images products users or any other in this example we have 105 of these so imagine that in our database we have 105 posts and user will go to her feed and want to send some of these posts to the user well obviously we don't want to send all of these data to the user at once right that would be too unpractical on an awful user experience there is no order and also downloading this much data will take a long time for the user imagine we had millions of records here instead of just a hundred if we were to send all of them at once well it would take the user forever to download them if they can't ever download them and actually even if they download them chances are that most probably most of the clients computers cannot handle to render this much data so how should we do that when someone goes to their feet or slash latest news or slash top post or whatever how should we send the data to the user so we need to set up a pagination system right and it can actually be in many forms either an infinite scrolling which you see a lot nowadays especially in social media platforms like youtube instagram linkedin where as you scroll new data will come in or one of those classic page edition systems like the one we have right now in our app so to do that the first step is that we need to sort our data right we can sort based on whatever we want a date title popularity number of comments etc so we first sort the data i've added a couple numbers here and there just to indicate order so we first sort the data and then what do you think we should do well let's have an example imagine that user is on page one and the limit is eight so this limit means that how many records we want to show on each page so what do you think we should do in this case well as you've probably guessed we need to send the first eight items to the user as simple as that now what if they go to the second page well just like before we need to send eight items to the user but this time we have to skip the first eight items because they are for page number one so we skip the first eight items and then we send the second eight records to the user now what if they go to the page seven well now we need to skip the items for the first six pages right so six times the number of items we have on each page in this case it's eight so six times eight equals 48 so we need to send eight records starting from the index number 49. so the logic is the same if we go to any other pages or how many we specify to be our limits so let's do a quick recap we first need to sort the data by whatever property we want then we need to skip some amount of data and then we need to limit and as for the formula for skipping hopefully you have figured it out by now so it's this limit multiplied by the number of the page that we are in minus one so if we're on page number one we have to multiply eight the number of our limits by the number of page minus one so it will be zero meaning we shouldn't skip any data if we are on page one likewise if we are on page two we multiply eight by two minus one which will be eight meaning we need to skip eight records if we are on page number two one last example if we are on page 19 we multiply 8 by 18 which will be 144 which is the number of records we need to skip right so this is the logic behind the pagination system and now let's go ahead and implement this in node.js all right so i'm in the server slash routes slash articlesroutes.js file and now let's start writing some codes so remember we said in our query we first need to sort and then skip and then limit so let's do that here right so in the db.find function as a first parameter we're passing a string meaning the name of the collection or the table that we want to find the data from and then for the second parameter we're passing an empty object meaning to query all the available data and then we can pass a third object here to specify our options don't worry we'll take a deeper look at this function but for now just know that this is the article's name excuse me this is the collection or the table name this is our query and this is our options like skipping sorting limiting and such for example if you want to say to select the documents which are older than a year or to select a document that has an id of 8 we would specify that here in the second parameter and then here as i said we specify things like sorting limiting skipping and things like that so let's first sort the data by the date property and then here we should either pass one or minus one indicating to either sort ascending or descending just like mangoes if you're familiar with it so i'm going to pass -1 because we want the latest articles first and then for skipping so skip colon remember the formula number of the items we want to show on each page in this case we want to limit to 12 so 12 and then multiply it by the number of the page minus one and then we have to limit so limit column 12. let's actually save this number to a variable we're using a couple times here and here so here at the top i'm going to find a new variable called const per page i'm going to set it to be 12. and i'm going to use it here per page and then here per page all right so this per page is the number of how many items we want to show on each page this is our limits all right so now this function is now giving us back 12 articles data based on these options that we specified let's actually log this variable to the console just to see what we have so console.log articles alright so i want to save and i'll restart my server now so back in my terminal actually we don't want to restart the server every single time we do a change i have included another script here called dev if we run this it will concurrently run npm run div nodes and npm run dev which each one of these will run this one will run our server and then this one will run the webpack so let's just run npm run dev instead of npm start every single time and also we have to specify our database here so let's change it to none because we're not going to use any database just yet so now i'm going to run npm run dev all right npm run dev and also i'm gonna shut down the webpack because we're gonna run it in the first tab i let's just close this one so i'm running tiff this time let's just run it finally right and the database has been uh yeah this is a web pack and then this is the message we'll get back when we start the server all right so let's hit this slash route get handler so to do that all we need to do is to do a refresh on localhost colon 4080. looks like something is not right let's get back to console oh we have an error so what is this error page is not defined oh yeah that's right i didn't define the number of page so let's actually we have to grab it so const page equals rec dot query dots page remember this page is coming from here so this is pagination whereas it was in components pagination this this query parameter so we need to save this we need to grab it in this route so cons page equals direct.query.page we also have to parse ins because we don't want it to be a string we always want it to be a number so oops parse ends there we go let's the server has successfully restarted let's refresh all right we got our articles back and looks like this time we're not getting a hundred articles back you can see that it's only 12 because we specified the limit to be 12 and let's take a look at console all right so it's i think it's a little hard to read but let's try our best so this is the array of articles so this is an array we're passing back to the user to the clients and it contains of 12 objects i'm not going to count them let's just trust that yeah we have 12 articles here or 12 objects here you can count them if you wanted to all right all right so at this point it might not seem to be so but actually our pagination is pretty much complete we can pass how many records we want to show on each page here and also the page number here and then we'll get back the right data that we need for example let's put four here for our limit and see what we'll get so i'm gonna head over to my browser and also notice that we have node one running so we don't need to manually restart the server every single time that we do a change and i want to refresh and there we go we now got four articles on a page let's try a couple more values so i'm gonna put two and now we have two articles and let's also try one and one is also working just fine all right same for this page number whatever we pass here when you get back the right data i'm not going to do it here but you can do so if you wanted to so now let's get back the pagination indicator in the dom so to do so from here we need to send another object back to the client pagination data right and this object needs to have all the data that the pagination component needs for its initialization so in the pagination components if we go to constructor here all of these data that you see pagination needs all of them to be able to render itself correctly and work as expected so we need to pass all of them so let's start from total pages so total pages and for now let's just put a static value for all of them later we're going to get back here and calculate every single one of these numbers so don't worry current page current page on a plus one what else we add total results showing from so total results let's press the hundreds can't type and then showing from let's put put one and then showing until for this output 12. so yeah we have all of these data here and now let's save and now to fire up the pagination components we need to go to client js folder then source src and then index.js and here we need to fire up this pagination so for that right here when you're sending the get request i'm also going to grab the pagination data pagination data like so and then i'm going to save it to the class property so this dot pagination equals pagination data let's also add this property in the constructor so our pagination data this dot pagination equals and i'm going to set null for its initial value and then all we need to do is right here we need to initialize the class so new path generation and then we need to pass this data to its constructor so this dot pagination and i'm going to call the render function on it all right now let's save and take a look we have an error so it says article data is not defined right here this stuff so yeah we made a mistake somewhere oh i mistyped here i don't know how i did it yeah i dropped a and also i mistyped this pagination data and let's take another look all right so yeah we got the pagination back on the screen and actually it's now working so i can go to page two i can go to page seven four but you notice that these numbers are not the correct values and also pagination is not showing us the right page that we're on right now to make all of this to work we need to get back to the article routes and calculate every single one of these values so all right so let's do that remove this console log here and let's start with the total pages so for total pages what we need to do is actually pretty straightforward we just need to find out how many total records we have and then divide that number by the number of records we want to show on each page and that will be our total pages so let's do that right now at the top i want to make use of another method that we have available on the db object which is db.count documents and this method will accept the parameter which is the collection name or the table name if you're using an sql database sound path articles and this will return a promise which will be resolved with the number of documents that we have in that collection or table so i'm going to save that and a variable called total articles len short for length and then i'm going to use it right here so total articles when and divide it by the number of our limits which is 12 and we have it saved right here per page right and this expression here won't always return a whole number and in those cases what we want to do is to actually just pass it to math.sil function so what this sill function will do that we have available on this math global object is that it's well around this number to the next highest integer so if we get a value like 8 points this it will just convert it to 9 which is what we want right right and then for the current page we actually already have this value and this is our page right so i'm just going to pass it here and actually the same for our total results we already have it and this is our total article articles len so total articles len this was our total results right and then for showing from this is actually how many records we skipped plus one right for example if we are on page two we have skipped 12 records and now we want to show from the record number 13 right so we just need to copy paste this code here and then add 1 to it so per page times page minus 1 and i'm going to wrap it inside of the parentheses and then plus one we need this parenthesis here because if we are on page number one we want it to return zero and therefore this will be zero and this whole expression will return one so if we're on page one we want to show from the first article right so showing from needs to be one if we are on page one and if we are on page two this would be one and then one times per page which is 12 plus 1 equals a 13. so on page 2 we want to start from the record number 30. all right and for showing until this is actually again just this one but plus 11. and actually we can just say per page times page this would be our showing until so let's save and now let's head over to browser and see actually let's also change this variable back to 12. all right so now i want to head over to the browser and refresh and now if i scroll down you can see here that we have nine pages where our total pages and let me zoom in a little and for total articles we have a hundred which is true and here we can see that we're showing from the article number one up until the article number 12 which is correct and i can go to page number two as you can see here 13 up until 24. and obviously his total results and his total pages will not change no matter to what page we go all right so it's not working but we still have a tiny little issue if i go to page oh actually it was not expected so we have an error somewhere sorry for that all right so sorry i had to possibly for a moment to figure out what was going wrong and here in the skip we forgot to wrap this around the parentheses so yeah that was a problem sorry for that if we omit these parentheses then we're first doing these and then we're subtracting one from the total results which is not what we want so we want to first subtract one from the number of pages we're in right now and then multiply that by the number of our limits to get the right number on how many records we need to skip all right so now if i save refresh and now go to page number nine so yeah it works and here we have four articles which are the remaining ones and here if you take a look showing from is correct but showing until is 108 but we have only 100 articles right so we definitely have a tiny little bit of issue here so let's fix this for this all we need to do is right here we just need to do a check to whenever this number goes beyond our total results so whenever this is greater sorry greater than total articles len then what we need to do is we'll just show total articles land right otherwise just show what we had before per page times page now if i save and head back over and refresh now i'm going to go to the last page page number nine here we can see we are showing exactly the right number from 97 up until to 100s which is the last article cool okay so at this point our pagination is complete and it's working flawlessly and i want to show you how to use this logic this code the server pagination code with roads with different databases so first we'll use mongodb and then we're going to use mysql so to use mongodb i first have to start the application the mongodb server and to do that i need to open up my terminal and i'm going to create a new tab here let me zoom in a little and i need to run mongod and i need to pass the db path to this so db path in my case it's in slash users slash agile slash data and then slash db so i'm going to run this command which will start the mongodb server and now i'm going to head back over to the node paginate terminal actually i'm also going to open up another application on my computer called robomongo with this we can see what's going on in our database so i'm going to choose localhost and right now we haven't started our application yet so let's now go ahead and run the c.js file to get some data into our mongodb database so i'm going to run npm run seed let's wait for it to finish grabbing the c data from the internet now what database you want to use this time i want to choose mongodb and we will get a question back do you want to first delete all the existing records and then add the seed data it doesn't matter what we add at this point so let's just add yes all right so let's get back to robomongo and i'm going to refresh here and this time if you notice we've got this node page in a database so this has been created by the c.js file and inside we have collections and then articles and you can see here we have gotten back a hundred records all right so we have our database correctly set it up now let's go ahead and rerun the server so i'm gonna run npm start and this time i'm going to choose mongodb because we're trying to learn about mongodb here and here the server has successfully started and we got this message back successfully connected to the mongodb database so now let's go ahead and refresh well we won't see any difference but actually this time these articles are coming from our mongodb database as opposed to a json file so yeah this time we're using mongodb great now let's see what's happening in our source code when we're using mongodb to find out we have to go to our database folder and then index.js so here we can see that if we specify mongodb we are requiring this which is mongodb right here and then index.js file so let's open this up so here at top at first we're requiring mongoose and then our article model which is this it's located in models and then slash article dot js we have title image body author and date so this is our article model and then we are requiring our keys which contain the url that we're going to need to pass to the mongo.connect function to connect to our database so let's see what we have here here we can see our keys we have mlab url and if you want to use your own database make sure to pass whatever database name you want to use or the port number that you have already specified all right so let's move on then here we are connecting to the database using mongoose.connect function and then right here we are adding those those two methods on the db objects so db.com documents and db.white so let's first start with the db.count documents here we're just doing a check on whatever collection name we pass and then we're using this count documents which is provided by mongoose to count how many documents we have in the article collection if you are going to use this with mongodb well here i've set up this object here just to make it easier for you to use it with different databases and see how the logic will defer but in your case you might not want to set up something like this and all you need to do is to just use this logic so if you want to just use mongodb here you would do something like this and so so you would say cons total articles len equals awaits and then article which is the model dots you can see here that visual studio code has automatically included it which is in database mongodb models and article and then you need to call the count documents function on it and then dot exact so you need to use this syntax instead of this if you want to use mongodb with mongoose and you can see there isn't much of a difference between them all right so that's for the counts documents methods then we have defined methods again we're doing check on the collection name and if it's articles we're using this article model to find and then we're passing the query null and our options so if you wanted to use mongoose here instead of this you would do something like this const articles equals awaits and then article the name of the module dot find your query and then dot sort so one sort by date minus one and then dots skip for skipping you just need to pass this like so and then dot limits and for limits just per page so this is what you would do if you wanted to use mongoose with mongodb instead of this all right so i'm going to undo these changes so i'll remove this and i'll remove this so that's it that's how we can set up pagination with node.js and mongodb now let's move on to sql databases for this app i set it up mysql but the logic is pretty much the same if you want to use it with any other sql databases like postgres so let's take a look so let me first actually start my mysql server so i'm going to go into my system preferences and then from here i go to mysql and i'll start my mysql server and put in my password now my sql server has been running so let's confirm i'm gonna go to my terminal and run my scale dash u user root and then dash p so i'm gonna connect my mysql database so i am now connected show databases excuse me show databases and now we have to create this database so if we go to database folder and then index.js actually we have to go to keys so here we can see this is our mysql url so we need to create a database called nodepagenet so let's do that right now so here i say create database node paginate all right then show databases and you can see that we have created our node page in a database let's use it use node paginate and then show tables we have nothing yet so now let's go ahead and run the seed file 7pm wrong seat and this time we're going to use mysql and for this i choose no so 100 occurs has been saved to the articles table let's run again and add a hundred more to our database just to see how well our pagination will keep up with 200 records instead of just the hundreds so we have to specify no because we don't want to delete the existing records and our 100 records have been saved so show tables again so we have our articles table let's select all from articles i know it's not readable at all but you can see here 200 roads so right now in our mysql database we have 200 articles so let's run our server again npm start mysql successfully connected to the mysql database great let's go ahead and refresh looks like we have an error so connected to mysql database cannot convert undefined so oh sorry i made another mistake here we here in our articles routes article routes i keep adding an estimate here this date minus one we have to add it to the sort object so this yeah because if you look here in our mysql syntax we're searching for a sort object so just wrap this date inside of another object called sort now let's rerun the server and we can start mysql and i'm going to refresh all right so everything looks good this time remember we have 200 records and if you look at the pagination it kind of looks weird because well obviously you don't want to show 17 pages here right so this time we have only 200 if we had a million this would be very ridiculous so what you can do here is that in the server logic just show a couple of pages i mean to specify what page numbers we should show in the clients for the pagination you might just want to show i don't know maybe eight pages or six pages and then automatically change what page numbers you're showing based on what page number the user is currently on and it's actually an easy thing to do you just need to find out what you want to do in the pagination and it really depends on your app and your business or the ux stuff for example how users are supposed to use your app actually let me show you one way to do it so that you'll see how easy it is so right here in our pagination data we need to send two more properties to use or two more values of the user from page and until page hopefully these names make sense from page means from what page you want to show the pagination and until page means until what page you want to show the pagination for example if i put one here and then here i'll put six it means that we're going to show from the page number one up until to the page number six we need to calculate these two numbers here based on the current page we're on right now and you know let's just put these two here just to see how this will work and then all we need to do in the pagination components is to first grab those data so this starts from page equals data dot front page and then this dots until page equals data dots and cell page then here in our for loop when we're rendering the pagination pages all we need to do is to initialize this i to this start from page and then up until this dot until page so now the pagination pages that we'll get will be based on our front farm page and our until page let's save this and check it pretty quick i'm going to first restart the so npm starts mysql and we also need to start the webpack let me see if i have it started so if you don't have it started just run npm start on the client side client.js folder and then do a save there we go and then i'm going to go ahead here and refresh and here we can see we're only getting six pages i can go to page number two page number four and page number six and then here we need to do a check to see if we reach the end just show the next pages and hide the first pages for that we just need to do a little bit of checking here so here at top i'm going to define two variables let's front page and then until page and your onset from page equals if you are on page number one set it to be one otherwise set it to be page number of the page minus one and then for the until page just front page plus five because when it shows six pages if we want to show eight pages for example change it to be seven and then we need to do one more check here if until page is bigger than our total pages and we also need to define this total pages so let me cut this code from here and then define it right here so const const total pages equals that code and then i'm going to pass it right here we don't need to pass it just put this this is an es6 syntax so it's already prettier formatted in a way that we don't want it to be then let's do our check if until page is bigger than our total pages which is of course not once we just need to send until page to be total pages and then set the front page equals until page -5 we need to set these two variables here so i'm just going to remove these two to use es6 and there we go let's go ahead and test it i'm going to restart the server first so npm starts mysql and now i'm going to refresh we still got our six pages if i go to page number two still six pages page number three you see that now the page number one is gone and now instead we have page number seven if i go to page number seven we have we're showing from page number six to eleven if i go to 11 15 and then 17 which is the last page you see that it works great so here we're only showing six pages and you might now be asking what if user got all the way to here and now he or she wants to get back to the first page well all they have to do is refresh you can if you wanted to add two more buttons here to redirect the user to the last page or the first page if you wanted to as i said it really depends on what kind of application you want to build how the users are supposed to use your app and things like that so whatever you want to do right here you can do that with this knowledge that you have right now for example you might just want to show three dots here and then show the page number one that's we can do that too we can do a whole lot here it's as i said it depends on what you want to do all right so that's it and how we're talking about mysql so let's get back to the mysql syntax and see what we're doing when we specify our database to be mysql so i'm going to go into database and then index.js and here we're requiring this if we specify the database to be mysql so let's see what we have there it's in mysql folder and then we only have one file here called index.js so let's open it and here as you can see we're requiring the mysql package and then our keys just like the mongodb and then here we're connecting to the mysql server we're also showing this message successfully connected to mysql database and then here we're creating our table our articles table with an id image title body author and date and then here's the important part our two methods db.counts documents which is which we're going to use this query for it select count all if you are familiar with sql you should know what's going on here we're counting basically all the documents that we have in the table name so that's it for count documents pretty easy just one sql line then for the finds we are again using this query as you can see here we're selecting all from the table name we're ordering by whatever we specified in the sort so let's get back there let me close these files if you recall you're saying sort and then specifying an object with a date property and then we were using it here so we can imagine this sql query would be something like this order by dates then limits this is the offset how many want to skip so this would be like 34 for example then limits number of our limits for example 12 and table name articles so you would use this syntax if you wanted to set up a pagination with node.js and an sql database so pretty easy and also just like we talked about to mongodb you might not want to have an object like this so what you need to do in that case is just copy paste this code from here to here if you wanted to use mysql so you can see the syntax here i'm going to undo these changes and also i noticed one thing here that if we specify a true or false value here it wouldn't matter in our mysql because i didn't add these two oops here so we need to do a check here at first to see if this is a true value or a false value we need to either show asc short for sending or d e s c short for descending so that's one thing you might want to do if you wanted to in this case i noticed something here we're actually showing the oldest article first so here we can see three months and if i go to the last page you see that we're showing the latest articles at last so we just need to add something here des c now if i restart sorry we should have used nodemon i didn't know that we're going to restart this much and now you see that the date is correct and we're getting the latest articles first so that's something you also want to add if you are using sql right so that's it that's how we can set up a pagination in node.js if you want to use mysql or mongodb we also have one more thing here if you want to use it without any database you can see the logic is pretty much the same we were first sorting here and then we're doing our skip and our limits so it wouldn't matter what kind of database you want to use you can use firebase you can use mysql postgres it won't matter just know the logic here we need to do sorting skipping and limiting all right all right so now let's talk about how you can use this pagination with react angular view or any other framework in library on the front end well if you go to the pagination components here in the client.js folder and then src and then components then pagination.js and if you start reading this code you'll notice that actually we're not doing anything crazy here for example all this function does here this render pagination pages is that it will run a for loop and each time it creates a new button element and then for each button it adds this class and if that button has a page number that we're in then it will add this class to indicate that it's been selected same goes for this disabled property and also which button will have its own click event handler which is responsible for scrolling us back to the top of the page and also sending the ajac request to the server with the correct page number and then the similar larger cure for the previous button and also for the next button and then here's our pagination info it's just a span element with this text inside of it and then right here we are adding all those elements to this parent div elements and then here at the end we're rendering that div element to the dom so if you understand this logic here you should be able to use it with any framework and library you want just render the buttons and then customize each one of them based on the data they will have so i took the liberty and rewrote the front end and react for you just to show you that the logic is the same no matter where you want to use it you know just in case all right so to see the react solution go to this link the our repository link is in the description box by the way and then here make sure that you choose the react solution branch once you did that come here into the code and then download the zip file the file that you're going to get is the same as we have right now but the front end will be in react and if you can use git well then you don't have to download this zip file you can just use git to pull this branch into your local computer let me just show it to you so if you follow along with me and did exactly as i did if you come to the project folder and then run git status you'll see these files that we've changed now to pull that branch the react solution branch first i'm going to stash all these changes so get stash and then i'm going to create a new branch get branch react solution solution and then i'm going to check out that branch hit checkout react solution once here let's take a look at our logs so we need to first remove this last lock so just run kids reset head and then tilde this will remove the last commit that we did and then once i'm here let's run get status again let's just remove all these changes i'm going to run get checkout all and then finally i'm going to pull the branch get pull origin react solution all right so we've pulled the branch so if we run git branch we're on react solution and if i go back to my code editor now well here notice that our root file structure hasn't changed also our server codes and package.json file hasn't changed what has changed though is the client.js folder this time into the vanilla javascript we're using reacts before we go into this folder let me also show you how we can get back to the previous branch in case you wanted to so here we are on the react solution branch if you want to get back to master just run git check out master once you're here just run get stash pop to get back to changes so now we are on the solution with vanilla javascript and if i now again stash all these changes again and i'm not going to explain what these commands mean what the skid commands mean because i said if you are familiar with kids if you're not just go ahead and just download the zip file for this branch and then just unzip it and then install the dependencies and just run the app as before but if you know git it's a little bit easier to do it this way but you should know it gets so get stash and then get check out react solution so this is how we can switch between these two branches or you can just commit all these changes if you don't want to run git stash every single time all right so now let's go ahead and explore the react solution right so in the client js folder this time we have the source just like before but just this time actually we're using react right so if i open up things that suggest well here we can see we're importing reacts react dom and then our app which is in components and then app so here we're creating this class which is rendering our pagination and also our articles so if you are familiar with react you should understand exactly what is going on here but what i wanted to show you is the pagination so let's open it and here we can see that it's actually very similar to what we had before so we have our for loop actually first we're grabbing the pagination data then we have our for loop and then we're creating the buttons for the pages page numbers here's the disabled attributes the class name and that little check then our unclick event handler then if i scroll down here we can see our pagination this is our previous button and then here are our pagination pages and this is our next button and this is our pagination info so feel free to read this code if you wanted to and that's it that's a solution with react let's also run it and for that we first have to go to our terminal here and actually first have to install the dependencies so just run npm run adds well we have to do this because now we're using react and we have to install a couple more dependencies to be able to use it and this is just to install the client js folder dependencies and the server was still using the previous modules so we don't need to run npm install on the server so just for the client.js folder all right once the dependencies have been installed just run npm run dev and this will now start our nodemon server and also our react server also i've used create react app here so it should automatically redirect us to the page once started there we go and this is our app and this time we're using react you can notice that it looks and behaves just like before well except that's only this time it's a little bit slower because we're using reacts and that's yeah that's pretty much it that's the solution with react right i also wanted to point out that this is not the only way to do a pagination in node.js there are more ways to do it but at the end of the day they are all similar for example another way we could do this would be to use the express middleware so just to make the pagination logic usable to be able to use it if we had any other routes and we wanted to set up pagination on them or we could transfer some of this calculation here to the client side to put less effort on our servers you might not want to do that in tiny apps with small traffic but for enterprise applications and apps with a lot of traffic if we move some of the calculations to the client side well then we would need lighter servers imagine that millions of users are using our software if we're getting a million requests each hour for this route if we assign some of this code to the client side uh i mean to let the user's computer to do them well then our servers will have to do a list of millions of calculations each hour and that means we have to spend less money on the servers again that really depends on your business and your application that what approach you might want to take you can also do infinite scrolling if you want to do that again the server logic won't change well except that we will no longer need this pagination data and also all of this code for the pagination data which is really nice and then in the front ends we just have to add an event listener to check that whenever we reach the bottom of the page well just send a new request to the server to grab the articles of the mix page and then render those articles right after the previous ones i'll soon publish a short new video here on this channel talking about how to set up an infinite scrolling so be sure to be subscribed if you want to see that and also just for fun i did a performance test for both our reacts and our no react solutions and you can see here that the react version is almost two times slower than the vanilla javascript solution i had ran the npm ground build for both of them to get the production built and obviously the reason that react solution is slower is because we have to include a huge amount of code i mean the react source code and its dependencies well most of which we're actually not using in our project so yeah that's why the react solution is slower and that's why i don't use react for small applications and this is actually a very small app i mean if we even had a profile page a home page log in and sign up i still wouldn't use react unless i wanted to sacrifice performance for a simpler development process anyway you can do this test yourself if you wanted to just check out between branches and then run the app every single time and open up the developer tools and go to the network tab and also disable the cache and then set the network speed to be slope 3g or 4g and your tests and also do let me know if you could optimize either one of these solutions and make them faster alright so that's it for this video i appreciate it if you came this far and now we should be able to set up a pagination anywhere you want whether you want to set it up with java mongodb and angular or node.js mysql and no javascript at all on the front ends and i mean actually use one of these more old-fashioned ways like mustache.js or ejs here we can see that i've written this code using ejs well which will then node.js is responsible for creating that html file and then sending that to the user and still we can see that the logic is the same as before we're doing a for loop for our page numbers and also customizing the buttons based on the pagination data that we're going to have on this side of course because we don't have javascript on the front end well then we don't have ajax so we need to change the buttons to be links a tags and therefore we'll need to refresh the page whenever we want to go to another so if user clicks on one of the buttons well the page is going to get refreshed right all right anyway now the point is that you should be able to build a pagination on any system or with any stack that you want also if you have any questions feel free to post them in the comment box below and i'd love to help you if you need it and if you've learned something new here and enjoyed this video please hit the like button and also share this video with other people alright thanks everyone for watching and i'll see you in the next video

Channel: Cododev

Views: 2,228

Keywords:

Id: vP9fOEAlo74

Length: 98min 53sec (5933 seconds)

Published: Thu Dec 17 2020