Designing a clean REST API with Node.js (Express + Mongo)

[Music] hi I'm bill sewer and welcome to another episode of mastering Monday this is a show that happens live every Monday night at 10:00 p.m. and it's where I try to show you how to improve your code and advance your career so today I've got something pretty special planned we're going to be looking at a REST API that was built in node with Express and MongoDB but unlike a lot of videos that are about this kind of thing today's focus is going to be less on how to use the technologies and more on a clean architecture style so what I mean by that is basically how do you divide your architecture or layer your code such that your business logic is isolated from the various frameworks and technologies that you're using and so what I'm going to show you is one way of doing that and I'm hesitant to say that it's like the only way because it's certainly not and or that it's the best way because the term best really depends on your particular situation but I'm sure there's gonna be a lot of interesting tidbits that I can share with you about how this code is structured and some of the trade-offs and decisions that appear within the codebase that I've had to make and so I'm looking forward to sharing that with you you can follow along if you'd like there's a link to the code on github in the video description and also on the chat if you're watching live so you may want to go hop over there put that in a different tab or on a different monitor if you've got a couple and then we can we can look at the code together so let's jump in mm-hmm actually before we look at the before we look at the actual code we're gonna go and look at the end point or the API itself so this is an API that's very very basic I haven't implemented too much functionality it's just an API for contacts and right now I've got a post that allows you to add new contacts and I've got a few get methods that allow you to retrieve all the contacts or a specific contact and a bit of filtering so I'll show you how it works first and then we'll we'll jump into the code and take a look at that so what I've got open here is that well you cannot see me how about we make sure that you can see me how is that gonna work mm-hmm oh I love doing this stuff live okay so did that all right and here I am okay so um hopefully I wasn't a blank screen the whole time and if I was I apologize but hopefully the audio came through so what we're looking at here is a application called postman and with postman you can make HTTP requests to a a server and see the results and so this is the endpoint that I created as I said it allows you to post contacts and get contacts so I've got a contact record here very simple it's got a first name a last name and an email address and if I send it's running obviously on localhost so if I send this message out and all goes well we can see that we've got a return value so it's coming back with a with an object that has a success flag of true and that says created and it shows me the contact that was created and we can see here a new contact ID was added and if I try to run this again what happens is I get an error because I cannot I cannot call insert with the same email address the email address has to be unique so that's one example of one of the business rules that is in this in this service that I've built and so I five vary the email address if I just stick a one in front of here let's say I can I can send again and again we see now it's added the new record and I'm gonna add a third record just for fun with some completely different values so we can see some variation let's say Gmail all right okay so you get the idea about how the how the post works and there's validation in here so if I try to out of contact with without an email address let's say that's gonna error out email address cannot be known okay so we've got some validation we've got some rules going on you know it's a standard post and so the get works similarly so if we just do a get of contacts we're gonna get back the records we've just added and I've also implemented I get for a specific contact so if I just add an ID to the route so I get back the specific contact we can also do a few other things here so we can we can set a max for example so right now this is returning three records if I set a max of two I'm gonna get back only two records I can also do some paging so I'm using the contact ID as a cursor so if I use the contact ID and I say you know get me a maximum of two records after this ID it'll return me the next two records after this ID so if we send there we go the next two records after that ID and I can also go the other way so we can go and get stuff before so if we if we flip this around and put this as before pass that in so now when I run this what should happen is I get back to records the to the two previous records before this ID and so that's just one way of implementing paging so you can see it's a pretty basic little API there's a few rules being enforced so let's jump in and look at the code and see how how that works so here we go there zoom in well I'm gonna zoom in and let's get rid of this so if we look at the structure here basically we've got a with our package.json and then we have a source folder that contains our source code and in here the main entry point is this index s and then we've got a folder for contacts a DB folder and some helpers and we'll dig into each of these in time but let's first look at our index J s so if we come here a little bit of room so you can see we're using Express so we've imported it obviously from the standard Express library we're using body parser just to allow us to receive JSON as our as our body so it'll parse that on its way in and then our endpoints here are very very ordered starting on our endpoints our routes or routes are very very light so we have this all which will catch any requests to contacts and send it to what I call a contacts controller and then we have a specific one forget that takes an ID parameter and then you'll notice that both of these things are being handled by this contacts controller so contacts controller is just under here it's just this function right here and this is a very generic function all it does is it takes the request that is an object that comes from Express that has one that basically represents an HTTP request runs it through something called a DAP request which we'll look at in a moment and then it calls this handle contacts request with the HTTP request and then it sends a response so there's very little business logic here at all and that's because this structure here this req res this is coming from a framework which is Express which we don't control it's an outside bit of code and so we have to be very wary or we should be very wary if we want our applications to last a long time and to be readily changeable we should be very wary of mixing our business logic too much with concepts that come from frameworks so the way I look at it is Express is really there to allow you to send and receive messages over HTTP that's what it's good at and I mean there's other functionality too with middleware and so on but keeping things simple Express lets you send and receive messages over HTTP which is one concern there's another concern which is about interpreting those messages and formulating a response and the way that your API interprets the message and the response that you're going to formulate that's going to be unique to your particular application and so what we want to do is we want to be careful about mixing things that are unique to our application or too much with things that are coming from a framework or another way of saying it is we want to be careful that the custom logic of our application is not overly dependent on a third-party framework we want the framework to be dependent upon our custom logic and so we'll look at a couple of things that sort of demonstrate that so first we create this HTTP request and we do that with this a little helper function called adapt requests so if we look at that function so we can go here and look at adapt request so this is a very simple little function all it does is it just takes this request object which is something that is native to or proprietary to requests and it converts it into a standard object it Maps it into a standard object that has sort of a standard representation of an HTTP request but just the data of an HTTP request so we're not passing through some of the methods that come from Express like record send which actually allows us to send across the wire information instead what we're doing is we're just capturing the data that was in the request so that we can handle and process that so remember I said sending the messages across the wire that's the responsibility of Express interpreting the message and formulating a response that's going to be the responsibility of our app so going back once we've got this generic httprequest object which is just basically you know a bag of data that is coming from the HTTP request we call this handle contacts request function and so handle contacts request is the first thing that's inside of our contacts folder and it actually comes from here this contacts endpoint handler and the contacts endpoint handler is created from here contacts endpoint yes so what contacts endpoint is is this creates it's a factory that creates for us a function that can handle an HTTP request okay so contacts endpoint if you're following along in the code it's a factory that creates a function that can handle an HTTP request okay and the reason that we do it this way is so that we can inject a contact list in here which is what we'll be working with to manipulate the data to add and remove contacts from our list and the other reason that we do it this way is so that we can build something that is completely independent of any kind of framework so this this handle function the logic in here basically its responsibility is to receive a generic HTTP message and create a genera a specific HTTP response object and so if for example the HTTP request happens to be a post it'll go and call post contact with our HTTP request and that's over here and you can see all the logic is in here now to build the response objects okay to make a standard HTTP response and the reason I like doing it this way is because now this module is highly highly testable so it's very easy to test now because I can inject a mock contact list if I need to if it gets used in some of the stuff that I'm testing and also I'm just testing on you know given this object do I get this other object in response so the function receives an object and returns an object and for me that's one of the best patterns in JavaScript is to write functions that receive an object and return an object because they're super super easy to test and they tend to be very flexible and easy to change expand build upon over time and so you can see here we're handling the post and the get and we've got helper functions all throughout here like make HTTP error which will just make an object for the HTTP error but if we focus for a moment on the post so the post will take the contact info from the request body and eventually it will make a contact object and add it to our contact list okay so this contact list is coming from from here so it was injected right and all of the logic about what makes a valid contact lives inside this make contact and all of the logic about how to add that contact into our database lives inside this contact list add method right so you can see that we're we're separating concerns so now we've got one layer or one module that is in charge of sending and receiving HTTP requests and that module is reliant on a framework now we've got another module that is there that understands how to receive and return HTTP request/response type objects right the data of a request then the actual logic our business logic about contacts lives again in a separate module and now we're layering our app and creating a lot of independence so if we look at this make contact this is over here okay so this make contact is again another factory function that's going to return a frozen object that represents our contact but the value of having this make contact function is that it is the place where we can encapsulate all of our business logic so what I mean by that is there are rules about what makes a valid contact in the context of this particular API and we want to put those rules somewhere where it's completely independent of any of the other concerns or frameworks of our app so this is just a plain old JavaScript module there is nothing in here that is reliant upon any kind of third-party framework at all in fact the code in here doesn't even know or care whether it's in the back end or the front end whether it's serving an web type API web service type API or it's just serving a regular web app it doesn't know the difference because it's very generic and focused on or it's specifically focused on the rules about what makes a valid contact and so we've got this validate method right that ensures that we have a first name a last name and an email address and then we validate the name which ensures that the name is at least two characters long and if it's not it's gonna throw a custom exception which is called an invalid property error with a meaningful message similarly we have validate email which is going to validate the email address again using a helper function that tells us what a valid email address is so if you try to put in an invalid email address you'll get another custom exception an invalid property that will tell you that this is an invalid email and then we've got this normalize function that we run everything through and the normalize function what it does is it make sure that the first and last name have an uppercase first character and it also makes sure that the email address the entire email address is in lowercase so that's how we normalize it and you'll notice it's impossible to make an invalid contact object you can't do it because as soon as you try to make it it's gonna run the validate and then Orma lies and if you attempt to make a contact with some invalid state or some invalid data what's gonna end up happening is you're going to get an exception so it's impossible or it's gonna throw an error so it's impossible to make an invalid contact and that's a really key point about business objects in general when you create an object that is meant to represent some concept within the domain of your application it should be impossible to create that object or put that object in an invalid state and we further reinforce that by freezing the object that we returned right and then so when we freeze object that freeze if you're not familiar with it in JavaScript what it's going to do is it's going to make it so that the properties of the object that are returned can't be modified now one small note it does a shallow freeze so if you have internal properties or if you have sub properties of a contact that are of a reference type so if you have an array or another object you need to freeze that as well so that you deep freeze it so that you make sure that the outside world nothing in the outside world can ever take your business object and put it into an invalid state so that's just a key lesson and again notice none of this code knows anything about Express or HTTP requests and responses nor does it know anything by the way about any kind of the database it just knows what a valid contact is and how to enforce the rules about contact all right and so then we need to look at our contact list so our contact list it's another factory function as you can probably tell I'm a huge fan of factory functions and I'll try and throw in the description a link to an article I wrote called well about a pattern and JavaScript called an ice factory and that's what these are these are ice factories so basically it's a factory function that returns a frozen object so here we're making a contact list and so this is in contact list is in the contacts folder if you're following along with the code on github and the contact list is there to basically encapsulate interactions with the database and again by using a factory function what we can do is we can inject an object that represents the database or our an object to help us interact with the database so that means again from a testability standpoint it's very easy to test this now because if you don't want to have live a live connection to the database or be doing live database calls within your test you can easily mock this out because it's being injected so it's just dependency injection right and so when we look at some of the functions within here you have sort of what you would expect out of a list so you've got an ad which allows you to add a contact find by email find by ID get items which is going to return a bunch of items I remove and replace I don't think I actually implemented here no I think I implemented remove I didn't work implement replace and update if you want go to github pull down the code and see how you might implement that yourself it's a little bit of a fun take-home task if you'd like but so this contact list is again interacting with the database so that it's hiding the details of the database from the rest of our application so when our contacts endpoint code goes and calls for example contact list add it has no idea what database is behind this so yeah right now it's MongoDB but it could be anything it could be a sequel database or an Oracle database it's a it doesn't matter which database is underneath we've actually completely hidden that detail that becomes sort of an annoying detail that's hidden away and the reason that it's really important that we return object dot freeze with this stuff is because we don't want code from outside of we don't want outside code that's consuming this module to be able to change anything so we don't want them for example we don't want any outside code to ever be able to reassign to add or to find by email because that would put our object in an invalid State and so with with a factory function like this when you're returning an object you're making sure that that object remains in a valid state as it gets passed around and like I said I'll find a link to the article and I'll put it in the description so that you can read more I went into some depth about why I like this pattern and why I prefer it to let's say classes in JavaScript but it's not to say that this is the only way it's just trying to reinforce the underlying principle that you shouldn't ever allow your objects to be put in an invalid state and one of the ways to protect against that is to make it very difficult for code outside of the object to directly mutate any of its properties okay and so yeah so this contact list you could say is an implementation of something called a repository pattern but I didn't call it a contact repo and I do that very on purpose so I try to shy away from naming things in my applications after a certain design patterns and the reason that I don't like to name things after design patterns is because in my experience what happens is it leads to the wrong kind of arguments within a team so as if I were to name this contact repo or make make contact repo and then someone on the team guaranteed would tell me that well no actually technically a repository is implemented that way and there would be a large theoretical discussion about what a repository is or isn't and that's the wrong type of thing to be arguing about when you were building an application and working with a team what you want to be arguing about or at least not arguing but having discussions about is whether or not the particular design of your application is serving the needs of the application not just today but in the future so to go off on a little bit of a tangent good architecture is really about reducing the cost of future change so if a system has a good architecture it would be cheaper and less risky to add features over time when it system has bad architecture or bad design it is increasingly risky and more expensive to add features over time okay and so that's that's one of the reasons or one of the main reasons why we kind of make these types of decisions is by ensuring that our business logic that the things that are likely to change because the customer wants to add a new feature or changes of process are isolated from the things that are likely to change because there's going to be some kind of a technology change okay so we've hidden we've hidden our database interactions behind this list and let's say for example right now I'm using the the MongoDB adapter so this database it's being injected here so in contacts at the entry point of contacts it's being injected from this make DB function which comes in this database folder here okay and that's where we're creating a connection to MongoDB now obviously I've hard coded you know my localhost right now in a real application you'd be using process variables and you'd have different environments etcetera etc but I just I live this up sort of quickly as a demo so I'm yeah so we've got the details of creating a database connection isolated into this into this function that is making our database and it's injected here at the entry point of our contacts folder right so where we make the database and then we inject it here into our contact list so again this allows us if we wanted to we could dependency inject a mock database so that we could test this code independently if we needed to test code paths that spoke to the database because there's some branching logic in there that were we're interested in so I guess the other thing I'll say about this is I happen to be using you'll notice here I'm going to be using the driver that comes from itself so this is the nodejs driver that's given to us by DB that tends to be my preference when I'm interacting with the database is just to use the driver that's given to you by the vendor but I know that a lot of people like to use object data mappers or object relational mapper and a popular one for is something like Mongoose and so if I were to use Mongoose what I would do is rather than here in my in my contact list instead of injecting this database which gives me access to the driver and the functions of the driver I would inject a contact model so a mongoose model for a contact and then i would interact with the database using the the contact model that's built with mongoose but a very important point is I would never use the Mongoose contact model as my business object so it still always have this contact object as a plain old JavaScript object and that is where I'm gonna put the majority of my business rules about a contact because again if I rely on the Mongoose model to implement my business rules for a contact now my business logic the things that are likely to change because the customer the client needs a new feature or has changed the way they work is tightly coupled to some framework that is going to change for a different reason which is you know mongoose mongoose is gonna come out with a new version or we're gonna need to move to a different mapper because there's no longer satisfies our needs because either we've moved to a different database and so you might think to yourself well hold on a minute in my applications or any applications I've worked with it's really rare that we change mappers or that we change databases and it used to be that that almost never happened now when it with my clients there is more of I shouldn't say that it used to be that it almost never happened what what it was was that it there was almost never a compelling reason to do that but now with the way things work more and more with my clients and finding their there are reasons actually to make these kind of changes so a very common one that's quite relevant to the idea of a contact is you might start out building your application where your contacts are in a proprietary in your own database and then you're gonna move to a model where the your client has invested in some kind of a CRM system customer relationship management system and now the contacts are inside of the CRM and it would be handy from an integration standpoint for your API to repoint itself to the CRM as opposed to pointing itself to a custom database and that's a lot more difficult to do if you've tightly coupled things with your database or with with Mongoose and what I mean by tightly coupled obviously the the list here is coupled to but it doesn't contain any logic about what makes a valid contact and even if I was using Mongoose I would keep it light in terms of what makes a valid contact that logic would not be in mongst it would remain in my custom contact business object which is here okay and so hopefully that's insightful and helpful I'd love to I'd love to take some questions I see there are a few people on so let me hop in there and see what people are saying and I love being okay so well um let's see if we can do can we do this haha all right so all right so I don't know if people can see the other if it's if it's worthwhile having this like this it might it might go well I mean I think in the replay you'll have it there but so yeah so there are questions about returning frozen objects so I I think I covered that but just in case so the idea is the reason that we are returning frozen objects is that your data well actually as T John has said there so basically when you use object freeze what it does is it does what's called a shallow freeze of your object so that the properties cannot be changed from the outside and so so yeah so so the properties would cannot be changed from the outside the only thing to remember is that it's a shallow freeze so if you have internal reference objects then what's gonna happen is you need to freeze those as well um in fact I can let's see can we be brave let's see if we can be brave and okay let's see if we can be brave and actually go to and let's did it today okay haha so let's see if I can make this can I make this bigger yes hopefully you guys can see this all right so let's make let's make an object and equals object Friesen sorry for the food boring but I'm old school I come from the world where we called everything fubar okay so so f is now frozen object so if I try to go F dot foo equals you know bass okay so because I'm running in a particular version of JavaScript and note I had to make sure to specify you strict so that object dot freeze would would not fail silently and so this is something that it basically depends on the version of JavaScript you're using and how you've set up your build so I've got babel seven running and I've got things configured in my project so that that it automatically enforces the strict rules so you would actually get the error um what's happening by the way if I don't use strict I'm pretty sure we're living on the edge we're coding or coding a lot so what happens is if I don't use strict and I go console.log see so it it actually logs bars so this this didn't work but the problem with this particular environment here on this rebel is that by not using strict it fails silently behind the scenes so this this line is actually not working as you can see I'm proving that down here so hopefully that's that's helpful and that makes sense and then what I was saying about what I was saying about shallow freezes so if I have something in here like be okay so if I have an internal object this internal object I can change it like I can change f dot o dot eight or something so I have to I would have to object I'll freeze this as well so hopefully that's that's clear and that's making a little bit of sense so if you switch to CRM as your data store would you still use the same database object to represent API calls to the CRM system yeah so um if I switched to a CRM as the data store what would happen is the only thing that would change in my code and this is the nice thing about working this way is that the contact list instead of receiving a database it would receive basically the endpoint or some some library that can interface with my CRM code then calls to my contact list would basically delegate and call out to the CRM so that to the rest of this code it has no idea that now these contacts are being stored and retrieved from a CRM our endpoint code actually doesn't change at all right so it still receives an HTTP request delegates out to the contact list and based on the response it creates an HTTP response object and so that's part of the value of working this way is you really you isolate yourself from those kinds of changes okay so oh hi my brothers on the chat hello bro and so okay so the chat will be on the replay apparently so hopefully this was instructive and unhelpful please don't forget to subscribe and hit the notification bell and if you have any other questions if you're watching this video now or in the future and the replay please feel free to put them in the comments and I'll be happy to answer them all so that's it that's a mastering Monday for tonight thank you so much for joining me have a great night I hope you enjoyed it and we'll see you next Monday take care bye bye

Channel: Dev Mastery

Views: 50,020

Rating: 4.9386582 out of 5

Keywords: Node, Express, Mongo, MongoDB, ExpressJS, Node.js, NodeJS, express.js, separation of concerns, seperation of concerns, designing for change, javascript, JS, Architecture, API, CRUD, JSON, rest api, restful api, express api, nodejs express, express rest, dev mastery, devmastery, Bill Sourour, devmastery.com, rest api tutorial, express rest api, node api tutorial, rest tutorial, restful api tutorial

Id: fy6-LSE_zjI

Length: 38min 31sec (2311 seconds)

Published: Mon Jan 21 2019