Building microservices with Node.js - part 1

okay hi everyone hopefully everyone can hear me uh yeah well i should say g'day g'day from australia you know thanks for for joining us um you know whatever time it is for you it's morning for me here 8am my morning coffee is just kicking in so so bear with me um i'd like to start actually with uh a tradition that we have here in australia i'd like to start by acknowledging the traditional owners of the land from which i'm broadcasting and i want to pay my respects to elders past and present my name is ashley davis and i'm the author of bootstrapping microservices as matt was saying i'm super happy to be here today speaking to you talking about micro services and node.js i'm just going to share my screen so you can see my slides and my contact details and all that other good stuff and i'm just going to try and get my screen arranged so that i can actually see the the chat and see the participant list because i need to keep an eye on people and make sure i can make sure i can check people in so let me just get set up okay that looks good cool so um you can see my contact details on every slide um please feel free to note them down please feel free to reach out to me and talk to me um you know i'm pretty active on twitter you've got my my public email address there as well and you can see my details on linkedin as well okay so this talk today it's part of a series of three that i'm doing um they're going to be fairly short talks or today today's going to be a little bit longer so the the parts two and three of this series are probably going to be a little bit shorter than today we'll see and these talks are coming up over the next month so depending on where you're watching this on dc javascript or coding boot camp or i think there was another meetup i've forgotten which one just just check the meetup page for that when the next talk is coming up it's not going to be too far off today is part one of the series and i'm talking about i'm going to be giving an introduction to microservices and i'm going to show you how you know how simple it can be to build a basic micro service with node.js in part 2 of the talk series i'm going to amp it up basically from a single micro service to multiple micro services and we're going to talk about testing micro services and developing micro services and actually running uh like you know a multi micro service application on your desktop computer using docker compose and then in the final part of the series i'm going to be talking about how to deploy your micro services to production using kubernetes so i've just got i've just got to check for you know people waiting again every few moments so if you if you see me stop for a moment i'm just letting people in i'll start with a bit about myself um matt did a nice little intro but i'll hopefully expand on that a little bit more i'm the cto of sortal and we have a product uh that helps you organize your digital assets using the magic of machine learning i'm also the developer of dataforge notebook it's like a jupiter notebook but the typescript and javascript and you can prototype your code with live visualizations and then later export your code to use in production as i said also i'm the author of our two books formatting data wrangling with javascript which is basically a book that you know shows you that if you're a javascript developer um you probably don't need python to work with data um like most of what you need to work with data is already in javascript you just need to learn how to do it and that's what's in my book data wrangling with javascript and of course there's my book bootstrapping microservices which has just gone to print like in the last couple of months so i'm very very proud and very happy to kind of have this out um there's a link on screen if you want to have a look at the the webpage for it you can read the first chapter free on through that link and uh and i've got uh some books to give away at the end so please make it to the end of the talk and i'm going to do a random giveaway and uh and for the rest of you who don't get a random giveaway i will have a discount code for you as well so you can get the book for a little bit cheaper hey in a sec i've just got a couple of people to let in so um this this book bootstrapping microservices it's a practical and project-based guide to building applications with microservices you know i start very simply in the book by showing you how to build you know one microservice service and package it and publish it and get it ready for deployment to production and then it amps up to like multiple micro services using docker compo compose and eventually you get your micro service running in production on kubernetes so it takes you through the whole process from from literally nothing um to having a a working micro services application in production using an interesting um example application called flix tube which is uh like a an imaginary video streaming service that you're creating but the recipes the tools the technologies everything i cover in the book is basically um you know you can use that to build almost any micro services application so it's very general purpose skills very transferable skills that you're learning so um yeah just just please hang around to the end of the talk if you want to get a chance to to win a free copy of this book and uh and the content you'll see in the talk today is very much similar to what's in the early chapters of the book and i in you know the second and third parts of this talk that are coming up later in the month i'll be talking about some more advanced aspects that come up later in the book so if you like what you see in this talk if you like what you see in my other talks um you will find more of that uh in the book and let's get on to it so i'm going to start with an intro to micro services and then i'll show you how to build a simple micro service in node.js now of course you know ultimately i'm going to be using docker to package and publish my microservice ready for deployment to production you know docker it's you must have heard of it it's that ubiquitous tool that's really changed it's really revolutionized our industry indeed and docker means that we're not limited to any particular tech stack so we don't have to use node.js or javascript to write microservices you know that's not what i'm telling you um you can use any language to build microservices right so you know feel free to use c-sharp python java ruby you could even use c plus plus if you're if you're a sucker for punishment so you can you can build microservices in any tech stack using docker but my preference is to build microservices with node.js and javascript and i'm currently using node.js to run microservices in production for my startup and also for for multiple contracts that i've done over the last few years you know so naturally you know given my background naturally when i present examples and all the examples in my book are in javascript and node.js when i present examples that's what i use node.js and javascript before we kind of get into what a micro services application is let's look at the alternative let's look at uh the monolith and that's that's what came before microservices basically it's called monolithic and i said i'm just going to let someone in uh it's called monolithic because um it is literally an entire application running in a single process it's it's like a whole it's a monolith uh you know traditionally this is the most common approach to development and it's actually still in common use so you will see monoliths being used by companies um you know it's not something that's gone away it's not something that will be completely replaced by microservices you know monoliths are good for what they're good for and they have their place in our industry now the typical wisdom is that building a monolith um it's easier to build than a micro services application and that's true to some extent but i also want to challenge that notion with modern tools a good automation you know a commitment to a simple architecture i think micro services are easier and cheaper to build than ever before so the choice of whether to go micro services or monolith you know it's something you need to consider carefully it's not as simple as it used to be in a sec i've lost the control of my slides ah there we go so what's so wrong with the monolith that we'd like to replace it with micro services now you know there are good reasons to start with a monolith you know it can be an easier starting point than microservices that that's for sure it requires less technical skills uh it could be a great starting point for prototyping or testing the validity of a new business a monolith is a great option for a throwaway prototype and also if you're if you're building an application that's always going to be small like you don't plan it to grow you don't want to evolve it or extend it over its lifetime then a monolith can be the right choice from the start so you know there's a place for monoliths i'm not saying throw out the monolith but you know there are problems with the monolith that might work that will make you want to think carefully about starting with it now a monolith might start very simple but ultimately like any modern application whether it's a monolith or micro services it is destined to become complex you can't avoid that you know you can't hide from complexity by creating a monolith so don't use that as your excuse for um choosing monolith over microservices monoliths are difficult to scale you know both for scaling your dev team and scaling the performance of the application they're difficult to restructure and that's something that gets worse over time they have a large deployment risk so that is to say that when you update your monolith when you release new code if you break the code you break your entire monolith and your customers have nothing so that's that's that's the size of your deployment risk when you're when you're uh releasing a monolith monoliths have a single large attack surface so if an attack if an attacker breaks through or a hack or whatever breaks through that single line of defense you have they are on the inside of your application causing all sorts of havoc now ultimately though um the real problem is is that it can be very difficult to move from monolith to microservices you know if you've started with the monolith and you later decide that it's too limiting and you want to kind of convert it to microservices so once your business and your customers are dependent on your monolith you have to move very carefully in order to restructure it to microservices so um the organization is going to need to update its technical skills the developers are going to need you know a fair bit of discipline and commitment and be able to follow a long-term strategy because it can take you know if you're going to take a big monolith and refactor it to micro services that's that could take some time that could take years even you're also going to need a great testing regime so you know if you already have a monolith and you want to change it to microservices don't even think about doing that unless you've got good testing in place um you know you just need that you like if you're making changes to it and you can't test that you're you're in trouble so you see once you once you're dependent on a monolith it can be extremely difficult to get away from it now of course you know the other option you have instead of making a monolith is to make micro services now what is a micro service you know i'll i'll tell you more about the benefits of microservices in the moment in a moment but let's just understand what a micro service is first just in case you uh this is a concept that's new to you a microsoft microsoft a micro service is a small independent software process with an independent deployment schedule now why is it important that a microservice is small well having small services minimizes our deployment risk having small services allows us to make small changes small iterative incremental and frequent changes and small changes can be more easily tested as well so we can use this to increase our pace of development at the same time we're mitigating our deployment risk why is it important for the process to be independent well it's important for fault isolation i'm going to talk more about that in a moment why is it important for the deployment to be independent well it's so that separate services can have their own release schedule it means that faster changing services can have more frequent updates and it allows uh it allows each team to manage the release cycle for their own micro services and this is something that facilitates independent and scalable teams okay so how how small should a micro service be now opinions will vary on this but i have my own metric um i would say that a micro service should be small enough that you can rewrite it in a couple of days or at most a couple of weeks okay so you know if if it takes you months to rewrite a micro service it's not that small it's not that micro is it um it's more like a macro service or you know something bigger we don't really have a name for those so that brings us to the micro services application and a micro services application is a distributed application composed of many tiny software processors that that's the microservices and the microservices work together to build the features of the application the processors typically communicate via the network and a microservices application is just a special case of a distributed application right you know these things have been around for years and uh and recently they've sort of evolved into this new new architecture called micro services where you know you take these distributed processes and you shrink them down to the micro level so the real difference between monolith and micro services is that the monolith is just one large service really you can think of it as one large service where micro services are many many small services working together to achieve a whole a whole application so a question you might have is why are microservices so popular now because like i said distributed applications have been around for a long time they've been around for decades you know so this isn't this isn't really new um but you know what why have microservices suddenly hit the mainstream well firstly it's because now we have you know modern automatable tools um starting with docker about 10 years ago or a little bit more a little bit less than 10 years ago um that revolutionized our industry and you know since then we've just had like we're spoiled for choice in terms of good tools to use and you know the the important fact here is that the tools are automatable automation is really important um because when we have many small services we're naturally going to end up with a lot of them and we can only manage such a large amount of services if we can lean on automation so automation is crucial especially for small teams who want to scale themselves up to manage large applications also now we have access to cheap and convenient cloud infrastructure that we didn't that we didn't used to have um it used to be expensive to run microservices you know especially if you had to create a virtual machine uh to run each microservice like what i did when i first started building microservices you know that's not a cheap way to micro run micro services but now we have you know container services and you know software like kubernetes that makes it a lot cheaper a lot more cost effective to run microservices and the final thing is the devops and the devops revolution and the infrastructure is code um paradigm i'm sure you guys uh hopefully have heard of this devops you know developers are now managing their own operations responsible for the complete life cycle of the of the services that they work on um infrastructure as code has turned application deployment into a coding task and so you know this is what allows um you know small teams to kind of own their own micro services all the way from concept development testing to production they own the whole you know a developer or a development team can own that whole life cycle of that of that of that process or that micro service this is only possible because the services are small you know the services have to be small enough that a single developer or a small team can manage them you can't imagine like if it's a big established monolith you know you need like a big team of developers to manage that but if you've got micro services you can have you know individual developers or small teams of developers managing single micro services or potentially groups of micro services so um small services uh micro services um they're basically able to be managed by separate teams and this is leading to more scalable organizational structures for companies that are using microservices so in summary microservices are popular now because we've just reached that point that critical mass point where in terms of cost ease of use ease of access and automatability microservices have become very accessible to the mainstream community or the mainstream development community at least and so naturally we want to use micro services for all the benefits that they have to offer and so that brings me to the benefits of micro services now something some people think that micro services make things more complex and that can be true you know i can see how that can be true but what i believe is that any application can grow complex it doesn't matter how you build it you know eventually it's going to become complex you know and we shouldn't fear complexity i think that our primary jobs as developers is to manage the complexity that we create in order to solve problems so as developers as good developers you know what we should be looking for are better ways to manage complexity i'm just letting another person in micro services are an architectural style that helps us manage complexity like that's what it is microservices is just just think of it as a tool to manage complexity it's not it's not something that's just put in there to cause complexity you know this can actively help us manage the complexity in our application so even though a microservices application will become complex and by the way a micro services application doesn't have to start complex you know it can start simple i assure you um despite what people say i show you step by step how to start simply with micro in my book bootstrapping microservices you know it can be done you can start a microservices application from a very simple starting point and then every step after that every iteration of coding can be simple but so you know a micro services application like any application it will go grow complex over time but the micro services themselves you know the architectural pattern that we're using to build the application is the means by which to manage the complexity and what i mean is that if you look at any established microservice application that's big you know it's going to look very complicated there's going to be a lot of moving parts but if you zoom in so you've got the biggest the whole application if you zoom in to the level of any individual micro service it should be simple um it should be understandable you should be able to read the code look at the interactions with other microservices and understand what's going on and that's that's what microservices are supposed to be you know they're supposed to be small uh simple and understandable uh you know like like that's what it needs to be to earn the name micro service it's got to be small it's got to be micro so that means that you know any any given body of code that makes up a micro service you should be able to read that and understand it fairly quickly and and then zoom out again and look at the entire application it can seem complex it can seem overwhelming but then zoom in on the microservice that you want to work on and you can you know it's it's simple it's understandable it's easy to test you know it's it micro services help us manage and compartmentalize the complexity you know application so microservices with micro services we get a fine grain control over the application that means we get tight control over deployment security scalability team structure and so on you know basically everything is is broken down into these small parts and instead of having this one massive monolithic thing that you know we've only got one thing that we can tune we can fine-tune all the separate parts of our application independently microservices allow us to compartmentalize the components in our application and it's just like how bulkheads work in a ship in a boat to divide the boat into compartments so you know if the boat springs a leak the compartment the bulkheads close the compartments close up and we isolate uh we isolate a failure or a problem into a particular compartment where it occurred and the same thing is true of microservices should any given microservice fail the failure can be limited to that one microservice and it doesn't have to bring down any other services it doesn't have to bring down the application and it doesn't have to cause disruption to our customers which is the most important thing you know the fact that any given service is small and we can roll it out independently means that our deployment risk is low so when we deploy a micro service we're risking just breaking like a tiny part of our bigger application if we're deploying a monolith on the other hand you know we're deploying the whole thing at once and we risk breaking the whole thing at once which isn't which isn't good for anyone microservices support a scalable team structure so the application can be split into many separate code repositories that are managed by separate teams and are independent of each other and so because those teams are independent and they're working they can work in parallel and they're not treading on each other's toes you know this allows us to scale up the size of our application and effectively distribute the workload across our growing company now this is the most famous aspect of microservices um they are they are scalable in a performance sense um when you are deploying a monolith you only have the one thing that you can scale and it's not very focused um the motor there might only be a small part of that monolith that you want to scale but you're forced to scale the whole thing and so it's not very cost effective it can get very expensive with microservices though you have more options for scaling you know the the application is already split up into these components and you can choose to scale only those parts of the application that have a performance problem so our scaling can be very focused when we're working with micro services and it allows us a much tighter control over our running costs elastic scaling also works better with microservices because microservices are smaller because they're smaller basically we can spin them up quicker to meet growing demand and we can spin them down when that demand is gone so again we can really focus our scaling and it can be much more cost effective now this last thing on my list here i want to mention uh it might not be as obvious to it to people who are listening in we can use micro services to isolate and control our technical debt so instead of having one huge monolith size ball of mud we can divide it up to separate islands maybe separate islands of mud um you know that are independent and distinct from each other so instead of having like one big pile of spaghetti we get it all nicely separated out at the application level and i actually think this makes microservices really good for prototyping and building mvps because you don't have to worry so much about the quality of the code because you can easily and iteratively replace microservices over time with better code and um yeah so so it makes microservices great for for building mvps for startups and i you know i recommend microservices to startups um you know you know giving depending on their situation and what they're trying to achieve of course it's not it's not right for everyone but micro services i do think they're really good for prototyping and mvps and and that might be a controversial uh opinion but that's that's what i believe and i've been using microservices for startups for a few years now so um you know i i've been through it now the ability to replace microservices over time and improve our code this is called designing for disposability so we're using micro services um where we're basically intending to replace our micro services over time to evolve and improve our application we're effectively building in an ability to change and that's really important in in modern times when you know things are changing quickly and any modern business has to kind of change quickly and adapt to survive or be competitive micro services can help us achieve that so um after all this you know let's get to know js um you know what why use node.js for microservices i mentioned already that we can you know we're going to be using docker so we can use any tech stack we can use any language to build microservices it doesn't have to be node.js or javascript but to start with you know even though javascript i you know i don't think it's the world's best programming language um you know that that would be hard to say but i do i do truly believe that javascript is the world's most practical language you can do almost anything in javascript using node.js means you can go truly full stack it means you can go anywhere in your check stack anywhere in your application without changing languages now personally i use javascript well actually actually mostly i use typescript now um but i use it for front ends backhands mobile apps and more like i literally use j at this point in my career i've been a developer for over 20 years i've used many different languages at this point like i'm full-time i i guess i would say i'm a full-time typescript coder now i never thought i never thought i'd get to that point but um i i love it i love being able to go in anywhere in the application and and just be comfortable just be comfortable with the language and and the way it works and everything that's there node.js for its part it was made for building distributed applications you may not realize this but the name node that indicates that node is intended to be a node in a network orientated application node.js is network orientated from the start so it includes everything you need to build network orientated applications and the fact that node.js was built for this kind of thing means that building microservices with node.js it it's particularly easy i'm going to give you a live demo in a moment so hold on you will see some coding in a moment and uh passing data between services uh or node.js microservices is particularly easy and that's because the de facto standard for communication between services on the internet is json and that's something that was born from javascript and indeed it is built into javascript uh yeah i think there's plenty of other benefits to working with javascript that i'm not going to worry about today um i'm sure you guys are here probably already interested in javascript and know something about it so i'm not going to i'm not going to go into bore you with details on that um but uh here we go sorry this last one here um i guess you've probably already noticed that node.js has a huge community you know there are lots of tools lots of examples lots of libraries tutorials plenty of books you know plenty of resources for you to learn and that's important you know you're not you're not working on something that's obscure or small you're working on something that's big popular and has a lot of people uh you know there's a lot of people that you can lean on if you need support if you need help reach out ask the community you know they're very willing to help and you know guide you through the process of using javascript and using node.js so just before we get on to the demo um i'll just quickly cover the the basic needs of a micro service a typical micro service needs a way to communicate with other micro services and possibly also the outside world it also needs a way to store persistent data you know typically we'll use a database or we'll need to store some files and some sort of you know file storage repository um today we're going to look at how to make a simple micro service we're going to give it the most basic communication facilities in the next talk in this series of talk we're going to explore some more advanced uh communication technologies but today we're just going to keep it simple and we're not going to cover storage today either so we're going to look at databases and file storage probably in talk 2 or maybe in talk three depending on on the timing of those talks so who's ready to look at some code uh if you wouldn't mind just start commence your prayers to the demo gods that this all works out so i'm just going to switch over to my command prompt here just give me a moment to get my myself set up okay so um what are we doing here so i'm running no js version 14.16.0 now you can install node.js for yourself if you if you haven't already go to the node.js website nodejs.org you can download the latest version from here i recommend if you if you if you want to run multiple versions of node.js i recommend using nvm so google nvm that allows you to install and manage multiple separate versions of node.js on the one computer it's really awesome okay so i'm going to build a really basic microservice i'm actually working with windows 10 here but i've got git for windows installed which gives me all the all the linux commands so i'm just going to make a new directory for my demo today and i i'm going to create a new a new node.js project so i'm creating my node.js microservice from scratch i just want to show you how simple it can be to go all the way through this process it's going to be pretty basic it's going to do basically nothing except hello world but you know it will be a good starting point for a micro service so i can start a new node.js project using npm init if i add the dash y command i get it filling out all the defaults for me so this is going to create a package file for me and a package file is basically you can see it gets printed out here it's the metadata for the node.js project and it records the dependencies and stuff like that so you can see that i've uh this is what this is what an empty node.js project looks like it's just got a package file okay i am going to open visual studio code from this directory and i'm going to create a code file so empty project we just got the package.json you know there's nothing much in here just all the details of the project you can see here that the main file of the of the project is is automatically set to index.js so index.js is basically the conventional name for the main script file in a node.js project so let's create that let's say new file index.js and let's just start with hello world nothing too exciting yet but we're going to build up we're going to build up to something more more complicated so i saved that i switched back to my terminal i can run that now i can run node index.js and it prints hello world so you know that that's that's the first stepping stone you know creating an empty node.js project that just does hello world now um at this point um i could create a git repository like it would be pretty standard practice that if i was actually building a production microservice i'd create a git repository now and you know potentially create one uh like a hosted repository on on github or bitbucket or something like that and push it up there let's not worry about git today let's not worry about git let's just pretend that you know i would be using that if this was a proper development process but what i'm going to do is i'm going to make a start command in my package.json so this is an npm conv uh like a yeah an npm convention really but it's it's an it's a node.js convention as well where we where we create a start command that tells it how to start our uh application and so now what we can do is you can just run instead of running what it did before node index now we can run npm start so this is the conventional way to start a node.js application is to run npm start and then it's configured in package.json to run you know whatever your main file happens to be whatever that happens to be you include that there and you could possibly also include you know configuration of environment variables here or command line parameters that you need or whatever now what we need to to build a micro service or you know the basic functionality we need is to have communication with other micro services so we need a way to accept messages or commands from other micro services and we need a way to send messages or commands out to other micro services now as i said before everything you need is included in node.js you don't actually need to install any dependencies or any other packages or libraries or anything like that but that said you know it can be a bit more complicated a bit more nuts and bolts and boilerplate code and all that um to configure the node.js standard library to do what you want so so typically we're going to install you know some uh projects and some npm packages to help us um to move faster basically to make our lives a little bit easier and typically i use express to start with so that is a rest api or a web server framework it's one of many there are many different uh you know there are many different packages you could use to do this another one i've tried is happy but express is the de facto standard for building rest apis in javascript so uh it's a good starting point and it's mostly what i've what i've used in production and i think it's pretty good and like i said you don't need express you can do everything you need to do with the node.js standard library but something like express just makes it that little bit easier right so that's uh that's how expresses basically how we're going to be able to accept you know incoming commands incoming messages incoming queries from other services and the outside world and uh the other thing we need is a way to kind of send these messages out which i'm going to use axios for that and again you can do all this with node.js but axios just makes it a little bit easier to send outgoing requests so i'm installing both of those things um you know they're coming down from the npm repository right now as we speak and uh that was pretty quick that's done already so if we go back to visual studio code and we look at we look at the package file here we can see in our dependencies now we've got axios and express installed they get extra they get installed into the node modules directory but you can see we have a lot of stuff in the node modules directory now and that's because um that that's because dependencies of dependencies as well get installed so express and axios actually depend and depend on a lot of the other packages themselves and so it automatically brings all that in so that's just under the node modules directory there now what i want to do at this point is is put together basically a basic express based micro service and the easiest way to do that is go to the hello world example on the express website which i've got here already so the express website has this great getting started example you know great great documentation generally here it's well worth reading through i'm just going to copy that and i'm going to i'm going to dunk it in my my index file here now there's many different ways you could i mean you could easily write this from scratch it wouldn't take much effort to write this code from scratch or typically what i would do typically i write typescript microservices so i have a typescript template on my github please feel free to have a look at my github um ashley davis look the typescript template um that's a generic no js typescript project so it's a good starting point normally if i want to start a new typescript project i just i just clone that and create a new repo out of it and start work on a new project from the template but i'm just starting i'm just going to start from this uh simple node.js hello world example from their web page it simply prints hello world when you hit the web server so let's test that so i can run this using npm start and it and it puts the information here so you can see that it's console logging that here that's where that comes from this bit here it's console logging um you know the link to click on basically and in windows terminal you can just click on that link and so we've got our little website now running that says hello world so not too exciting yet but we're getting there we're getting there um one thing i like to do at this point is get set up for live reload so with live reload that means i can change code and the microservice will just automatically restart by itself typically i do that by installing i install it as a dev dependency not a production dependency that's important i install nodemon so node 1 is kind of like a drop in replacement for node in a way in that you can run your script file with nodemon but when any when any any of your javascript files change it's automatically going to reload your web server or your microservice or whatever it is and i can run it just instead of running node index.js i can just edit that and and run node one instead but actually because i'm running it as a package from the current directory i have to use the npx command to kind of to run that but that so that's that's running now it's running like it was before except you can see that you know if i change the code here i'll get i'll get my code and my terminal side by side i could change this to hello microservice and that automatically restarts the code on the left here so it's a great way of working a really streamlined fast and efficient way of working it works great with tdd and automated testing because you can you can change code and have your tests automatically re-run you know while you're coding it's such a powerful thing to be able to do now what i want to do is get a script set up to run that as well so i typically have my own convention to do this um called start dev this isn't a node.js convention it's just my own conventional name for that and i usually just make it run uh nodemon index.js so you know no matter no matter what pro basically no matter what microservice or node.js project indeed that i'm in to run it i either run npm start i don't have to remember anything or that's that's like running in production effectively if i want to run it um in development sorry it's it's npm run start dev and so that's what's going to run nodemon now what's next okay so how do we configure a micro service we need a way to configure this we need a way to kind of say that you know i might want to roll out a few different copies of this or you know i might want to roll this out into different clusters and i want to configure it differently and there's a bunch of ways you could configure a node.js project you could configure it with configuration files you could configure it with command line parameters but typically the most common way of configuring uh a microservice or a container as it were is through environment variables so i've got an example here let's say we want to configure the port so this is running on port 3000 but we might want to run it on port 80. typically i'll run my micro services on port 80 which is the standard the default for http so we can we can basically read an environment variable called port now we have to check whether that's undefined or not whether it's actually being supplied if i can spell properly undefined and and if it's not undefined you know we can we can parse that as an int so let's just copy this into there so we're parsing so environment variables always come in as a string so if we want them to be in it we have to parse it as an int and what i'm saying here is basically if the port environment variable is not undefined then we parse it as an and use that as the port otherwise we default to 3 000. you know 3000 is a pretty good default it's like a you know a conventional port number for you know a web server running in development it's going to save that my it's automatically restarted over here you can see that it's automatically gone to port 5000 as well and the reason for that is because i had previously set environment variable to 5000 but i can set that to anything right i'm on windows so i would set i would use the set command and set my port number to say four thousand um if you're on linux or mac os you'll use the export command instead but it's the same it's the same thing basically that's not gonna work on windows so i'm not gonna do that so i've set that port number to 4000. if i restart now you'll see that my thing is available on 4000 now so we can configure this like i can configure this uh externally so if this was built to a docker image i could deploy this as a container to a kubernetes cluster and i could tell it which port to run on and i could configure anything you know anything that needs to be configured with this microservice can be done through environment variables and it's especially in this case it's optional so in this case i'm saying it's an optional to pass in the port otherwise you get a default but often it won't be optional like you imagine if you've got a micro service that talks to some you know external service like say dropbox and you need to kind of provide some authentication so that it can authenticate with dropbox and you know whatever however you integrate with that whatever you want to do with that but you need to pass in those credentials for dropbox as environment variables you don't want to hard code passwords and stuff like that into your code that's a really bad idea so you always want to pass you know sensitive secret stuff in through environment variables and it's nice um to provide a good error message as well if that doesn't exist so if you if we were saying if we needed to have um [Music] so the dropbox credential passed in here right we're gonna we're gonna authenticate with dropbox and you know use it services somehow but if that was um let's say that was equal to undefined let's just throw an extra error right you know you miss the environment you know what i mean like you want to provide a good error message here that says that you did not include that environment or that thing that you were supposed to configure so it's just nice that you know when you when you're rolling these things out to production if you have forgotten to include the configuration it's gonna it's gonna throw an error and die and and you can go and look at the error log and find out like what was missing and then fix it so that that's just a brief look at how to configure a microservice now we're going to look at basically how to um how to make a request or how to make a send a message or a command to another micro service now we don't even have another micro service running we've just got this one micro service that we've just started so you know practic practically there's nothing else that we can talk to yet but what i'm going to do is i'm just going to fake it so i'm actually let's add a new a new route actually so this is how you might handle a command coming into this microservice some command it can be anything right can be any name you want can indicate what what's going to happen there'll be a request and a response and then um yeah you'll do something you'll do some action here do do the command what whatever's being requested of this microservice you know it's going to read from its database save to its database you know do some computation do some calculation whatever is the job of this microservice it's going to do it there and eventually you have to kind of um send a response code otherwise you just leave the client hanging so you could you could return data or you could just return a status code hopefully that's the right thing so let's just try this so that's some command yep so it's returning status okay and if we look back at the console you can say see do the command so we've essentially we've just instructed that microservice to do something and i sent that request from the browser but you can imagine you know in a real application that request could be coming from a different micro service or it could even be coming something coming from like a web hook outside the application itself now what i want to do here is um kind of forward forward a command or you know forward a query or trigger you know send a sender command to some other microservice that's what i'm going to use axios for so i'm going to use a http get request and i've got an external service that i can use that i like to use for prototyping called jsonplaceholder let me just grab the url that i need i can find it oh there we go so i'm just copying across the url for this thing this is a placeholder rest api so you can use this yourself if you want to practice using um making your requests to rest apis it's really cool it gets you back a json list of to-do's you could just imagine let's just pretend that we're talking to the zoo's micro service okay just to kind of set the scenario so we're making a http request using axios and then we're going to get the response and uh let's just return the response directly to the um the requester or i should say we'll return the data in the response to the requester and just to be good uh good citizens will handle the error and if there was an error we'll log it and error occurred and we'll actually print out the detail of the error itself and then we're going to return an error status code so it was error status 500 which means something went wrong in the back end so i'm saving that so now what i have is i have the ability to accept the command from some of the micro service you know send a command or send a query or or you know however you want to interpret this to the to do's micro service let's just pretend that that's what it is then we're returning the response to the client and uh if there was a problem we're doing an error code 500. the status code is implicit here that automatically sends a status code 200 which means success i'm going to switch back to the browser if i run this again i should see my list of to-do's let me just zoom out a bit so you can actually see that so here we've got um all the to-do's coming back right so you imagine um you know if this was a real micro service that was doing something it might not be just returning data here you know it might be doing some sort of transformation on the data um storing it into its own database or something like that you know it could literally do anything um that it once now there's not much here right let's look at the whole code i'll zoom out so you can see the whole code this is like the basic code and the basic communication ability for a fledgling microservice you know it's simple but believe it or not this is a great starting point for a node.js microservice um there's different things that we would like to add to this to make it to you know to to be able to do its job we might need a database we might need a way to store files and load files that's something i'm going to talk about in part two of this talk it's coming up later in the month so i'm not going to talk about that today and also there are other ways that you want to communicate with other microservices so at the moment we're just using http requests to accept incoming requests and send outgoing requests but we might also want to use some sort of asynchronous or indirect message queuing and in part two i'm going to show you how to use rabbitmq to do that so that's a more advanced technique for communication between services um we're keeping it basic we're keeping it simple today and uh and i'm gonna wrap up soon so uh i need to move on but what the last thing i want to show you is how do we get this into a docker image and and then boot a docker container so we need docker like why do we need docker it's because we need a way to package this we need a way to publish this so we can deploy it to production and that's the problems that docker sold for us let's just save this code so i want to i'm going to stop this running in my terminal because i'm going to use it to build docker in a second and now i have a couple of docker images downloaded already if i do a docker image ls that shows you that i've got node 14 and node 14 alpine downloaded already um i would have liked to have started this demo with nothing and just show you how you get everything from scratch but unfortunately these images take a while to download and i don't want to waste your time you know waiting for something to download so i've pre-downloaded them but they're the only images i've got locally and i'm going to build an image now for this micro service and in a minute um you will see it in this list you will see that i've um i've built the image and that it's available here how do we do that well to start with we need a docker file um and we just call that dockerfile literally just dockerfile that's the default name for this thing uh let's get rid of that i don't need that uh oh the other thing we need we should have in a node.js project is a docker or any any kind of docker project probably needs the docker ignore file and this is just like the git ignore file if you know what that is or the npm ignore file if you know what that is it says what to ignore not to pass to the docker daemon because it can be quite slow if say your whole node modules directory here gets gets copied into docker we don't need to do that so we add node modules to docker ignore it won't matter on this small project but it does start to matter when you start to get a bigger project with more with more dependencies installed you know like if you have megabytes of dependencies you don't want all those being copied copied into the docker daemon okay the dockerfile so the dockerfile is like a script that basically says how to build the docker image for your microservice or you know docker's not just for microservices obviously but that's what i'm using it for at the top of the dockerfile we say you know where do we want to what do we want to start with what image what existing image do we want to base this on and i will base this on node 14 um alpine now it could just be node 14 so that bases it on a linux distribution um that includes node.js 14 already installed so we've got our dependencies already installed in our image for node.js the alpine version of that is just a cut down linux distribution it's better for production because it's a lot smaller it's a lot smaller than the non alpine distribution so that's why i'm using that and typically we'll set a work directory so this is the directory that i'm going to work with inside the image that i'm creating um and i think you know this is just a sort of a i guess a tradition that it's like app source app oh sorry user source app you know if you do the docu tutorials i think that's what you get and so that's probably why everyone uses that i don't think there's any particular reason why it should be that then you start copying files you start copying files into your image so typically i want all my package files so i can install my dependencies now i'm going to run a command inside my image so now just bear in mind here that i'm using windows 10 but what i'm what i'm doing here is i'm doing things inside a linux image so it's a linux image with node.js installed so i've definitely got to use um i've got access to all the linux commands here but anything i run here like this npm install i'm going to install my dependencies for my project anything i run is going to run inside the image and i'm saying only install uh production dependencies here so i don't want the dev dependencies like the one i installed earlier was nodemon but you could have any other dependencies for testing or to help you with debugging or development and you don't want to include them in your production build so just make sure you only include your production dependencies now i've got a i've got one code file to copy that's index.js so i'm going to copy index.js into my image now typically i would actually i wouldn't i wouldn't do this typically i'd have all my code in the source directory and i just copy the whole folder of source code but you know this is a simple microservice and we're you know we're setting it up simply here what do i do next just one more thing now i i need to give it the command to run when it starts the container so when it when it boots a container from this image when it's starting the micro service you know what is the command that starts it well it's just simply npm start so that that should be the command no matter what node.js project it is that should be the command that you can use you know if the developer of it has followed node.js conventions npm start is what starts it so hopefully i've typed that in correctly with no issues let's build it the simplest way to build a docker image is to go to the directory where the dockerfile is so you can see that's the dockerfile here that i just created and then i'm going to run docker also let's look at the version of docker i've got first so i've got docker version 2010 too i've installed this using in a sec let's see sorry just checking if there's anyone coming in but uh so i've i've installed um docker desktop you can get that you can definitely get that for mac and windows i don't know if you can get it for linux you might have to install it using apt-get for linux but that's okay that's easy to do as well with windows 10 though you have to install the wsl2 subsystem so that that basically allows you to run linux on windows under the hood as a virtual machine it's not too difficult to set up and it's totally worth it it's totally worth it to be able to use docker on windows to install wsl2 and if you're if you're if you're working on windows but you're a linux fan you'll love it you will really love wsl too anyway um so to build this the simplest thing to do in this directory is to run docker build dot the dot's very important that tells it to build against the docker file in this current directory so that's building now um you can see here it's running the commands like the copy command and it's doing the npm install so it's doing that mp install inside the image and it's finished so i should be able to do a docker image ls now i should be able to list the images that i've got and you can see here that i've got a new image that was created nine seconds ago so that was what i just created then with the docker build i didn't name it so it's got no name i didn't i didn't put a version number on it so it's got no tag so you could you can do that fairly easily and you have to do that if you want to publish it i'm not going to publish this i'm just going to build it and run it locally just to give you a taster and i can run it now i can run i can say docker run now if i if i just use docker run by itself you know this will this will actually run it you'll see that you know it's it's we now have a micro service the one we just built running under a docker container so it's running on running on linux under windows which is which is incredible really but um we can't access it it's on port 3000 but we can't access it and the reason is because it says localhost but it's actually running in its own local host like it's basically in its own virtual computer so the container is like a whole virtual computer basically that's running inside a real physical computer but we can't directly access it in order to be able to access it i need to do some port forwarding so i need to be able to map ports from the host operating system of windows into the container which is linux we use that we do that using dash p and then we say what what is the port number on the host operating system and what is the what is the port number in the virtual operating system so this is going to start it again now but now i should be able to access it on port 3000 so you can see i can run this now this this is the hello world micro service but it is running under a docker container it's not running directly under node.js on my desktop computer hopefully that makes sense so let's have one last look at this docker file before we leave it um so you know this this is it this is a a really simple node.js project uh you know we've built a really simple kind of rest api slash web server you know it's it's a great beginning for a micro service and it's just super simple to put together there's nothing there's nothing to this and then we built the docker file so the dockerfile allows us to create the image which is basically the package we're packaging up our microservice so that we can deploy it now we've we've tested it locally we ran it locally but ordinarily what we do at this point is we do something like a docker push and we tell it we want to push the image to our private container registry in the cloud and from there we could publish it onto our production you know kubernetes cluster or wherever we're running our containers we're not going to do that today we're going to do that we're actually going to do deployment to production in part three coming up in about a month so stay tuned wherever you found this this talk in the first place go back to that meetup and and sign up for the next talks as well so we've done a little bit of coding you've seen how easy it can be to put together a microservice in node.js and that's about that's about it for today um you know we've had the intro to microservices we've seen how to build a simple micro service in part two and three we're going to scale up to multiple micro services and we're going to have a look at how to deploy our micro services to kubernetes so please join me for those talks coming up in the next month and i just like to leave you with a link to my book bootstrapping microservices again you know if you like what you've seen today you'll find more of it in my book it's a practical and project-based guide to learning microservices and actually getting a microservices application into production i have two books to give away today i'm going to give them away at random to people who are still here with me but there's a code on screen if you want if you want to get 35 percent off if you don't get a free book use the code and get 35 off i'd love it if you read my book okay so i have got i'm going to i'm going to find a random number generator that i had already loaded here somewhere we currently have 16 participants so i've got two books to give away so you've got a pretty good chance of uh of winning a copy of the book i'm just going to get the participant list on screen i don't know i don't know if you guys can see this i'm just going to open the chat window as well if i can find it okay cool i can see the chat now i've been uh sorry if there's been any questions during i will get the questions in a second uh oh yeah someone has asked a question um shouldn't it be command you know square brackets npm start but you can do it either way you can you can do you can just put like a string based command or you can put like that array based command it doesn't matter you can it's your choice i don't i don't really know if there's any benefit to doing it one way or the other for me i like it to look like you're running a normal command on the command line so i use just a like a string and that that's always worked well for me anyway uh where's my random number generator that i had here we go so we have 16 participants and i'm about to roll the dice from 1 to 16 let's let's count from one like we're normal people and not coders so basically what i'm going to do is i'm going to i'm going to generate this random number and i'm going to count down the list um not including myself actually i might be included in the 16 so yeah i probably am so let's let's make it let's just count 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15. okay so there is there is only 15 people not including myself so uh let's roll the dice and see who gets the book seven so that's one two three four five six seven george barnaby you have won a copy of the book congratulations i'm just going to write down your name so i remember to give you a code in a moment so george barnaby please stay on the line don't don't leave the meeting until i give you a give you a code for the book okay let's roll the dice again and see who gets the other copy that i'm giving away today number one that's tuba tuba please stay online uh for a moment and i'll give you a copy of the book in a moment awesome congratulations to those two uh who got a free copy of the book i'm just gonna i'll just stop the screen share now and i'll get the chat back is there anyone who wants to ask a question let me see if there's any other questions in there someone said it shows me as number one well i don't know if you could see my list of participants like zoom might have hidden that from you um but basically like um for everyone who's watching you're probably in the number one slot so sorry if that was confusing i was counting down from the number in my list basically which i don't i don't it's not alphabetical order i don't know what order it's in but you know that that's the way it goes it was it was a random number generator um what time zone are you in i am in um australian eastern standard time zone i am in brisbane and it is um it is just past 9am in the morning for me just looking through any other questions uh will the recording be shared uh yes um matt matt has said that he will share the recording today i think so um stay tuned uh and i'll um as soon as i have a recording as well i will tweet about it and i'll also be tweeting the slides and this and the slides and um the example code as well i'll put the example code on github and share it with everyone um i haven't i haven't done that as yet but i will do that later and i'll and so keep an eye on my twitter account keep an eye on you know wherever you saw the meetup in the first place keep an eye there because you'll see the recording and the slides and the example code there okay we've got some good questions coming up how does the port forwarding work in a production environment um well that's a little bit more complicated but it basically works the same way like um i deploy to kubernetes and um i will create a configuration file that allows me to deploy a container to kubernetes and similar to how i put you know dash p3000 column 3000 on the command line just then i'll put something similar in my in my kubernetes configuration um that does that port forwarding but typically i just expose my micro services on port 80 and and i serve them on port 80. so there's nothing complicated that needs to be done and i will show that um i think that will be part three of this series of talks coming up later in the month i will actually show how to kind of deploy a micro service to production and i will show you how to do that port forwarding in uh here's a question uh in terms of data how many tables does a mic doesn't mark sorry i'm not sure what i'm reading here does a micro service should have and how do we segregate functionalities do you i i i guess you mean database tables and the answer is uh if you're talking about database tables the microservice can have as many database tables as it needs or wants there's no you know there's no particular limit um it really depends on you know what the microservice is what data it needs to manage and and i haven't talked about databases in this talk at all like that's coming up that's probably going to be coming up in part two where we're going to scale up to some more advanced topics so please come back for talk too and i'll show you how i would integrate a database into a micro service but you know a database uh like a micro service you you usually follow the rule that a micro service should have its own database you can you can share a database server between all your micro services but each micro service should have its own database but it can have as many tables in that database as it wants um there's a question here what is the difference between rabbitmq and memcached i don't know i i i've heard of memcached and i think i read about it once but i can't remember what it is um so i can't answer that but rabbit mq is a like a a messaging queue so i'm going to be showing you how to use rabbit mq in talk 2 in talk 2 of this series um because that's what i use i use rabbitmq in production it basically means you can have indirect communication with other other micro services which typically means you want to publish a message like um publish a message to other or broadcaster message that other micro services are interested in but it's indirect you don't really know if other micro services are handling that message and uh and it's a great way to kind of set up streams of messages between microservices and i will give you um hopefully a live demo of rabbitmq if i can you know if the demo gods let me do that uh in talk to uh next question you mentioned typescript can you contrast that to express they're two separate things so they can't really be contrasted so so um javascript and typescript can be construct contrasted javascript is a dynamic language um and and typescript builds on javascript to add static typing and a build phase and a compile phase so using typescript allows you to kind of have like extra error checking on top of javascript that you wouldn't normally have and if you come from a statically typed background like if you are previously a c sharp c plus plus or a java developer um typescript basically adds those static typing features on top of javascript so javascript doesn't normally have that so you can you can think of typescript as like an add-on to javascript that kind of improves the development process it can't be contrasted it's like apples and oranges it can't be contrasted to express you know you can you can use express whether you're using javascript or typescript it doesn't matter um they can't be contrasted really uh in production can you use the reverse proxy like traffic i haven't heard of traffic but one that's fairly common is um ng and ix uh nginx i think they call it um to be that can be used as a web server front end or a proxy um a really good one actually that i've heard lots about in podcasts and i want to try out is called um envoy and that's actually on the same level as kubernetes and it's it's a you know it's a graduated open source project uh in the cloud native computing uh foundation so if you are if you're after a proxy reverse proxy you um check out envoy that might be able to do what you want sorry i can't you know uh i can't say anything about traffic because i don't i don't know about that what else we got here oh memcache is similar to redis cool um yeah i have used redis so what was the question what is the difference between let's say what is the difference between rabbitmq and redis um they're completely different so redus is like an in-memory database um designed for like caching caching data um so you might have like if if you have a regular database like sql on mongodb and it's slow to load you might have the kind of load of data go into a memory case like um like memcached obviously or redis rabbitmq is for message queuing so rabbitmq is like it's not a database so it's not in that case it's not really comparable to memcache they're separate tool separate software for separate purposes how do we handle cross-cutting concerns like security distributed logging and tracing and are there any packages that we can import and provide out-of-the-box support uh yeah and this is a huge area of like active development and research you know typically if you want to kind of have an add-on to a micro service or or an add-on to a container with kubernetes you'll do what's called a side car um so that it that is uh like a container that's deployed alongside you know whatever it is and augments it it augments it for security you know for proxying um you know you're logging you're tracing um it's called the the design pattern is called a sidecar and but there's a bigger a bigger kind of structure architectural pattern called a service mesh uh which personally i haven't used i haven't needed you know it's a pretty advanced topic but you know well worth it if you're in an enterprise you know looking to roll out a massive micro services project and you want some sort of control you know over the whole thing and you want to be able to kind of roll out these these cross-cutting concerns you know across micro services you're what you're after is a service mesh um i've heard istio is good i think you know it's an open source project that builds on kubernetes so that is most likely what i would use if i felt the need for something like that uh i've heard good things about it if um i i think i don't think it's up at the level of kubernetes i'm not sure that it's graduated or it's a graduated cloud native project yet in the cloud native computing foundation but it might be it might be um like it might be kind of moving through those phases i don't know enough about it sorry but um google istio um that is something that you can use to roll out you know these cross-cutting concerns okay i'm just trying to see if there's anything i've missed here in these questions oh yeah so um someone said you know um traffic is you know handles ssl certificates for you um cool i use cloudflare actually i use cloudflare for my startup for creating um for creating a proxy and creating ssl certificates it just makes um you know using secure http it makes that really easy to use um hopefully i've got everyone's questions i'm just scrolling through i can't see anything else uh hopefully i haven't missed anything oh can you say something about the city pipeline i will i will definitely say more about that in in talk three you know i'm gonna actually you know probably i'll probably give you a demo of of a simple cd pipeline um that's usually how i start is i make a code change i push the code change it triggers the cd pipeline and it triggers the deployment of a microservice to kubernetes so i use at the moment um you know a variety of different things depending on the project i'm working on for my startup sortal i use bitbucket pipelines so you know essentially what cd continuous delivery is it's a shell script running in the cloud like it's it's not that complicated that that's how i like to explain it simply you have an automated shell script running in the cloud whenever you push code changes so you make some code changes push it it can triggers your cd pipeline and um you get your basically your micro service or your web server or whatever whatever you've got you know pushed out to production so i'm using bitbucket pipelines but i'm also uh for some new contracts that i'm doing and also for new new um kind of personal projects and prototypes i'm using um github actions which is which is fairly new on the scene i mean it's probably been around for a couple of years now but it's been through evolution so it's changed a lot but i'm really loving where it's at now so github actions is is just a great option for for doing continuous integration or continuous delivery i thoroughly recommend it does that help but please feel free to come back and talk three where i'll actually give a demo and we'll look at some code for a cd pipeline i'll actually show you code cool okay i think that's about all we've got time for um her contract first design pattern in designing my is that is that like contract driven testing or something i'm not really sure what that what that means all right i'm gonna i'm gonna send out the um the codes now for the people who have won so hold on for just a moment sorry i'm just trying to find where i put these back the codes for the giveaways just give me a moment okay so the first one we're giving away is to george please take a copy of that code and the second one i'm giving away is to tuber sorry if i've pronounced your name wrong cool so that's two copies of the book given away to george and tuba thanks so much um what you need to do to to to recover that to use that code is go to manning.com uh sign into manning.com um uh get a get a copy of my book basically in your cart and when you go to purchase it make sure you use that code and you should get it free um if you don't if there's any problems you know please feel free to reach out to me i'll put my contact details in the chat here so you can direct message me on twitter through ashleydavis75 or you can send me an email ashley at code capers dot com dot a u um and that goes for anyone if you want to have it if you want to talk tech have a chat ask a question please feel free to get in touch with me um and yeah to uh let me get you the the link for my book again hang on a sec just let me copy it so if you follow that link you will get to the book that's on manning's webpage you can read the first chapter there for free or if you want a free copy please enter your code grab a free copy of the ebook otherwise use the code that i put up earlier to get a 35 discount and i'll be putting this all online as well so check out my twitter feed or check out the meetup where you where you found this talk in the first place and later on today hopefully you're going to see the video a link to the video and you're going to see a link to the slides and you're going to see a link to the book and all that so thanks for listening and i'll see you for talk too hopefully bye from australia

Channel: Code Capers

Views: 443

Rating: 5 out of 5

Keywords: microservices, nodejs, javascript, cloud, devops, docker, development, coding, live coding, kubernetes

Id: iYpcODCkjRc

Length: 78min 19sec (4699 seconds)

Published: Fri Mar 19 2021