Why use Microservices, Docker & Kubernetes?

some time ago this chart of SLX notification system went viral because people were surprised by how complex it is so if you sent a message here in select let's say blah blah if I now press enter here there's going to be a request to the select server and if there are other participants in this chat s will decide whether to send a notification to them according to this flowchart so it starts here should we send the notification well is the channel muted yes or no well even if it's muted maybe there's some threat message and the user is subscribed to that and then it goes down all the way way here with all of these options eventually resulting in a final yes or no decision why am i showing you this because I want to show you that bigger companies structure their applications differently than maybe you are structuring them so I quickly want to explain to you what this debate about monolith versus micro Services is and then what Ducker is and then why you would want to use something like kubernetes but kubernetes is very complex so there is now a solution called Acorn that makes it super easy to deploy apps to kubernetes Acorn is also the sponsor of this video the the team behind this are some big names from the kubernetes world and they have come up with a platform that makes it super easy to deploy apps without knowing anything about kubernetes but before we get there why would we even think about kubernetes what is even kubernetes what is Docker what is this microservices what is a monolith so as a developer you may be creating monolith app so if I have a user here and here I have my app and this user or the client may send requests to my app and when a request comes in I want to run some code so if we take the example of Select maybe there is some code here that relates to sending a notif ification right notification code and maybe there's also some code related to just messages actually sending the message to somebody else and if the user signs in maybe there is some authentication related code and maybe the user uploads an image and we have some code related to that image upload code and I could keep going but I think you get the point which is that we now have one big monolith app so all the code is contained within one big all-in-one app and this is fine for smaller app but what you're going to see in the real world is that especially bigger companies this is not how how they structure their applications and there's a good reason they don't do it that way because this has many downsides for example if we look at the notification code you can see how complicated this is in slex case so it's possible that there is a team dedicated to dealing with this notification code so there is a team working on this but then there's maybe also a team that's working on authentication codes a different team or different people and now let's say the people working on the notification code maybe they made an update and they want to redeploy the app well that's a problem now because if they want to deploy the app it's going to interfere with the other people working on the app because it's one app you're going to have to deploy the whole app so this may interfere with other people's work that are working on other parts of the code and maybe the people that are working on the notification code they want to use Java as their programming language and maybe the people that are working on the image upload code they want to use JavaScript well that's not really going to be possible because this is one app you're going to have to pick one programming language and there are many other issues with this monolith setup as well so instead what you may want to do is split this up into separate micro services so basically little apps for example if I zoom out a little bit here I can create a separate little app here just for the notification code so I'm going to put it in here and instead of calling it code I could call this servers notification servers I will just call it notifications for now and then I also could have one for the image upload code another little app essentially and another one just for the messages and then another one for let's say the authentication right just for all the Au related code now the client will not directly send a request to each of these Services it will just send it to one place which will route it to the other services when I say service it's just a little app essentially so I will just call this router and then this will decide where to send that request to or which services to invoke maybe for a particular request it needs to use both the notification Service as well as the O service because maybe they usual logged in and we want to notify somebody of that so this is starting to look more like how an Enterprise company would deploy their apps and I used slack here as an example but for example Spotify gets millions of requests and they have teams working on different parts of the software so in Spotify for example there could be a service called user activity just keeping track of what the user is listening to and then maybe they have another service for recommendation which songs to recommend and there could be many more there could easily be a couple dozen services or little apps that a big company is using instead of putting all of that in one big app and these Services may also be talking to each other right so for example the recommendations are based on the user activity so they are in contact in some way which makes sense because if the user is listening to a particular genre of music you probably want to recommend within that genre of music now obviously this is going to increase the complexity so why does this justify that extra complexity especially for large companies well there are many benefits actually so the notifications team can work in Java and over here for the recommendations maybe they do a lot of work with machine learning and machine learning has a lot of python so the people here want to use Python and the people working on the image up here maybe they're using a third party for images and the third party has a really good library for JavaScript or or in JavaScript or typescript so they want to use JavaScript or typescript here which is possible because these are their own isolated little apps so you can pick your own programming language this is also really beneficial for big companies for recruiting Because if they're using one programming language they can only find programmers that are working with that specific language however now the programming language doesn't really matter it it's up to the developer essentially so very beneficial for recruiting new developers as well it also makes everything much more efficient in terms of using Computer Resources as well as cost because let's say are some big events happening and people are suddenly sending each other a lot of messages that means we want to assign more compute resources to this messages app or server but not to anything else and maybe also for notifications here but not everything else so we just want to increase compute resources for the messages service and maybe also for the notification service or app but let's say that everything else can stay the same so now we don't have to scale up our entire app only the services that actually need it so if some Services aren't getting much traffic for example image of upload maybe you put that behind a more expensive plan so only Pro users can upload images let's say and maybe that decreases the load in the image upload service well you can simply scale it down without scaling the whole app down right so you become very efficient with compute resources and call another big benefit is that they can be deployed independently so let's say the notifications team they found some bug here and they need to quickly fix this bug so they fix the bug and now they want to deploy it to production they can do that without interfering with any of these other services and so they can just deploy while the recommendations here will just not be deployed again and what if there is some bug in the user activity surface and it crashes this surface well now this will fail in isolation it won't affect the other services and especially with user activity for example this is not even a surface that's really necessary to use the app so it's nice that this fails in isolation without affecting the rest of the app the the rest of the app can still continue if you had a monolith all in one this could crash the entire app also let's say this image upload that we were talking about that third party library that you're using for image upload maybe it only works with Note JS version 18 let's say but maybe in some other servers you want to upgrade to noj S version 20 well now you can already upgrade this one without affecting this one this one can stay at version 18 maybe it's necessary for some thirdparty Library so you can upgrade versions without affecting the other parts of the system and I could keep going there are other benefits let's say the messages you push it to production and there's a problem you can roll this back without affecting the rest of the app so there many benefits of splitting up your big monolith app into separate smaller micro services and that's why especially in big companies Enterprise you're going to see this structure a lot and now maybe there's a problem after an image upload there is no notification being sent so maybe somebody from the notifications team they want to run this image upload service on their own computer so they're going to get this JavaScript code on their machine and they will try to run the JavaScript code but you need a JavaScript runtime like nodejs to run that JavaScript code on your computer and since that person is working only in Java they may not have that noj s runtime installed on their computer or maybe they do have it but it's a different version and that's only noj s there could be other dependencies as well that they have not installed on their own computer so to solve that we can use Docker we can dockerize this application as it's called which basically means we're going to create a container and in that container we will have the actual app as well as the dependencies it depends on so it would have the app as well as the nodejs runtime in this example so then that other person can just run the docker container without worrying about installing the nodejs or other dependencies so Ducker makes it very portable and there are other reasons for using Ducker as well but Ducker essentially just separates this app it isolates the app from the machine that it's running on and that improves portability so we're going to dockerize all of these little apps or micros Services another word for this is container so we're going to have a container for each one of these services so now we have all of these containers and we need to manage them and we could manage this manually ourselves for example we could manually just monitor or check these services to see which ones are getting a lot of traffic and for example if we see that the notification service is getting a lot of load and it's close to maxing out the machine that it's running on we may want to decide to scale this up so we could simply spin up an equivalent service somewhere else in another server and maybe another one on another service so this is horizontal scaling and we could do it ourselves manually right we can just spin up a new server and just create another instance of this surface you could also do vertical scaling which is just making the machine that is running on more powerful with more CPU more RAM and you could do that yourself manually and now let's say that this user activity service still has that bug it wasn't solved properly so it goes down again well you could be manually monitoring everything and you can see that it goes down and you could just manually spin up a new server and so spin up another instance of this surface to get it up and running again right so you could manually monitor everything yourself but it's more convenient to use a tool that can orchestrate all of these containers for you and that's where kubernetes comes in so kubernetes is the most popular container orchestration tool so some of the things it can do for you for example is automated scaling up or down so if the notification service is getting a lot of load when it reaches certain thresholds of usage of the machine let's say it can automatically scill this up or down or let's say that this o micros surface is running on a server and the server is malfunctioning or just not working properly well you can duplicate the service or maybe you already had multiple instances and kubernetes can just reroute this to the healthy note Also let's say we're working on the notifications uh surface and we're updating it when you update an app it will stop working for some time while it's being updated and restarted so to prevent that downtime kubernetes can automatically spin up another instance here route the traffic to that one while this one is being updated and after some time this one has been successfully updated so then it can just go back here again and just delete these resources again so you can update your app without downtime and so kubernetes is a very powerful tool to orchestrate this whole system to make it really resilient which is very important to especially big Enterprise companies when you have millions of users or millions of requests or there's a lot of money at stake even a little bit of downtime is very costly so you want maximum robustness and that's why I'll often see a tool like kubernetes being used in large Enterprises all right so then we get to Acorn here because the downside of kubernetes is it's very complex so if you ever try to look into it you probably saw how intimidating it is there's so much to learn and that's actually why Acorn was created so Acorn comes from a group of people that actually are some big names in the kubernetes world if you are familiar with kubernetes you've probably heard of Rancher and the idea behind Acorn is that you can deploy your apps to kubernetes without all of that complexity an acorn behind the scenes will take care of all of that kubernetes infrastructure so that you as a developer can just deploy to Acorn without worrying about all of those kubernetes concept so we can get all of those benefits from using kubernetes without worrying about the complexity that comes with it so I want to show you how you can get those benefits from kubernetes without using kubernetes by just deploying two Acorn now I don't have seven or 10 different uh apps here to show that I'm just going to have one app for now and let's say that's a python app it's a flask app and you don't need to know python I could have also used nodejs or nextjs which is essentially just a no noj s process or Java it works the exact same so if we take a look at this app here what it does we can see that it's going to connect to a database so a database in a way is also a service so what we can also do is have our database actually also just be a server right and we can also dockerize that and then also deploy that with the rest of our micros service and that's actually what we're going to do here so this app will connect to a separate database so we have some environment variables here now what does this app actually do though so here if we look at this there is a web server here so when you go to the homepage it will query all the to-dos from a database it's getting all the to-dos from the database and then it will pass that as a variable called to-do list to the template. HTML file so that's right here so if we go in here we can see that here it's looping over all of those to do and it's outputting that markup all right so very simple flask app very similar to how you would do something like that in Express for example and Nots I'm using this example because this is what you'll see on the acorn website and I really like this example so I will just walk you through the example that they also give you the first time you go through the dock so here we also see some routes for adding and updating a too and then also here one for deleting okay now we don't really need to know exactly what's going on here it's just a flask app and what we just learned is there are benefits to making that a container so we're going to dockerize it and you do that by creating a Docker file so if you remember we use Docker so that we can bundle this app with all the dependencies that it needs so it can be run everywhere it's very portable so in this file we will put all the things that it needs so it needs flask flask SQL Alchemy and byy my SQL so you can see here in the app code those are the things that we're indeed going to use so our app depends on that so that's what we put in this file and then we can dockerize our application here by creating a Docker file and here you can see we are referencing those requirements so from this Docker file we can build a Docker image as it's called and that image will be on some registry somewhere on the internet so then if somebody else wants to run this app the other person can just pull that image from the registry and run the app regardless of what operating system or what other dependencies are needed because these dependencies will already be part of the that bundle or image essenti and we could do the same for other apps as well right so here we just have one app but I could create five more and we can do the exact same create a Docker file for each one and then we would have a bunch of dockerized applications so another word for those would be containers and we want a container orchestration tool like kubernetes to for example automatically scale up or down and or automatically reroute to a healthy node but to set up kubernets ourselves is very complex so instead we're going to deploy our applications to Acorn which will kubernetes under the hood but we won't need to know about that we only have to deploy and acorn takes care of the complexity so now that we have dockerized our application here how do we actually deploy it to Acorn well you simply create another file and that's called The Acorn file now Acorn actually does have an extension in visal studio code so you can download that if you want unfortunately for me it does not pick up this icon for the file yet but that's okay so in an acorn file you can describe all of the apps that you're going to use right so if we go to our example here we would have one Acorn file for everything right so Acorn ties everything together here right so if we look at this specific example you can see that we're going to use a database and this is a Maria database but could also be post grass or mongodb this is a standard database so there's already a Docker image online so we don't have to specify all of this for that database we can just grab the standard one from the registry and then here we have our own custom app so here under containers we can describe all the apps that we have and in this case we will only have one so we're going to build that from the current directory so it will find this Docker file in the current directory it consumes this database meaning in the app here we are indeed trying to connect to a database we want to publish this port so we can connect to it and then we have some environment variables that are needed in our application right so here we need these environment variables to connect to the database so we need to inject that into our app so with Acorn it's actually very convenient to work with these secrets because acorn will generate these secrets for you and you can then easily reference them like this to service. db. secrets. admin. username and also for password so then these values will be injected so that our application can actually use them and then here we have an if statement for only development so our system right now is just two right so in the example here I had like eight different microservices and in this example here we have two right so our own custom app and then this standard image that we're just pulling from the GitHub container repository right so we could add more apps here we could have many more like what we have here and you would just specify all of that here in one file right so the Aon file is actually quite similar to a Docker file and the syntax is also quite similar to Json so it looks quite familiar in many ways and we don't need to use any yl files or anything else this is all that we need so then I can go to the acorn website and I can log into the dashboard and here I have nothing yet so this is going to be the empty view so here we have three options so I can click on this one already have an acorn so what we do what we're doing here is very similar as with the docker file so with the docker file from there you can build the image and the image can be uploaded to some registry and it's the same with an acorn file so also just like Docker you can create an artifact from this and it can also be uploaded to a registry so then you can deploy that here in the cloud by just picking that Acorn image from the registering now they also have some standard acorns here for some common applications right for post grass mongodb red now we don't have an acorn yet in a registering and we also won't use this catalog for now so I will cancel here so we can also write the acorn file right here in their playground so we are doing it here in my code editor but you can actually do all of that here in the dashboard as well and it will actually automatically spin up an app here I will Top This for now and we'll continue this later you can see now we have some additional data here from the playground I will delete it for now so now we want to build this and then also run it so I'm going to open up my terminal so Acorn has a CLI so once you install that you can use Acorn commands here so I can say Acorn run and then reference a name and I will call that cool Dash example so if I press enter here it will build and then also run our app here and that may take a couple of seconds right so you can see it's building it now by default there will be an acorn project actually and it will put the app in there so now here we can see the result of this in the dashboard cool Dash example I can open this up and it will show me the containers in here so we can see there's a service here the the Maria DB uh service and then we have our own container for our own custom app and when you click on that you will see the resources that it's using right so here CPU memory number of requests well that should be zero because we haven't gone there yet so actually it will also immediately give us an endpoint here right so here you can just click on that and you will be able to go to this project and you can see now this is working and so our app here basically consists of two Services right so we have our own custom app and then we have a separate database and I can now see if that works so I'm going to say test and we'll see if it actually in inserts it into the database okay so it's so we see it here in the UI but if I refresh is that data persistent and yeah it's persisted right it's still there and I can also Mark this as complete and I can also delete this yeah right so you can see how easy that was and here I'm essentially just using two microservices but I can do this with many apps many little apps I can get the benefits from kubernetes but you could see I didn't even write a yo file I don't need to know anything about kubernetes now to make that work because Acorn abstracts that way Acorn worries about kubernetes you can forget about it as a developer you can just focus on deploying app so here now you can see I can monitor these resources I can see there has been a request and you can see the other things as well I can inspect the acorn file here in the dashboard as well right so what we get out of the box here is monitoring logging also secret management really beneficial here as well but also things like Auto scaling and debugging so under the hood Acorn uses kubernetes but you as a developer don't need to worry about that you can just deploy apps without knowing anything about kubernetes so Acorn could be a game changer for you if you're thinking about using kubernetes or you're looking for an alternative because maybe it's a little bit too complex for you or the developers that you're working with or maybe you want to consider using Acorn or maybe you're just looking for a cloud platform to easily deploy your backend apps to well you may want to consider using Acorn as well so what we did here this was all in my code editor here but you can also go to this playground here and here you also get an acorn file so here you you can also describe that whole system and you can do that right here in the dashboard actually so as soon as you open this up it will create an app for you and one thing you can also do is share this with other people so if you have an idea you can just share this with other people they can click and open this link and it will open it up in a dashboard for them as well right and they can create an app for themselves out of this so now I created this ancient Bird app so here now you can see it in the overview ancient bird now with these apps it will automatically give you an endpoint as well you can actually access it so that's really beneficial as well and you can see it's just a very simple engine server and I don't need to worry about Ingress or anything else it's all handled for me under the hood all right so there's one more feature that I want to show you so we were working on this cool example app so one thing you can also do is you can go into death mode so I can simply say Acorn Def and it will actually have live reload capabilities so here it will run a quick build all right so then at some point it's finished and I can click on this link here and open it up in my browser and you can see I'm back to my app here but now if I change the acorn file or any Docker file that's connected here here I I will set flask debug to zero if I save here you can see it picked up on that and it will automatically rebuild and the cool thing is this is automatically synced as well here in production as well and they have some other cool examples on the website so one of the things you can do is actually start up your own Minecraft server here with just one click right so then you can deploy this here within Acorn it's a really interesting platform check it out I want to thank Acorn for sponsoring this video and I hope this was helpful hope that you learned something thanks for watching have a nice day and bye

Channel: ByteGrad

Views: 17,702

Keywords:

Id: y2ICZYOU09Q

Length: 23min 4sec (1384 seconds)

Published: Tue Nov 21 2023