Unlock Fast and Efficient Local Development with Kubernetes and mirrord

setting up development environments is scarred especially when working with serious or big systems so let's try to make it a bit easier [Music] most of the time it is easy to set up whatever we need to work on a specific application Visual Studio code terminal Docker or local kubernetes cluster a few other tools and be ready to go those are not the problem I'm sure that every single developer is perfectly capable of setting up everything needed to work on whichever application he or she is working on code easy building binaries and container images not a problem running tests Piece of Cake Building and running tests every time we make a change to the code well why not heck we might even offload some of that to remote environment so that easy becomes even easier however the problem is rarely the application we are working on unless we are working on something silly like weblogic but if that's the case you might want to look for a new job rather than improvements I'm about to discuss so what is the issue the real problem is everything else everything but the app we are working on you might be working on a back-end application it is being used by a separate front-end application that front-end application might need a few other backend applications to function properly each of those backend apps might be making calls to a few other backend apps then we might have databases message queues caches and so on and so forth how do we deal with that do we install and maintain all those dependencies locally if we do our laptops might Crush under heavy load and even if that does not happen we'll be spending most of our time installing configuring and maintaining all those dependencies remember it is not a one-time thing it's a continuous process all those are changing all the time and if you run them locally we'll have to keep up with all those changes there is a new release of a dependent backend application help upgrade the local business of it database was upgraded upgrade the local instance of it database schema was changed apply the change to the local instance of the database you get the point right it's hard and frankly it's a waste of time all that time that each of us would need to spend maintaining a copy of a production-like system could be better spent working on the application we are supposed to be working on that's the real goal here right so what is the alternative one option is to use production-like environments you probably have it you might call it pre-production or staging or something else it's a cluster where all the apps are running and it is similar to production we typically use it for testing of one kind or another however if we do have such an environment and if we do develop locally or even remotely the questions we should ask are first how does the app we are working on communicate with the rest of the system and second how does the rest of the system communicate with the app we are working on how are we going to solve those issues let's start with the first one since it is easier to deal with if the app we are working on needs to talk to another app we can simply configure our application with the DNS of the one running remotely right well not really if you're working locally and the app we want to talk to is remote we would need to open it to outside traffic well that's okay in some cases front and tops being the most obvious example it is a bad idea in others specifically those not designed to be accessible outside of the cluster where they are running we would need to expose that remote application specifically for development purposes and while that might not be the end of the world it is not ideal on the other hand if you're developing remotely inside the same cluster where Baseline applications are running things are much easier since all we would have to do is make sure that our app knows what the DNS of the other application is so we should just switch to remote development and adopt the tools that help us with that like for example octeto or tilt well maybe we should but let's not rush into that decision until we talk about the second question so here it goes how does the rest of the system communicate with the app we are working on imagine this situation we are working on a backend application it requires a database and is accessible through a front and up database well we already discussed that front and up well it's not that easy if you are developing locally we cannot easily instruct the fronted application to talk to our local instance of the application it's your laptop and not a node in the cluster that the remote app is running if you would run our application remotely we could instruct the front and tap to talk to it but that would be a terrible idea it's a shared cluster and we would be interfering with other developers the front and top should talk to whomever it talks to except if you explicitly tell it to talk to the app we are developing that could be through request headers URL paths and so on and so forth so it does not matter whether we are developing locally or remotely in the same cluster where the Baseline apps are running we need a solution to redirect traffic to and from Baseline applications from and to the application we are developing now the most common solution to that problem is telepresence yet it never fits my needs fully for reasons I will not go through right now right here what matter is that I was looking for an alternative that is easier more elegant and yet one that does everything I needed to do I think I might found it in Middle day so let's take a quick look at what it does and how it works imagine the following situation I have a kubernetes cluster that is similar to production it's a production-like environment that they use for testing new releases before they are promoted to production here's a proof I will send a request to one application which in turn will send a request to another application wait for the response and return it to me return the output and that's it it works it's not really production or production like but good enough to demonstrate some of the problems that mirody might or might not solve it's a mystery for now at least what matters is that there are two applications finger and silly demo each running has two replicas now let me switch to a second terminal session from where I will be developing my application that happens to depend on one of those two apps so CD finger source and now let's say that I've finished writing changes to the code and I want to test them since the app is written in go I will execute go run to run it locally now bear in mind that this video is not about apps written and go the principles and the solutions should work in almost any other language now that my app is running locally let's see whether I can test it by using C URL to send a request to it now that over there failed miserably my application Binger complains that it cannot reach the dependent up silly demo now I could run silly demo locally as well but as we already discussed that would probably mean that I would need to run the whole system locally and that's not something I want to do my poor laptop would freak out and I would have to spend more time in management of the local copy of the system than development now let's solve that by instructing my local application to talk to the other apps as if it is running inside the cluster where the rest of the system is running I'm going to stop my local application from running and run it again but this time with a Twist so here's what the situation currently is the remote application Pinger is talking to the remote application silly demo running in the same cluster what we want to do is work on my local application that is mirroring that same application running in the cluster so when I send a request from my local app that request should go to the remote application education as if my local application is running inside the cluster so I want to fake it and make my local application behave as if it is running inside the cluster so I will use mirror date to exec a local process that targets that same application running in a deployment called the Pinger I will use demo as both the agent and the Target namespace and finally I will instruct mirror D to execute go run command that will run my application in the same way I run it earlier you will notice that mirror this is complaining about multiple deployments apart from my personal note that I believe that it would be enough to see the complaint once instead of so many times I will ignore it for now we'll get to it soon now let's see what will happen if I send the same request again remember the request failed before because it could not reach the silly Dem application what will happen now let's see crl with the same address and voila this time it worked the application I'm working on locally is connected to all the other applications running in the production-like cluster I'm running it locally with all the advantages of doing just that while it still behaves as if it is running inside the same cluster with the rest of the system is running my local application is mirroring its counterpart running inside the cluster moreover my local application is not interrupting the system whomever else might be using the application running inside the cluster is oblivious to the fact that time running my local version of it we can see what happened if we take a look at the pods in the demon air space when I run mirror the exec it deployed an agent inside that namespace mirror the running locally is intercepting all the traffic from my application redirecting it to the agent which in turn is forwarding requests to whichever Services the application would be using if it would be running inside the same cluster mirror D is effectively allowing me to work locally while being connected to the rest of the system now having to write all those arguments was a bit tedious so let's try to simplify that by using a configuration file computation over there we can see the same arguments I used before there is the path and the namespace of the target application as well as the namespace of the agent now let's run mirror the exec again but this time instead of specifying all the arguments we'll use the configuration file if I send the request to the application using C URL the result is the same and that's cool right that was the easier problem to solve so let's try something more complex I'll switch to a different application located in the silly demo Source directory over here I have a different mirror D configuration apart from the Target and the agent that we already saw this time I'm adding uh one of quite a few military features I will instruct it to steal incoming requests from the target application running on the port 8080. effectively what I want is the ability to send requests to the remote application which should forward them to my local application instead of the app running inside the cluster let's see whether that works by executing mirror the exit again this time I will send 50 requests to the remote top we can distinguish output from the local application by the with mirror the message so let's run the loop the output is strange approximately half of the requests were handled by the remote dependency those without with mirror Day message while the other half of the requests were handled by the local application I mean those are the ones that contain with mirror the message so what's happening well I have two replicas of the remote app and the free version of mirror D Works only on a single pod now intentional limitation of the open source version of mirror day which we can overcome by using mirror D for teams which we will discuss later on the other hand there is probably no need to run multiple replicas of an application in production like environments unless we're running load testing which is not the case here so if your application like cluster is running a single replica of each shop you should be fine or maybe not we'll see there's another issue with the open source version of mirror day since mirror D attaches itself to specific pods it would be interesting to see what happens if those pods are removed by let's say a new release we'll test that by deleting all the pods from the seal them up that's not the new release but the effect is the same since those pods are managed by a deployment it detected that pots are not running and created new ones now let's send another 50 requests to the remote application this time mirror D did not steal any of the requests the Pod it for stealing from is not running anymore we'll talk about that in a moment for now I want to discuss a different problem stealing all the traffic from a remote application is a very bad idea that effectively means that no one else can use that application in the production-like environment since their requests would be forwarded to my local application as well what we need is to distinguish my requests from everyone else's my requests to the remote app should be forwarded to my local application while everyone else should be forwarded to the application running inside the cluster we can do that with mirror d as well let's take a look at the better configuration file this one is almost the same as the previous config the difference is in the header filter value set to test mirror day I'm sure that you can guess what it will do but let's say it in nevertheless here it goes mirror the exec again but with the improved configuration and then I will be sending 50 requests to the remote up and nothing all the requests were forwarded from the remote up to the other remote app in the same cluster none of them reached my local instance of the application does that make me sad well not at all that's good news I did not include the special header in my requests so they were not forwarded to my local application the remote application behaved as it should behave when somebody else is using it and what everyone else should see is uninterrupted traffic between the applications running in that cluster now let's do the same 50 requests but this time with the special header this time mirror D did forward request to my local application or to be more precise it forwarded those coming from one of the pods since as I already mentioned the open source version of mirror D Works only on a single pod finally once I'm finished developing my application locally all I have to do is stop the mirror the process that in turn will automatically remove the agent from the cluster we can see that by listing all the pods in the demo namespace it's still there let's give it a moment more and get the pots again and there we go now it's gone as if it was never there now I want to move to the commercial version of mirror D but before I do let me say that there are quite a few other options we can Define in the configuration file also there is a visual studio code the extension that makes it easy to debug applications while still mirroring and stealing traffic from and to the apps running inside the cluster now let's move to the commercial version of mirror day mirror the open source has quite a few issues many of them caused by Omission that can be fixed with a commercial version called Mirror D4 themes to begin with mirror D the open source version spins up containers with privileged permissions which is likely not what you want to do on top of that it fails miserably when multiple developers want to work against the same application running in a cluster finally it does not work well with multi-replica deployments in other words if you're working Solo or there are only a few of you and you do not deploy new releases to your cluster and you have no problem with privileged access the open source version might be good enough for you that means that mirror day is not a good fit for anything but hobby projects or companies with only a few employees if that's the case you probably do not have a large scale system to justify the need for something like mirror day what that really means is that the open source version is not a typical project that hopes to be adopted by companies first and then once it grows in adoption to start charging for an Enterprise version you can think of mirror d as a trial version of the commercial offering which contains limited set of capabilities it is open source and it is free forever but it is not useful to almost anyone for anything but a single person trying it out before making the decision to purchase the license for the commercial version so was the commercial version of mirror day to begin with it's called Mirror day for themes that names use it well it is a requirement for any team planning to use it what does it do well the four teams Edition provides an operator that does the heavy lifting inside the cluster and it enables teams of any size more or less use mirror D without any of the before mentioned limitations of the open source version now to be clear those are my best guesses mirror day for teams is in a closed beta at the time of this recording it might change by the time it goes GA moreover there is no information about the pricing nor even whether it is commercial so for all I know it might be free as well even though I don't think that's very likely anyways if you feel that mirror theme might be a good fit for your organization I strongly strongly suggest joining the beta program or if it went GA by the time you're watching this video to get the commercial version okay now that we went through both the open source and the commercial versions of mirody we are ready to talk about pros and cons or cons and pros and see whether you should use it let me start by saying that mirror D is a direct competitor to telepresence is it a better option well before we get the answer to that question let's see what the pros and cons of mirror they are and we'll go with cons first let me start with the most obvious negative 0.2 mirror day documentation is bad the quick start does not show what the main features of mirror they are it does not even mention still in traffic which from my perspective is the start feature of the project configuration section is going all over the place and contains quite a few pieces of incorrect information I could go on and on and on with the problems related to dogs but I won't because you get the point right it's bad now this is a project in an early stage and documentation is often something that gets improved later so I expect it to get improved soon heck by the time you're watching this video and given sufficient pressure to mirror the theme it should be much better by now so documentation is hopefully a temporary problem a bigger issue is that open source version of the project is useless for anything but as single person to try it out before deciding to purchase the commercial version open source version is not something I would recommend to anyone planning to use it seriously now let's switch around and go towards Pros the good things and the first one is that if I ignore the deficiencies of the open source version and I focus only on mirror default teams I can safely say that it is the most elegant solution of the kind that I've seen so far mirror the four themes is a great product it is very easy to use assuming that you figure out how it works in the first place through the docs it does what it's supposed to do it does not waste more resources than needed and so on and so forth so the first and the only Pro is that it is great as long as you don't use the open source version all in all mirror day open source is not something I would recommend to anyone unless you want to try it out before purchasing the commercial version now if you do want to get mirroredly for teams the commercial version I can safely say that it is a great product and that you should get it thank you for watching see you in the next one cheers foreign

Channel: DevOps Toolkit

Views: 12,017

Keywords: Kubernetes, efficient development, local development, mirrord, unlock fast development, devops, viktor farcic, devops toolkit, network mirroring, network stealing, request mirroring, request stealing, review, tutorial, k8s

Id: KewUfKFPlMQ

Length: 24min 33sec (1473 seconds)

Published: Mon Aug 28 2023