Fly - NoOps Application Management All Over The World

what would you say if i would tell you that you can run full stock applications your applications and databases and everything else all around the world globally without operations what if we could make it so easy that you would not need to worry about defining applications managing applications managing infrastructure thinking about regions and zones and where your users are and how to scale and all that stuff would you be interested in something like that whenever managing applications we need to think about many different things we need to define what those applications are we need to figure out how to install them and how to upgrade them we need to think about hardware or infrastructure we need to think about regions we need to think about scaling we need to think about monitoring and so on and so forth very often we want to go deep we want to figure out how aws works or azure or google or whatever you're using we want to deal with networking and storage and regions and kubernetes clusters and what is running in those clusters and how we monitor those things and how we do logging and all the rest around it because we want to control we want to be able to fine-tune the results we want to get the maximum out of what we have at other times we want simplicity we want a solution that will do most of the heavy lifting and we just press a button or execute a command or a couple of them and off we go we are ready and in those cases we might adopt let's say shipa that allows us to just run applications without really dealing with manifests and definitions and this and that or we might adopt something like google cloud run or aws slide sale or one of those vendor-specific solutions that give us either an easy way to run applications on their own infrastructure and their own services or maybe something like shippa that allows us to run applications in an easy mode wherever we choose to run them but we need to figure out how to make that place where they're running usable but all those solutions are missing something either they're too opinionated or they do not deal with infrastructure or they are blocking us into specific vendor or there are many reasons why we might or might not choose any of those i will assume that you want a couple of things you want to run your applications easily you do not want to deal much with them you want to run them globally you want your applications to be where your users are so if you have users in asia they should be somewhere in asia and if you have users in u.s somewhere in u.s and in europe and africa or simply put whatever those users are is where your application should be because that provides low latency better experience and a potentially cheaper operational cost for you and i will assume that you do not want to deal with infrastructure so if you want an easy mode for applications if you want to run them globally and if you do not want to think much about infrastructure clusters servers whatsoever then what i'm about to show you might be a good solution or it might not be we'll see and the product or that it's not really a product a service that i'm going to show you is called fly you know like flying and instead of me explaining you what fly is and how it works i'm going to jump straight into the demo and then i will be commenting things while they're happening and after that in the end we will talk about the experience and see whether flying might be a good choice or not whether it's horrible horrible horrible solution or the most amazing one i ever saw i mean it's rarely either of the two extremes it's usually somewhere in the middle anyways let's jump into the hands-on section of this video and see fly as a potential solution for you to run applications globally everywhere or nowhere depending on where your users are without much operational cost or almost no operations and with potential cost savings there are only two requirements for you to run applications through flight and that is to install their cli and to register i'm going to skip those two because hey i already have the cli and i already registered i already signed up actually and i'm going to jump straight into logging into the service and then using it with my demo application if you're already inside of a repository that contains code of your application maybe dockerfile even though that's not a requirement all you have to do is execute flycattle lounge and then it will guide you through a couple of questions that might be important even though some of them are silly i believe anyways it will guide you through the questions that will potentially make your application run globally so the first thing i need to select is the region where i want my application to run later on i will be able to select additional regions and if you ask me it is silly that i'm even asked that question but i'm going to comment on that later for now just remember that you can select the region and let's see what else can we do actually there is nothing else that we need to do or should do except confirming that yes i do want my application to be up and running i do want fly to deploy my application and as you can see it figured out that there is docker file in my repo so it is building the container image and it is building and building and now it is pushing that image to a registry and let's see what's happening after that it created a release and then it deployed that release somewhere i do not necessarily know where that somewhere is probably somewhere in amsterdam or around amsterdam and then it is monitoring the deployment or in other words it is waiting until the application is healthy and it is running and it is doing whatever it needs to be doing and at the very end we can see the logs of the application that's about it the application should be up and running somewhere around amsterdam there are a few additional commands that we might or might not want to execute like we can see the information about the application there's not much to see there except maybe the version and the status and then there is the status of the application which to me is very confusing because hey influence status are more or less the same thing and the information is similar except the status gives us a few additional bits and pieces of the information of what's going on specifically we can see the deployment status which you kind of could see in the info and we could see instances now instances are important especially later on when we scale the application now the important thing here is to focus on the instances there is one instance and it is critical so it is most likely not really running and doing what it should be doing and that might mean that the cli did not figure out really really really what my application is about but fortunately i can modify the settings i can fine-tune what my application is and the way how to find unity is through a toml file so when i executed fly launch it created a tunnel file and let me output it on screen and see what we got there information there is kind of hit and miss some parts are important some parts are interesting some parts not so much and i'm not going to go into details of what every single entry means and what are the entries that are not in this file but we could add but focus on a few things that matter to me right now to begin with the internal port is set to 8080 that's the default and my application happens to run on 80 so that's probably the reason why the status of my application is critical instead of whatever is not critical whatever the status is that says hey everything is great so i will have to change that but let me see whether there is anything else interesting over here if i would need to highlight one important thing here is the services concurrency entry that has the hard limits set to 25 and the soft limit set to 20 and the type is set to connections we can have other types but what matters in this context is that my application is most likely going to scale up when there are more than 20 concurrent connections or requests to the service and the hard limit is 25 meaning that it will not scale automatically by the time it reaches the soft limit but after a certain period of time but if it reaches the heart limit it should scale up immediately and the rest is blah blah blah i'm going to skip that not really interesting what i want for now is to fix my application to make it run and as i already mentioned the issue is probably in the port so let me edit that file and fix the port and now i can deploy or to be more precise redeploy the application once we launch the application every new release hot fix bug fix whatever we want to do with the application when we want to deploy a new release is done through fly cuttle deploy and in this case i did not change the code of the application but i did fix the configuration of fly itself and that deserves a new deployment so let me just do it and the process is more or less the same except that this time it is not creating a new thermal file it is building the container image actually there is not much to build because most of the layers are all the layers already exist and then it will push a new release to the registry it will deploy a new version or new release of the application and it will wait until it is up and running and the section with distances is kind of misleading it says that there are two in total even though there should be one and one is passing one is critical what that really means is that the previous release is still running somewhere and it is still critical but the new release that i just made is passing so my application potentially should be up and running and i can double check that by opening it in a browser and yeah there we are big success the application is up and running it is in the browser it is doing all the things that it should be doing and it is running somewhere around mapsterdam and i'm going to fix that next but before i do let me comment quickly on how the application is accessible it generated a sub domain for my application which is essentially the name of the application dot flight dot dev and the name of the application is unique if it's not unique then that would cause trouble so we have a unique name which is so fixed with the flight.dev and that's the address of the application others through which we can access the application of course that can be customized you can have your own domain if you want to but for this demo i don't and since my memory does not serve me well and i suspect that i will need to use that address later late travel storage in an environment variable also before i go into moving to multiple regions let me double check whether i can see the logs and whether everything looks okay and of course i can it's just fly catalogs and you see all the logs of the application and so far i must say that i'm impressed with simplicity if simplicity is what you're looking for this is actually extremely extremely good i did not worry about hardware i did not worry about application the only tiny tiny tiny problem i had is that i had to tell fly that it should not use the default port but the product i have for my application which is 80 instead of 8080 but apart from that so far this might be the simplest way to manage applications i've seen it might not be the most powerful one it does not allow me to tweak much what i need what they want but that's probably not the goal this is not a service for people who want to be in full control who want to get their hands dirty and so on and so forth this is the service or it smells like a service for people who just want to write code and run their applications and those applications to be global now that i mentioned the word globally one more time let me take a look at the regions where my application is currently running and then i'm going to extend that to maybe deploy to a couple of more regions and this output shows another really really cool thing so my application is running in amsterdam because that's what i chose but fly selected a couple of additional regions as a backup solution in this case that's frankfurt and lhr i think it's london or something like that doesn't matter a couple of regions around close to the region i selected and the reason it did that is that if the region if pump serum goes down those are the two regions that fly would automatically reprovision redeploy the applications so the downtime is minimized as much as possible so think of this as fallback regions and that's a really really nice thing now let's say that i have users across the globe maybe in europe and u.s and asia let's say and i want users to have the best possible experience with my application and that among other things means low latency so the closer the application is to the end user the better it is the faster it will respond the faster it will load the latency is lower and so on and so forth generally speaking we want our applications to be as close to users as possible so i'm going to add two more regions to my pool of regions where the application is running i'm going to add let's say seattle for u.s users and hong kong for asia users and we can see from the output that the region pool now contains those three regions and there are six additional backup regions so for every region that we add two more backup regions are created as well and that's a decently good fallback uh strategy that allows us to lower the downtime if uh that's unavoidable and it is unavoidable we are going to have downtime so let's lower the chances of experiencing downtime uh by adding additional regions and that's absolutely awesome especially since doing that did not really create additional work for me as a developer or whatever role i'm having right now now let me take a look at the status of the application one more time and see how did those regions affect the application itself and this is kind of strange or buggy or non-consistent to begin with it still shows that the status is both passing and critical even though the new release is working the old release is still hanging in there somewhere and i don't understand why it is still critical it should have been removed by now but i will ignore that what is more curious is that one of those two regions are being added to the list of instances so soon probably within a couple of moments my application will be running not only in amsterdam but in hong kong and i'm not sure why that is happening because i'm the only person using the application right now and i can guarantee you that i'm not in hong kong or anywhere near i'm in barcelona so let me execute status again and see what's really happening what's going on over there and surprises keep piling up now amsterdam is gone and hong kong is still pending which is bad for at least two reasons first of all how can you remove an application that is currently running in amsterdam while creating a new region and without really waiting until the application that new region is up and running this simple operation of adding regions just created downtime of my application because it is still not running in hong kong and it stopped running in amsterdam so it is not running at all right now one is shut down and the other one is yet to start so let me see the status one more time third time should be the charm i guess or whatever the expression is now it is up and running my application is operational again so that's settled it's working but what i do not understand is why hong kong the only user that use this application so far is me i send the request to the application couple of times and that request was sent from barcelona it should have stayed in amsterdam because i'm the only user right now and the closest location among the regions that i selected is amsterdam it is not hong kong so it is not really calculating well where to place application how to make it accessible globally to the users and how to make it as close to users as possible because i'm not in hong kong or anywhere near hong kong or in other words among the three regions i chose it selected the one that is furthest from me and not the one closest to me now let's see what we have in the options of scaling how what can we do to scale this application to be more responsive to accept more users and so on and so forth and the first step is to take a look at the current status of scaling by executing fly cuttle scale show and we can see that there is only one instance or one replica of the application that's probably okay because i'm the only user right now but what i want to check is whether it will scale automatically and i will get to scaling in a second for now i also want to see the options of the sizes right and we can see them by executing fly cuttle platform vm sizes and we can see that i am currently using shared cpu because that's the three tier but you can select between the options of having one cpu two cpus four cpus eight cpus if your application needs more than eight then probably fly is not a good option for you anyway you're already running at scale that is probably not a good fit for a fly anyways in other words the options are good for the target audience probably even the atc is too much and now i'm going to put it to the real test or a simulation of a real test i am going to use siege to bump my application with requests i will create a process that will run throughout 300 seconds or five minutes and during that period of time it will bomb the application with thousand concurrent requests during five minutes and i really want to see what will happen with the application when i do that now this will take a while it will take approximately five minutes and while waiting for this process to finish this might be a perfect opportunity for me to tell you that you should subscribe to our channel but you already know that and what you might not know is that you can join the channel or donate to the channel either two options are great and your support is really what makes this channel run and what keeps the expenses of the channel to the minimum so please consider joining or donating to the channel and now let's fast forward to the end of the process and see what's happening at one moment siege started throwing connection time out messages and that is normal if my application is running still a single replica it's normal that that influx of requests cannot be handled by one replica of an application especially a silly application is mine so that was to be expected i'm not blaming fly for that i might be blaming fly if it did not scale up my application and it detected that there is a huge amount of requests coming in but we are going to get to that for now let me see what we got here 97.58 of availability again that's normal that's to be expected a few other bits and pieces of information from siege i do not care about those for now but they do care is the output of fly cuttle and if i look at the instances i can see that only one instance is running fly did not scale my application when it detected that there is a huge amount of requests coming in also it is still running in hong kong which is beyond my understanding and i really don't have an explanation why hong kong but hey you cannot have it all in this case it's either a mistake i made or fly is not having same defaults you can argue either way i could argue that fly should scale automatically by default and others might say no you need to set it up explicitly obviously the opinion of others prevailed if you want fly to scale automatically you need to configure that yourself it does not scale automatically out of the box and i can confirm that by executing fly cuttle out to scale show and we can see that the scale mode is disabled and that's not a big deal i can easily change that by executing fly catalog to scale balanced and then i can specify what is the minimum number of replicas and what is the maximum number of replicas and minimum number can be anything from one to whichever number you want and maximum to whatever number you need it is disappointing that it cannot scale to zero replicas but we are going to talk about that later for now i want to see the status first did the status change after i changed the scaling preferences probably not but let's check it out yeah the status is still the same it still thinks that i'm in hong kong it is still running one replica but that's to be expected because right now nobody's using application there is no reason for it to scale beyond one replica if nobody is using it i will do another round of five minutes of bombing the application with thousand concurrent requests and see what's the output was the result of that since five minutes would be too much for you watching me watch the screen i will fast forward and you just remember this is the time for you to contemplate whether to join this channel or donate something to your channel in any case let's fast forward to the end of the process and see the outcome the siege freaked out again this time i think that the problem is in my machine it cannot handle really sending so many requests my poor desktop is not capable of handling that probably in any case i do not want to go into availability of fly at least not right now what i do want to see is what happened with the regions with the number of instances did it scale up and this is both fantastic and really bad news the good news is that it scaled up automatically now two replicas of my application are running the bad news actually there are a couple of bad news first of all is that it selected seattle as the second region even though amsterdam is also available and i am closer to amsterdam than both seattle and hong kong so it did scale up and it did distribute application to multiple regions but it failed to figure out where the users are and today i am the only user it should have selected amsterdam randomly you know if there are three available regions and you're running in two regions then there is 66 chance that one of those two regions will be amsterdam statistically it should have been runs to them even if it does not know how to distribute the load and where the users are and that's the main thing that it advertises fly says that it will run applications closest to your users and in this case it is running anywhere but closest to my users considering that i'm the only user the second bad news is that scaling does not really work properly if you remember the tomophile said that the soft limit is 20 and the hard limit is 25 so if i'm having thousand concurrent requests it should have run at least 40 replicas you know thousand divided by 25 that's 40 right simple month isn't it now what i suspect is that fly would eventually scale to 40 replicas as defined but probably if i run those tests over a much longer period of time but that's not an excuse good enough because five minutes is a relatively long period of time it's not that i run thousand conquered requests over five seconds or half a minute and then fly would say hey maybe this is a temporary spike five minutes is more than long enough for it to figure out that it should run more than two instances of my application so did the scaling work yes it scaled up did the scaling work properly no it didn't now knowing that scaling up works kind of not really but it does something it scales up to a random number very low number of replicas what they want to see is whether it will scale back down after a period of inactivity and to which region it will scale down so what i'm going to do is go and get coffee and maybe have some snacks or something like that and come back to the situation maybe 15 or 20 minutes later and see whether it's scaled down because what i do know is that it is not fast it is not scaling up and down within minutes or seconds it needs a lot of time so i'm going to give it a lot of time and since i know that you're impatient i will fast forward the video to the end of those 15 20 minutes and see what we're going to see so here we are it did scale down after a while and now i'm running one replica only there are no requests coming in so it's normal that it's one and not two or three or five because there is almost no no actually there is no traffic to the application the sad part in this story is that it cannot scale to zero replicas because that would be absolutely awesome that would make it serverless of sorts but hey one replica is better than two or three or four or five there is one thing that i'm really curious about i want to know whether the information about additional regions and the information about scaling that i specified through the cli is persistent to autumnal file because what is the point of having config file if the changes to the application not recorded in that config file so what they want to see is by the regions and the scaling information is in the configuration and it is preserved and what's or not so no next nine never gonna happen it is not really preserving the information it gathers from cli commands like scaling and adding regis to the thumbna file and that makes it half useful half magic i do not know what's going on uh why do we have config file in the first place if cli does not preserve changes to the information about the application that thomas file so it's another hit and miss type of situation wait i forgot one thing that might be important i forgot to show you the ui in case you're the type of person who likes graphical user interface pretty colors and all the things that come with it there is an overview with the basic information about the application you can see the the addresses and the status and the host name not really that important i mean it is important if you don't want to use the cli but then i'm not sure that you would use fly in any case so cli is really the focus of fly and this is just the replicated information of the things you get through the cli there are a couple of useful commands that might get you started that's kind of nice let's see what else do we have metrics you don't see much in this case because i just launched a new application but you have basic metrics let's say basic and if you want something beyond basic you should probably install prometeuse it does not come out of the box there is no especially easy or different way to install it you would install it as just another application in fly and then configure it one way or another i'm not sure how because i think that this goes beyond the simplicity that fly is trying to offer what i'm really trying to say is that i would like prometheus and grafana to be baked into the service and not force me to install it separately because simplicity is the main thing about life it's not simply if i need to do extra work then it's not so tempting anymore and there are certificates in the activity and settings flies mostly about using the cli and the web ui is there simply because everybody should have web ui just in case here's what's happening here is what uh fly is actually doing it is creating micro vms which are highly optimized types of virtual machines and it is deploying containers in those machines it's probably using docker or maybe heroku or something doesn't really matter because for you as a user that's not important what does matter is that for every instance of your application it is creating a micro vm and it is stuffing it is putting container running in that vm which is not necessarily the most optimum way to do stuff you would normally behind the scenes transparent to the user run some kind of scheduler that would probably be kubernetes that would further optimize it that would make it faster that would make scaling better but that's not what it's doing and the only upside i can imagine is that vms are more isolated than containers so that makes it slightly more secure but less optimum than if they made the right technical choices which i believe that they might not have but then again i might be wrong in any case for you as a user all those things do not matter what does matter is that you have a container built by fly or built by you and that container is running in one or many regions depending on what you configure it and it is scanning up and down not very efficiently but it is still better than nothing so should you use fly or no that really depends on whether you're the target user or not and i'm not 100 sure who the target users are my best guess is that the perfect user is a person or a team that does not want to deal with deployments does not want to deal with infrastructure does not want to deal with regions does not want to deal with scaling or more or less does not want to do anything but write code and have that code converted into binaries and containers and running somewhere and serving its users it is not doing any of those things especially well you can have better performance better operational quality lower down time better almost everything if you do it yourself using let's say kubernetes especially if you use kubernetes in aws or azure or google or wherever you're running it so you can do better than what fly is offering but fly is giving you no ops type of approach so if you do not want to deal with complexities of doing all those things yourself then fly is actually an excellent solution it might not be the best one my favorite would be google cloud run but it is very close to that it is no ops type of solution that does not excel at anything except that it gives you features that you would need to work hard for making them yourself and i suspect that it is mostly for smaller teams smaller guys you know this is not for a huge enterprise this is for a small team maybe some startup maybe five ten fifty people top that just want to focus on code and forget everything else everything else should be handled for them it's a no-ops approach to ops there are two major downsides first of all it is not serverless it does not scale to zero replicas which is a huge missed opportunity that's what makes me say hey google cloud run is somehow similar but still much better solution than this and another thing i don't like is that it wasn't simplified even more like why are you asking me what is my initial region you should just accept the image of my application build an image and accept it and run it wherever the users are when the first request comes in put it in the region closest to that user and then scale up and down move to different regions and do whatever you need to do there is no real reason to ask me for the region it's supposed to be you fly service that should figure out where the application should be running it shouldn't be me i do not know necessarily where all my users are it should select regions for me not force me to select regions and that would be unacceptable in a different context but flies for users who do not want to care about tops and if you don't care about tops if you want no ops then do not ask me even for the regions and but we are on the subject of do not ask me then i do not understand why scaling is not baked in by default why do you not scale by default and then scale only if i tell you to do so and if i tell you to do so then please record that in the thermal file that you're using in the first place all that being said i might have sounded more negative than i am i think that this is a service worth trying if you're the target user so try it out it is awesome after all it's just me i'm capable of finding faults and then bad things in absolutely everything i touch and i'm not trying to say that fly is not a great service it is a good service maybe not great but it is a good service and it is worth a try you

Channel: DevOps Toolkit by Viktor Farcic

Views: 1,723

Rating: 4.9436622 out of 5

Keywords: no ops, noops, no-ops, fly, app, application, regions, devops, devops toolkit, review, tutorial, viktor farcic, fly.io deploy, fly.io firecracker, fly.io review, fly.io locations

Id: tuPmhciyfIA

Length: 34min 42sec (2082 seconds)

Published: Mon Sep 27 2021