The only Cloud services you actually need to know

nobody wants to host their own database AWS isn't doing anything creative they're actually just taking open- Source soft logic wise we can do pretty much everything with these Services nowadays there's so many different cloud services that even just knowing the names of all the services that AWS offers is difficult let alone understanding what they actually do well the good thing is that I'm going to explain to you why most of the cloud services are all really the same under the hood and I'm also going to be giving you roughly my top five services that you should actually understand and know and everything else is just a repackaging of those top five we're going to go even deeper than that and learn about the differences between managed and unmanaged services because I think that's kind of a buzzword nowadays it kind of gets into the whole meaning of what does serverless actually mean we'll also touch on the difference between a regional and a global Service and we'll also be talking about all kinds of different cloud services and providers from the big boys like AWS gcp as well as the startups versel kind of the midsize companies like snowflake data dog and kind of talk about how they fit into the overall picture now the single most important service to know and to know really well is something that you're probably already familiar with you can in a sense rent a computer as a service it's called a VM a virtual machine now technically you might not be renting a physical computer but you can think of it as that cuz you'll have resources allocated to your VM for example some CPU some amount of ram disk space which you can usually add on to so from your perspective you're getting a computer well it turns out that pretty much every single service is built on top of this any cloud service you're going to use is in a sense built on top of VMS if you want to run some type of compute or server it's going to be running on top of some type of machine using some CPU and other resources if you want to store a bunch of files it's going to be stored on top of a computer because even though the files are going to be stored probably on disk well to read and write to those files you're going to need some type of computer to run a database I probably don't have to tell you it's going to require all of these everything is just built on top of computers and pretty much every cloud provider will offer some like this Amazon has ec2 Google has compute engine I'm sure Azure has their own the good thing about these is that they're so extensible if you really wanted to you don't need any of those other services you can do everything you need to using just VMS and so this is an example of an unmanaged service because the cloud provider isn't really doing anything other than provisioning the resources and then letting you do whatever you want to and of course they're not going to like physically mail the VM to you're going to be connecting to it remotely probably via SSH now what if you want to store some files such as images videos and other types of assets that you probably need for your app yeah you can store them on the VM and then serve them from the VM that would work but computers aren't specialized for handling files so a better solution a more scalable solution and just more efficient in general is to use some type of file storage mechanism on AWS it's called S3 simple storage service it's basically an object store that's what they're called and so an object store is kind of like a black box even though I've drawn it as like a white box here but all you really do is read and write files to it you can delete files of course but it's very simple the interface from a programming perspective is really really simple compared to manually handling files via like a VM you don't worry about where they're being stored you don't worry about allocating additional space with a VM you'll have let's say like 100 gbt or something you know you try to upload a file and you run out of space you don't have space on cloud services it's easy to get more space but it's something you still have have to kind of manage yourself as a developer well object stores like S3 abstract all of that away for you you'll never run out of space you'll probably never need to do anything to scale your read and writs now as you can imagine these files are still being stored somewhere they're being stored on a disk somewhere persistently and to read and write the files of course you're using CPU and probably Ram but again you don't have to worry about that this is an example of a managed service because AWS or your cloud provider is managing all the infrastructure for you it's also sometimes called serverless but it's not to say that there isn't a server involved there definitely is you can't do any of this stuff without computers but it's abstracted away from you and that's really powerful cuz all you have to do is worry about your application logic reading and writing the correct files you don't worry about the infrastructure at all it's obviously less powerful in the sense that you don't have precise control over what this service is doing but you usually have enough control like if you wanted your files to be stored in a particular region usually services like this will let you choose the region that you want to they'll usually also automatically replicate files for you so that's another benefit of using a managed service because if we stored a file on our own VM what if the VM dies or you know some type of Hardware failure well if you only stored it on one computer you lose that data that was on that dis well object stores solve this problem by replicating the files for you now object stores are good for storing data but they're not good at storing everything what if you had actual application data not files and images but like you want to store your database data well databases are really really complicated so you usually want to run some type of specific database software like MySQL or some other type of relational or non-relational storage like databases are basically just a really really complex wrapper over these resources particularly dis space well of course if you want to you can just run your own MySQL on top of a VM and you can replicate it among other VMS if you want to like create replicas of it if you wanted to do sharding and scale your database like that you can have some of your data split among the machines that's doable but it's complicated isn't it and so to solve this Cloud providers will usually give you a managed solution for storing your data your database data in the world of AWS it's RDS relational database service that's if you want a relational database there's also Dynamo if you want a non-relational database once again the whole point of using a cloud service for this rather than hosting your own database is because nobody wants to host their own database nobody wants to deal with those issues unless they have to now is a good chance actually to talk about proprietary software in many cases AWS isn't doing anything creative they're actually just taking open-source software kind of like we discussed right now my sequel or postgress is open source to solve this problem of making the user experience better is just to take open source software and then host it and make a manage service out of it that's a lot of what relational database service does but there is something else Dynamo DB if you want to use this type of database like this database service you can only do so on AWS so this is an example of proprietary software if you built your app around Dynamo DB and now you decide that it's costing you too much money you want to move to a different service well you're going to have to migrate all of your data because AWS is the only one who offers Dynamo DB if you want to go to Google Cloud you can't and so this is commonly talked about as vendor lockin generally you don't want to be locked in but recognize that it's probably not going to be a problem unless your app actually starts scaling because if you're spending like a 100 bucks a month on Dynamo DB hopefully your company is making more than that much money but if you start spending thousands and thousands then it becomes an issue and you'll also find that different companies have their own versions of databases for example Google's version of a Dynamo DB would be either fir store or Cloud spanner Microsoft has their own Cosmos DB I think it's called I've never used it the idea is that once you know about one service you you know about one database that AWS offers then going to a different cloud provider is pretty trivial because they have their own version so once you know one you've learned the others like every cloud provider has an object store they all have a VM service they all have their own proprietary or open source database Services I talked about how a lot of services just take open- Source software and then just make a managed service out of it well one of the services that didn't do that was AWS Lambda commonly known as functions as a service this was in a sense taking this CPU component the compute component like if you wanted to actually create an API or run some type of compute not necessarily storing persistent data you would use something called AWS Lambda and pretty much every big cloud provider has copied it Google has Cloud functions I'm sure Azure has their own but the idea is all you have to do you don't worry about the infrastructure at all you don't worry about the VM the disc you hardly worry about the CPU and the ram you set like memory limits and things like that but basically you take your code as a developer and that code is pretty much deployed for you you don't have to worry too much about the environment you just take your code and it runs now the problem with functions as a service this is one of the most serverless services that you can do and the reason I say that is because serverless is really a spectrum there are services that are completely managed by the cloud Lambda would be a pretty good example of that and then there are other services like VMS that are not managed at all and then there are database Services which are half managed and half not sometimes Dynamo DB is very managed but the idea is that there's a spectrum and Lambda or functions as a service abstract away pretty much everything for you but that does come with some downsides for example if you want to manage any type of State in your API which is becoming less and less common with rest apis and microservices you probably won't need to do that but if you do you probably won't be able to use Lambda unless you use some type of external storage because when you run a Lambda function you don't really have access to your disc that's abstracted away from you it can be good sometimes and sometimes it can be bad so it depends on what type of problem you're trying to solve but with Lambda most apis can be solved with this way and if they can why would you ever want to manage your own VM it's a pain so now if you just look at these four categories of services you can build almost anything with these the one big category I've probably been lacking on is networking and I probably won't go super in depth into that for example like load balancers CDN and things like that but logic wise we can do pretty much everything with these services and you probably don't even need ec2 you don't need VMS cuz VMS you can handle everything with anyway but just looking at this we have persistent file storage we have persistent like application data storage and we have compute we can serve our front end or our back end where we go from here is kind of opinionated so there's two categories I kind of wanted to talk about one is just observability in general and I'm not an expert when it comes to AWS with observability but I think the services are going to be pretty similar to Google Cloud but it's about like your logging your monitoring data and your alerts being able to observe how your application is running via like servers side logs or metrics for example if you want to see like how much CPU your VMS are using or how long your queries are taking that would be monitoring data and if you wanted to know if one of these Services went down for some reason or the latency got too high you'd probably want to be alerted of that via some push notification in real time so that's what observability is about and the big cloud providers have done a decent job of giving you tools for observability but there actually are other services such as data dog it's a relatively big company actually and their whole proposition is that they can handle observability better than the big cloud providers even though there's actually a bit more friction in using one of these external tools because there's more work involved in like setting up the monitoring agents and things like that but generally speaking people are big fans of services like data do because they make this stuff actually easier for you it's not just that when you think about something called Data warehouses the big cloud providers have I think aw us is is called red shift Google has big query and they do a decent job some of them do better than others but even given that they're actually are other platforms like snowflake like data bricks that are basically companies dedicated to solving these problems and if they're spending all their energy on one specific area then they think and in many cases they do do better than the big providers tools like AWS have made development so much easier than it was years ago where you needed your own servers still people complain that AWS is really really complicated to use and so now there are companies like versel which pretty much at a high level are rappers around AWS and I don't mean that as an insult because in a sense AWS is just wrappers around like other open source software the whole goal of these Cloud providers is to give you an easier way to use AWS resources just like how these resources RDS functions as a service they're just wrappers around VMS everything is just an abstraction on top of other things and that's okay because the whole point is to make development easier now lastly I wanted to talk about the differences between Regional and Global Services cuz it's kind of a buzzword you see it thrown around a lot the definition is kind of fuzzy for example if we talk about a Content delivery Network a CDN well that's inherently a global Service because you have a world m app and a CDN is basically just a distribution of servers around the world that store some static data usually and so you can't really have a regional CDN now take the other side take a VM for example VMS are inherently Regional you can't have one VM split in five different regions right you just have one VM in a location you could have a second VM somewhere else and a third one somewhere else and you know you could have these VMS under a load balancer serving traffic in different parts of the world that's perfectly fine the service itself is regional though a VM is regional sometimes it gets kind of fuzzy if we're talking about databases like Dynamo DB or Google Cloud spanner and before I even get into those let's just talk about a regular relational database service like RDS and I think Google's is cloud SQL I really prefer the names in Google Cloud they're just a bit more intuitive but with a regional database service you have your world map and you have your database service hosted in a certain region even though it can be partially managed for example like replicas database backups and a lot of things can be managed for you by the cloud provider it's still in a single region and it'll be serving traffic probably around the world so some places in the world are going to receive data a little bit slower usually it doesn't matter though but when you get into Global databases these are super super super powerful because they actually manage sharding and replication for you and they distribute your data all around the world even though it's distributed around the world they will work together generally your request will be routed to the nearest location the power of a global database is huge even though most people probably won't need it it is very powerful in just handling the scale for you and making the latency as small small as possible because data is stored as close to the user as possible kind of like with a Content delivery Network and generally you'll find that Global Services usually cost more and managed cloud services also cost more because the more work that the cloud provider does for you the more they're going to charge you of course sometimes it's worth it though like this is a very hard problem to solve you probably don't want to implement your own Global database by yourself you probably just want to pay somebody to do it for you this is a lot of open-ended discussion but I think that's what's most important I could have gone into an example of one of these Cloud providers but looking at it from a high level I think is most important and getting hands on with it which is something we didn't do is something I'd also recommend to you like don't just watch me talk about this stuff try using some of these Services try building an application out of them

Channel: NeetCodeIO

Views: 76,473

Keywords:

Id: gcfB8iIPtbY

Length: 17min 17sec (1037 seconds)

Published: Tue Feb 27 2024