Cloud Computing - Abstraction

please go to Eli the computer guy calm in order to view schematics code and more for the projects that you are learning about welcome back so today's video I want to talk a little bit more about the concept of abstraction especially in regards to the cloud computing environment so we start talking about abstraction basically what we're doing is we're taking something like the old fashioned idea of a server and we're trying to break out all the functional constituent parts and actually interact with those constituent parts on their own so where in the past you would have a physical box you would have a physical machine with a CPU and a hard drive and a power supply and all of that and then you would install an operating system onto that physical machine then you install the services onto that physical machine and basically everything was on that one physical machine which means if anything failed on that one physical machine everything failed and that one physical machine right so basically if you have all of these moving parts in that machine right you've done all your your configurations you've setup the operating system you've done all kinds of fancy things and then literally the CPU fan dies well everything just goes away that is an inefficient unreliable way of trying to build infrastructure so the concept that was created was kind of this idea again of abstracting all the components out so instead of simply having one server and that server has the processor and the RAM and the storage and the operating system and the services and all these other things the thought is like hey what if we all break these out in their own constituent components what if we have the compute Hardware over here so this as the CPU and this has the RAM and then if what if we interact with the storage as its own isolated thing and that storage is over here so you have the compute processor here and then you have the storage here and then you think about well what if we can take the operating system and we can take that operating system and essentially we can just put that into a file so it's not married to any particular hardware it's a file that can be moved around like any other file on a computer system so now we can shift that around to two different pieces of computer hardware what if we did that and then you know with some of these services let's say backup services or maybe authentication services that type of thing why does that even need to be part of that that operating system that we've created can that be a service that simply provided by a provider off of the internet right so now instead of having one box worth with everything on that one physical machine and if there's any failure on that one physical machine everything goes down now we've broken things out to their own constituent parts so for compute you know a what's called a VMware cluster so a hypervisor cluster and so you can have a cluster of physical servers that are now running the compute they're actually running the instances of the virtual machines if any one of those physical boxes fails you can simply you can simply roll over the instance on to another physical box and the system stays up and running again as far as storage goes you can use something called a storage area network ace and so this is actually a multiple storage of devices within a cluster they're all synchronizing data back and forth doing load balancing and that type of thing and basically the the instances of the VMware hypervisors are able to communicate with us and they're able to push and pull information if a single device in the sand fails it doesn't really matter that fails and then the hypervisors are simply able to communicate with a different hardware device in the sand and so in this way you're able to give your infrastructure a lot more reliability you're able to get your give your infrastructure from a lot more resiliency and you can also do things to make your infrastructure a lot more efficient kind of think about this as the subcontracting world if you own something like a building so in the old days if you go back 20 or 30 years and you own let's say an office building so you are the owner of the office building so you would hire the security guards for the office building you would hire the the facilities the janitors and such for the office building you would hire the people to do the maintenance for the office building I think about this that's a pretty inefficient way of doing things like you're an expert you're an expert being able to rent out office buildings and buy office buildings but you might not be an expert on janitorial services or security services or anything like that so we think about this idea of abstraction basically that's what we have in the subcontracting world where now we have building owners and they will go to a security company and they will hire security guards from the security company to provide security services for their building go to a company that specializes in security and simply hire their staff then you go to a janitor company that all they do is janitorial services and so hire their janitors to be able to do janitorial services for your building so on and so forth and that's kind of what we're doing in the computing world right instead of having the one server and so the one servers hard drive you know that's that's the storage for the entire server the the CPU the RAM you know that's the processors that's a compute for the entire server and if there's any problem and any of that everything kind of goes to hell well the cool part now again as we start abstracting things out now your storage your storage is dealt with by a SAN storage area network or maybe a storage service from someone goes your or AWS and so basically you now go to it's almost like subcontracted hey I need storage services from you so I'm going to go to an infrastructure that is built specifically for storage you're not gonna play video games on ass and you're not gonna do VoIP calls on ass and you're not gonna do whatever else on ass and all ass and does is historic and so it's really really really really really good as to work and it makes it more efficient makes it more reliable makes it more redundant again we can create a cluster of compute servers so basically these are servers more or less all they have is a CPU and a ram they have a little bit of storage in there but basically more or less all they have is a CPU and a ram and so they are very very very very good at compute and so they will now only be doing the compute and so this is a way by abstracting all of these different functional functional tasks out you can make an infrastructure that is much better gives you better return on investment and overall will work better for your clients and your users at the end of the day so to explain abstraction a little bit better we're gonna go over to the whiteboard I'm gonna show you three different examples of different types of infrastructure where abstraction can be used when you're thinking about modeling that infrastructure so I'm going to show you the standard virtualization so this is where you have something such as a VMware hypervisor cluster or a Citrix hypervisor cluster that communicates back to a sand and is able to move instances of server virtual machines around so that's one way of thinking about abstraction I'm going to show you something else called software-defined networking so this is where you take abstraction into the world of networking and you actually abstract out all the functional components that are required in order to send and receive data over a network and you actually build an infrastructure in a different way than we have in the past and finally I'm going to show you something called a serverless architecture and a serverless architecture is where essentially we get rid of the old-fashioned idea of what a server is you don't really have Windows servers you don't really have Linux servers you don't have UNIX servers you simply have a services that you're able to access in order to perform certain tasks so compute becomes a service storage is simply a service even things like authentication you simply use an API call and that is now a service you're not worried about things such as antivirus programs and backup routines and all that kind of stuff you create your product that is able to use these services from other providers and all you have to worry about is the code that you're writing so that let's go over to the whiteboard so I can try to give you a visual idea of how this abstraction works within a different context because again it's really cool and I think it'll make a lot more sense so here we are at my whiteboard and the first thing that I want to do is kind of go over the old concept of a server so we're talking about virtualization you know what are we talking about here so basically we have a server in the past and with that server everything was in the box so the CPU was in the box the RAM was in the box the storage was in the box the operating system was installed onto the physical machine and then the services were installed and configured onto the operating system and then if you have something such as a backup system right that's more or less physically connected to the server itself and so that system would then back up the server and so all of this is tied and connected to one physical machine so the CPU the RAM the storage the OS the services and the backup is all this one physical machine and the problem with this is is that something such as a CPU fan fails or a power supply fails or anything else basically the entire machine goes down right there is no real redundancy here there's there's nothing happy you know basically if one small problem happens everything kind of goes to help the other problem here too is in order to make any improvements on this type of system you actually have to have significant downtime so let's say you want a faster CPU or you want more RAM or you want better storage right best-case scenario you have to turn the server off and then basically plug in the new CPU or the new Ram boot everything up and hopefully everything works so so best-case scenario we're talking about maybe it's 20 or 30 minutes of downtime worst case scenario back in the day since the operating system is tied to the physical Hardware sometimes you would have to reinstall the operating system undo completely brand-new hardware then to a massive tedious horrible migration process and that literally may take hours and hours or even days to accomplish so basically having all of this connected into one box married to one box is was a pain in the butt even in the best of times and so what we have now is we have abstracted out the different layers we've abstracted out the compute we've abstracted out the storage we distract it out the OS and we've even abstracted out these services and the backup into their own components so that it is easier to be able to deal with and you have more reliability and redundancy so we're talking about compute now what you have is you'll have a cluster of something like VMware hypervisor so basically these are all than us basically they would look like physical servers like you think of them and essentially they have CPU and RAM on them so they have whatever processors you have you they have RAM and then then they have on top of that something installed called a hypervisor so basically a hypervisor is a very very very special the operating system that it's only responsibility is running instances of virtual machines so you're able to turn an operating system into a virtual machine and this hypervisor that's an operating system that is then able to run those virtual machines so basically then you have these hypervisors and they're all connected together there's some kind of management so there's some kind of management software that it allows all these hypervisors to be able to communicate with each other and so all these hypervisors do is basically all they do is the compute all they do is they run the instances of the virtual machines so you take your Windows 2012 server you take your Linux sir possibly take your UNIX server and basically those simply become virtual machines so you have a 2012 server and a 2016 server and a Linux server right and now these can be run on any of these hypervisors and the other nice part about it too is again if there is a failure here then this this hypervisor system is able to just automatically restart up the instance on a different hypervisor so you don't even see any downtime or if you want to take one hypervisor server offline and put in a better one so this has a better CPU on it literally all you have to do is you have to add the new the new virtual server and then you simply take out the old the one that was there and now now you can run your instances very easily again you have almost no no downtime so then now we have these instances of operating systems and they need somewhere to run and so they will then be running on something called a sand so a sand is a storage area network and basically what this is is this is a number of different storage devices so these are computers and they do run operating systems but all they are responsible for is story so basically they do stories very fast do they do stories very quick they do stories very reliably we're very reliably so within the sand these different these different storage devices they will communicate with each other and they we'll replicate data so they will replicate data amongst themselves they'll make sure everything is backed up they'll make sure everything is redundant so on and so forth and so what you have here is these servers here that have the hypervisor installed in order to run a virtual machine they go to the sand and they say give me this virtual machine and then they are able to then run that virtual machine so now we have the compute the compute is abstracted out the operating systems themselves are abstracted out and the storage itself is abstracted out and then you can have really cool things that place I keep doing that then you can have really cool things where on the cloud you can then have additional services so there's come to something called dr AAS because everything's an AAS anymore but actually this is really cool this is really super sexy there's something called disaster recovery as a service so it as disaster recovery as a service can be is you have these instances of your servers and so with this this is kind of like the upgraded version of backup where the disaster recovery as a service it will create copies it'll create copies of your virtual servers so your 2012 server it will create a copy and a 2016 server it will create a copy or a Linux server it'll create a copy and it might even create multiple versions of it again we talk about things like incremental backups or time backups and so it will create these versions it'll create these backups and one of the really cool things is that then all this is up in the cloud and if your entire hypervisor cluster fails your entire hypervisor cluster fails again let's say a tornado hits it or something else what's really cool is you can actually spin up these virtual machines within the disaster recovery as a service environment up on the cloud and they can start providing the services that your hypervisor cluster was providing before and keep everything running literally even if your entire building gets taken out by a tornado and so that's why this whole concept of abstraction is really cool because again once operating systems are as easy to move around as files then you can start to do this cool stuff where you can literally have them copied online and if your local physical machines fail they can simply be spun up online they can be providing services until you can get the local machines back up and running and then you can move them back and everything's pretty pretty simple so this is the basic concept of a virtualized abstraction virtualized idea of how abstraction works and how you can divide out all of these different components into their their functional parts so from here let's talk about Sdn or Software Defined Networking so we're talking about Sdn it's just that Sdn Software Defined Networking oh yeah I'm good at writing anyways you know come here for handwriting so basically what we're talking about with Software Defined Networking let's talk about the old days of how networking is done so basically in the old days you had your router you had your switches right so these these are the hardware so this this is Hardware on the routers and switches they have their own little operating system so they have their own operating system that says how packets should get rounded that kind of thing says how routing protocol should be dealt with out to have a deal and then on top of that operating system you then have the rules right so basically again we go back to the idea of each physical machine has the physical hardware it has the operating system it has the rules in order to be able to administer things or if something fails you have to go to each individual device in order to be able to deal with it and of course you know that that can be a it can be a problem so if you if you have a switch fail that means you have to have a high-priced technician and go to the switch unplug the old switch plug the new switch in make sure all the configurations or upload all the configurations and everything is running properly so there was a concept with Sdn is it basically the idea is how can we abstract that out again so if we have if the hardware and the operating system and the rules are all tied to one physical machine is there any way we can break that out abstract that onto different levels so it's easier to manage and deal with it so what they have done is is again kind of like the virtual world they've been able to go out and they've been able to turn the hardware more or less into relatively dumb equipment so basically this equipment this will have a basic OS installed and then what it'll have is it'll have some way to communicate back with a management server so what happens is you create a management server again depending on what particular vendor you're using there will be a management server and so what happens is if you have a router or if you have a switch when you plug that into the environment they then communicate back with the management server and they say hey what are my configurations supposed to be and so then in the management server you have a bunch of different configuration files and depending on what that router needs or what that's which is supposed to have the management server is then able to offer those configurations so now what you're able to do is you're simply able to plug equipment into your network your equipment then reads everything coming in communicates with the management server says hey what information do I need and the management server is then able to provide that information so you don't have to go to each individual networking device to be able to code things in you're able to simply go to the management server and from there you're able to write all the rules that can then be pushed out to all of the different networking devices the nice part about this too which is really cool is when you start going into the virtual world right in the virtual worlds when we start talking about virtualization and instances of operating systems communicating with other instances of operating systems we still have the DD weeks we still have issues with firewalls we still actually have to have that networking layer right and so one of the cool parts about software-defined networking is with the management and with the rules it can also communicate with the virtual machines about how they are supposed to communicate on the network and be able to communicate with each other and so that's one of the great parts here is this management server now with with all the rules that you can you've created it can communicate with everything on your work with how it's supposed to communicate with each other and it makes life a lot easier whether it is the actual physical routers and switches that you're dealing with or whether it's virtual routers or switches now everything again is broken out it's abstracted into those different layers and so this is where software-defined networking can be very very useful especially in modern environments now finally we go to server LS architecture server this architecture I think is cool I love serverless architecture though I'm sure a lot of old school system administrators probably hate it because basically it means their jobs more or less are going away so again we start thinking about abstracting and we go we go to the furthest levels of abstraction and so you start thinking about is like well why do you need really need virtual machines at all right so when we talk about a virtualized environment we're talking about servers let's layer this idea okay well we're gonna have all these virtual servers we're gonna create clusters of virtual servers and so a lot of ways the idea of a virtual environment basically looks like the old-fashioned way of dealing with servers and it's just a little bit fancier so there's a concept here of a service architecture of hey what if we stop really worrying about those servers you know what if I stop what if I stop worrying about you know the backup routine of the server so what if I stop worrying about antivirus on the servers and permissions and all that kinds of stuff on the servers what if what the server server offers because what I really care about that the server offers is our particular services right you know what I really compared care about is just simply compute I don't really care about Linux computer windows compute I just want compute to do a task so let's say I want to encode a video I don't really care whether i encode a video on Linux or Windows I just need a video encoded so I just want the compute service of being able to encode a video or maybe or maybe do something such as you know resize images so let's say you're creating something like the next Instagram people are uploading photographs to your app and you simply need compute literally just to Reese the image again you don't care if it's windows or resizes the image you don't care if it's Linux you just need the image to be resized and so the idea of service architecture say yeah what if can pew can just be its own box again what if what if storage again we start talking about storage we talked about sans you know do I want some kind of local sand because I'm worried about security or do I simply want to use AWS use one of their block storage do I want to use something that as your has or you know something on some new new fancy startup has right you know basically what if I can just look at storage again I don't want to worry about raid cards I don't want to worry about you know SATA versus scuzzy I don't want to worry about you know solid state versus platter drives I just want story I just want storage to be a box and then you think about even things like authentication right basically people logging in so that they get certain permissions do you really care if I authenticate is provided by Microsoft Active Directory or some kind of act directed Active Directory services form from Linux or anything like that really all you want is authentication you wants people to be able to log in username password or biometrics or whatever and then they want you want to beat them to be able to have some kind of access control key and that's it so what if what if access can then become its own box service and basically that's what you start getting in the service architecture as you have a computer somewhere you do have a server you have a system somewhere that is basically just running code so you write code and either no J yes or Python or something else right and so basically let's say you again you're going to create something like an Instagram so you're going to create the back-end for an Instagram and so somebody has their little client computer and they take a picture of a house right so they take a picture of a house and they want to be able to post that so other friends can see it well they're able to connect to wherever you have your Python or your node.js script so right they take a picture with with the app on their phone and then they communicate with whatever servers running your Python or nodejs your Python or nodejs go okay here's the picture so what I want to do is the first thing that I want to do is I want to store it onto something right so I so you're uploading a picture so it needs to grab it it needs a store it somewhere so that so it doesn't get lost in the process so it's gonna store it somewhere and then what's going to do is it's going to say okay once it's stored somewhere then I want to run compute against it to basically be able to do something like resize it so you take a picture and as I say it's a 20 megapixel picture and your version and the version of Instagram only wants 8 megapixel pictures so it wants to resize from 20 down to 8 so what is going to do is it's going to going to then make a call from the server and it's going to say hey compute hey compute we have a file on our storage and we want to take it from 20 megabytes down to 8 megabytes so you do the resizing for us and then once you finish that resizing process we then want you to restore what you want we want you to store it into a different bucket or a different folder onto storage and so that's that's what that happens right so you're not worried about what's what specific software that's being running you're not worried about the operating systems being run you're not worried about antivirus or anything else like that you're just saying ok a picture picture is going to get stored here compute is gonna get called to resize it once that image has been resized we're then just going to store it into a permanent place on storage and then and then basically what's going to happen is so that person is now shared that picture to all of their friends and so somebody else is sitting you know at their little smartphone oh this is like a horrible phone so ice so they're sitting at their smart phone and then what they do is they say ok I don't want to log in and see what my friends have friends have updated so they commute ups there's a little weird here so anyways so they communicate with the server that has the Python or nodejs or whatever else and they say I want to log in so that I can see pictures and so what happens is then the Python the nodejs this system then goes to the authentication service that's being offered and says ok you log in give me the username password that gets passed to the authentication service the authentication services says ok this is Bob says ok this is Bob and so then it's Dennis says ok so bob has access to these different pictures and so Bob now has access to be able to view the picture that this person over here uploaded and then was rien coded and so that's what I'm talking about when we start talking about service architecture is with all of this the main thing that you have to be thinking about where in here did I talk about Linux where in here did windows matter where in here did UNIX matter where did he in here where you know the basic configuration files for permissions or whatever in Linux or Windows none of that is the case everything here is its own service and you're simply calling and you're simply using the services I am using the resize service that uses compute I'm using the storage service I'm using the authentication service and so that then gives you a finished product where we've we've abstracted out all of us those different layers and so we're now interacting with services and not really even servers the way that you think about it so I hope that makes the concept of abstraction a little bit clearer to you and the important thing here when we're talking about concepts and we're talking about models these can be used to look at a lot of different type of systems right so we can look at the old-fashioned again we're talking about servers with with server configurations and user accounts and all that kind of stuff and so we can think about abstract and out the layers there so that we create virtual machines and those virtual machines are run on hypervisors and then you have storage over here and you have something like that or disaster recovery as a service over here and so basically we've abstracted out those different those different fundamental components when we're talking about a more old-fashioned type of server system or you can talk a look at things with with abstraction when we can't go into networking again taking the old model of networking where you have the and the switches and those routers and switches they have their own operating system on there they have their own configuration files it can be very difficult to modify or do any kind of maintenance now we can abstract out the layers and so the intelligence layer the routing layer these different layers can be different components that we can interact with and therefore we can make networking something where it's much easier to be able to administer both in the physical world and importantly in the virtual world a lot of people don't think about networking within virtual environments but again if you're using virtual machines against servers such as Server 2012 or 2016 or Linux or whatever you also have to think about how is the data going to be routed within those hypervisors within the virtual environments what servers are going to be able to communicate with which other servers and then finally to look at again worse with serverless architectures where basically we're looking at an entirely different paradigm for how to provide services you're not dealing with UNIX you're not dealing with Linux you're not dealing with Windows you're simply dealing with services what do I really want and this is an important thing to be thinking about as a technology professional is your users don't care about Windows they don't care about Linux they don't care about Unix they don't really care about Mac right they care about solutions what do they really want they need images resized so what if you can simply call compute that all that it does is resize images right that makes your life a lot easier so now you're simply calling for the ability to be able to resize an image and again you don't have to worry about the permissions you don't have to worry about the updates you don't have to worry about all these other things that go along with standard system administration you simply call the ability to resize it resizes your image it dumps it in a folder and away you go again you start looking at things like authentication services and storage service and us and all this and what's really amazing now is basically you can have one web server somewhere that has a whole bunch of Python and node and some other type of languages basically code on it and so basically that code that code simply calls these other services and so you're able to build finished products and finish infrastructures and then a new way and for many companies and for many products that you're going to be producing a much more efficient and reliable way to so those are some of the things to be thinking about with abstraction again abstraction is a Nydia it's a concept it's a model of looking at things and it's a good way of looking at things in the modern world of technology so if you're sitting there and again you're thinking about storage instead of thinking about storage as in a hard drive or a SATA versus scuzzy or solid-state vs. plaid or whether you should go for a large drive or whether you should do raid or anything else think about what if what if storage is just storage so you start looking and go okay that's storage what's the best way of doing historic for this particular machine maybe the best way of doing storage is simply throw in a solid-state drive into the physical box and that's your storage for some other thing though if you're thinking about just from storage not as a hard drive a store if you might say wow you know that it would be better if we created a sand and then the server was communicating back and forth with the sand that would that might be a more reliable redundant way of doing things or you may think about you know if you're pushing a lot of data you know you have to store a lot of things again let's say you're doing Instagram or something like that and you're worried oh I know I don't want to run into a place where we get too much data before we can upgrade our systems and basically our systems get overloaded maybe then with storage then you're connecting with an API again up to AWS or up to a shore and there it's basically limitless you know however however much storage your credit card can afford right so it was basically limitless so then it can simply grow as need be right and so this is how you start thinking about something as simple as storage you just need something basic and fast that is fine on the physical device itself maybe you go with the sand maybe you go with some other one some other API like AWS or as your and that's how you need to be thinking about how you're going to be building infrastructures and how you're gonna be moving your infrastructures into the future thinking about these core individual components versus how we used to think about it of the full physical box itself again back in the old days it really sucked like back in the old days we were dealing with with the Windows NT 4 or something and your hard drive filled up right I was a miserable experience I so again like think about this like like how it was before if your hard drive filled up you literally you had you got to back up the hard drive you had to then shut down your server room you then had to plug in the new hard drives the new raid array that you then had to boot up but then you boot it up into some kind of recovery console using a disk or something then you would have to be able to pull the data onto the drive or the raid you'd have to make sure the partitions were the right size right there's all this stuff that was involved with it we're now with storage it's great again like you use something like a sand and you just shove it a new hard drive in there and many times it'll simply Auto configure itself and Wow now you've got a whole bunch more space and so what used to be you know hour at least hours of downtime many times a day of downtime has now turned into a very simplistic you know couple minute process but with nobody even realizing that there's a problem so this is how you need to be thinking about that thinking about dealing with systems is instead of thinking about the physical box itself start thinking about just the computing just the operating system and just the storage think about it just the services and then ask yourself is it best to have the services specifically installed on this machine or can those services be provided from the cloud right is it best to have this operating system physically installed on this machine or is it better to install hyper-v or some other kind of hypervisor onto the physical machine and then run your server within an instance on that machine you you lose some performance as far as resources go because now you're putting an additional layer between your operating system and the physical hardware on the other hand and that physical machine dies you can literally just take that virtual machine file and toss it you know toss it on to another physical box and you can be up and running like literally in five minutes where if you have the operating system in the stall onto the hardware itself and that hardware fails well now you get a nice little recovery process and reinstalling the operating system and a lot of tediousness and so these are the kinds of things that you should be thinking about when you're going to be building your infrastructure and if you've already got infrastructure of things to be thinking about how you can move into the future so as always I enjoy doing this video man look forward to the next one

Channel: Eli the Computer Guy

Views: 12,569

Keywords: Eli, the, Computer, Guy, Repair, Networking, Tech, IT, Startup, Arduino, iot

Id: ctFbluGVDS0

Length: 35min 33sec (2133 seconds)

Published: Wed Oct 09 2019