Infrastructure Automation to Build on-Premises Private Cloud

hello my name is Glen Davis and I am a principal Solutions Architect with the DevOps and IAC team at Dell technologies today I will be showing you a demo of infrastructures code or IAC for a private cloud environment IAC is the ability to completely build an environment from the ground up using DevOps principles which includes using software and automation the important thing to understand is that this is not just scripting IAC uses developer tools it uses source code repositories version control pipelines for continuous deployment and configuration management in the end it should all be idempotent item potent is the means of running the same thing over and over again without making any changes and the result should not be any different between the first time and last time it is run in addition if there is something that fails along the way you should be able to rerun it from anywhere before the failure and the result should be the same as if it was run from the very beginning this demo is by no means exhaustive it is just supposed to show you what is possible in your environment in addition to doing IAC in production another area which lends itself to infrastructure is code is simulating a production environment in the lab for example this demo is being done using a laptop it happens to be a very beefy laptop but it's just a laptop so here are the tools that we're going to be using today for our on-prem cloud infrastructure we going to be using vSphere for source code and version control repository we're going to be using git and github our CIC the pipeline is going to be run through Jenkins configuration management and automation is going to be done using ansible to provision our bare-metal servers in our case they're going to be simulated bare-metal servers we're going to be using Foreman and to provision our virtual servers we're going to be using terraform the reason that we're going to use Foreman for one and terraform for the other is because terraform will not be able to build bare metal servers using pixie boot however Foreman will not be able to quickly deploy virtual servers the same way terraform uses templates so let's take a look at our architecture as I mentioned before the architecture should match the inventory so I put them side-by-side so we could see them both so as you can see here we have the two bare metal servers if we look here we have a group called provisioned bare metal those are our two servers and they are ESXi and ESXi ESXi one and ESXi two these are the names of these bare metal servers then we have V Center which is right here we have ESXi the group that is talking about configuring the software and we have both servers again and that's showing these ESX eyes here because that's the software and the hypervisor being configured we have a router that we're not going to we're not going to do today but in order to show the completeness I wanted to show the router here we have the DNS DHCP server we then provision that sento as VM which is right here we then have a template that gets created from the VM and then using terraform we have three hosts we have web server one web server two and the load balancer and the load balancer name is web app we then have to configure the software on top of the load balancer and there's the load balancer group and it's going to be going to the web app and then we have the web server software that also needs to compete configured be configured it's going to have nginx on it so before we actually start building this complete architecture I wanted to show you what the pipeline looks like so the pipeline is made up of four pipelines one is the master pipeline that's going to call the other three sub pipelines the bulk of the work is going to be in this vSphere building out the vSphere environment we have this preparer infrastructure pipeline but for simplicity's sake we're not going to do that today although for completeness we wanted to show this actually being here and then after the vSphere environment gets built we're going to build out the web application if we did do the preparer infrastructure what that would entail is creating the top of rack routers configuring DHCP and DNS and configuring idrac this again it's going to run through this but these are just stubs so let's get started so all we have to do to kick off this pipeline is commit our code so here's our inventory we could assume that this is a very brand new inventory and we want to start this up so we're gonna go ahead and add it we're gonna commit it and we're going to push it to get up so we'll go into the master pipeline and you'll see that we're up to run 90 immediately run 91 starts and we see that I'll just it's that build infrastructure that I just created so we'll go in there and we'll see that the first thing it's doing is preparing the infrastructure and that was very quick so now we're going to go ahead and build the vSphere environment and this like I mentioned is the bulk of the work so we'll go in there and the first thing it does is deploy the ESXi cluster because there's a simulation I have to manually start up the servers and you'll see that it looks for DHCP and it gets its DHCP IP address it gets the file that it needs to deploy ESXi and then starts building we'll look at the actual run of the pipeline to see what it's doing so as you can see right now we're checking when the host server is installed and basically this is happening inside Foreman and Foreman is listening to the ESXi server and it's waiting to get an idea that it is complete so I'm going to log in to Foreman just so you can see and you'll see that there are two servers here ESXi one and he is excited too so here is ESXi one and here is ESXi two which should be understood is that if these were bare metal servers they would be working independently and would be building using their own server resources because we're doing a VM this is a shared resource so it's a little slower but Foreman could build hundreds of servers at once because the resources are not on form inside the resource is on the actual server there the resource requirements in the load is on the server that is being installed so in our case it's on a VM so it's not the same so we'll take a look at ESXi one and ESXi one shows right now it's pending installation so like I mentioned before Foreman is waiting and listening to when the ESXi servers are complete you so the ESXi servers are finishing up they just need to reboot but as far as Foreman is concerned we'll do a refresh we'll see that Foreman sees that the servers are now installed and that's ESXi one here is ESXi two so we have both servers now completely installed after ESXi is done we will then start the deployment of V Center so as you could see here V Center is not reachable because there is no V Center so once the deploy gets started and here it is we're deploying visa our clients you so now that vCenter is deployed we're going to configured fee Center and now we're going to add the ESXi clusters these TSX I servers to the data center that we created so we're going to go and we're going to log in to V Center now while we're logging in to V Center which takes a little while the first time we're gonna show you that the other components that we're we're done very quickly so we'll quickly look at how the center was configured so we basically we created availability zones and we created the cluster that was it and when we added the ESXi cluster we had to get the correct key so that vCenter could authenticate the cluster correctly and provide all of the settings that were needed to to get ESXi installed so it's now configuring the vSphere Network so while that's happening we'll go back over here and look at the availability zone one was created and the cluster with a cluster name called cluster was created with two ESX eyes and then we have the vCenter server appliance VM also installed just expand this and we also created two data stores when the ESXi servers were created and they're normally named datastore one but because of the overlap of both ESXi server is called datastore one we rename them with the name of their ESXi so we know to keep them separate and then the vSphere network as we showed here gets created and once that's created once the network is created we then start creating the VM template that I talked about earlier so the VM template is made up of this sento as VM and the sento as VM has to get installed from scratch so just like the ESXi server Forman will deploy this ESXi server sorry this sent OS server and if you look at Forman right now we have two if i refresh we all now have three and the sent OS server is pending installation so the center was VM template task is running and we're basically waiting for that to finish you so now scent OS VM is installed if we look at Foreman we'll go ahead and see that it's installed and we'll go back to the all hosts and we're pretty much done with Foreman for today so now we're gonna make the template the template is going to be made in the center and you'll see that we're basically cloning the virtual machine to create the template and that template is as we saw earlier it's called sent OS sorry it's called sent OS seven template so now that we're done with setting up all of the VM architecture we're going to now work on the actual web application and we're going to deploy that so the first thing we're going to do is deploy the load balancer as you could see here with cloning the template now to create the load balancer called web app and if you look here web app is here and we're gonna start up that image and it's booting and if we look at here we're provisioning using terraform it's done terraform does a few different tests to validate that the image is actually installed and working correctly so this takes a little over a minute so we're going to just wait and and then we're going to move on to doing the exact same thing with the web application and when I mean the web application I mean two of the two web servers so as you can see it now is provisioning the web servers web server 1 and web server 2 it's doing the same thing with terraform creating the two templates at the same time and this is what I mean by look how quickly it is to create a virtual machine here as compared to what we did with Foreman so now we have two web servers and terraform is going to do the same testing on these two servers and when it is complete you will move on to installing the load balancer software so now that the web server is the hosts are installed now we're going to go ahead install and configure all the software for the load balancer and the web application this is going to be fairly quick once this is done we'll be able to see whether this is working or not right now we still can't reach the web app application so now that the web application has been completely installed everything is done we'll go back to the master pipeline show you that everything is complete and now if we just refresh the screen here we should get and we do web server one and we could then look at web server two and there we go there's web server to the refreshing isn't great but there we go and we go back and forth between web server one and web server two by just received refreshing so the last thing I'm going to do is show you and this is what makes infrastructure is code so powerful I'll go back to our diagram and as you could see I have a third web server here called web server X we're going to add that to the inventory we're not going to touch any of the code we're going to just change the inventory commit that code and have it run and we'll see what happens so the first thing we need to do is change the inventory is to add the host so we're gonna add a web server three host and give an new IP address and then we're going to add the software to install the software for that web server host and we're going to change the color we were right now using blue and green I'm gonna go ahead and go to the color picker we're gonna go ahead and use red just to make it a little different oops okay and this commit add web server 3 and push the code and what we should see now is a new pipeline started and this is going to be a lot quicker than the original one the original one took and let's look at that so here is add web server 3 we'll look at the original one the original one took 52 minutes to run so now we're going to go to the new one and the new one is already done with the prepare infrastructure and it'll go and install go to the vSphere pipeline which will also be quick so this took prior this took 47 minutes this will take a lot quicker so now that the the vSphere environment is set up you'll see that only took four minutes whereas the last one took 47 minutes you may ask why did it take four minutes why didn't it take 30 seconds and the reason is there's a lot of error checking that goes on in testing before to make sure that everything really is working because if something isn't set up correctly it'll go and redo it and fix it and make sure that it is working so after that stunned we'll go back and deploy the web application now this is where the interesting part is here because we're actually doing other work so we're deploying the web server and you'll see that the terraform provisioning it says okay - one and two these have already been provisioned so these no longer need to be provisioned when it needs to be provisioned as web server 3 so that's what's going on and that's what's going to take the minute like it did before so while that's finishing up you could see that now there is a web server 3 here web server server 3 was created and now that it's finishing it's going to go back to configuring the software so this actually the load balancer even though it was installed earlier has a change it now has a new web server so it needs to be configured with that new with that new web server 2.2 so it has 3 web servers and then it will finish installing and configuring the the other web servers as you can see the only thing that changes is web server 3 because the other two don't need to be changed so this will take another second it'll be done and once it's done we'll be able to test our web application okay it is now complete we'll go back over here we have web server one we're going to go and do a refresh and there it is there's web server three and there's web server two web server one and just goes through that's it for today thank you very much for watching this demo and hopefully you've seen something that you find very interesting take care

Channel: Dell EMC

Views: 3,820

Keywords: Dell, EMC, Infrastructure-as-code, iac, devops, pipeline, infrastructure, consulting, application, vsphere, provisioning

Id: 1_dRHj4xlAY

Length: 25min 17sec (1517 seconds)

Published: Wed Aug 07 2019