Get Started in DevOps with Aws, Ansible, Jenkins, Docker, Terraform!

hi guys Scottie P with Scottie full-stack com coming to you with a new video on job description that covers a lot of DevOps technologies it's for a DevOps role through a tech recruiter here in the Tampa Bay area so I wanted to cover that and go through and make a quick tutorial on how to build a portfolio to submit to a recruiter such as this so these skills will be able to be transferred to other jobs so you know feel free to take what you want and piece it together and play around with this further the job is asking for dr. Jenkins experience cloud experience Google asher we're gonna use AWS doesn't really matter in the end nobody actually cares they all really do the same thing infrastructure as code which would be tear form or cloud formation we're using terraform ansible for automating and I see ICD which we're not really going to jump too deep into but we are gonna just deploy an image via SSH from Jenkins to a web server here's the project that we're gonna build here we have on the bottom in the blue section is going to be our computer it's us local machine here at home we are gonna be using these technologies just as the job description asked for terraform is gonna use to build a Jenkins server and a web server again in AWS then we're going to use ansible to automatically configure both of those servers to be to have everything installed that we want them to utilize so then from there in the cloud we have our Jenkins server that's been built and configured in our docker web server that's been configured and we are going to SSH from Jenkins into the web server and run at a falcon app just very brief app with some text being displayed with two two routes that then we can hit from just our browser publicly and be able to view that site so we're gonna make an HTTP request and then we're gonna get the response back from that docker application those web servers and Jenga server are gonna be in ec2 AWS so this is going to be a lot of fun prerequisites are that are needed are going to be docker and AWS account AWS vault which we're going to use to authenticate in torie to this account from our machine it's going to allow terraform to speak to our AWS account so if you don't know what that is and you don't have it check out the blog post that's attached to this video and in the description and install that again I have links for all this all in the blog posts and even more description on how to install some of these so check that out now you're getting your ansible you're getting you terraform you're gonna need python which you should already have and you're gonna need a text editor or an ide i use vs code because i like it a lot and a lot of people make fun of me for it so go ahead get all that stuff and because you're definitely going to need that before we begin so once you're ready jump back on and let's start with our terraform piece and getting AWS setup so looking forward to it guys see you soon welcome back I hope that you didn't have too much trouble installing all of those prerequisites if you have any questions again go to the blog post check it out join the slack Channel ask what you want and I will get to you in whatever ways I can and as quickly as I can so alright so let's go ahead and start out with the terraform piece setting up our servers via terraform so we have a jenkins and a web as we saw in the last segment and we're gonna do that through AWS so let's get our user set up get our access key in secret key we're going to get a make a group while doing that a security group in a security group sorry a user group and I am group and then we're going to setup AWS vault to allow us to exec in and then be able to use terraform to make changes to our AWS account so here I am inside the AWS management console again you know I reiterate this everywhere on blog posts and whatever else but we're gonna be doing everything in the free tier as much as we can so this particular tutorial should be completely free you don't need to worry about payment so here we're gonna start out in the I am section I have it here is I'm recently visited if you're new to AWS it's under security it does be in compliance and we're going to click on it here we are we're gonna go to users and we're going to add a user we're gonna name this user you can name it whatever you want I want to name this user just DevOps one because it makes it easy for me to know I'm gonna give it programmatic access and we're gonna go to permissions create the group and I'm just going to name this group DevOps oh one you named it I'm gonna give this user administrative access be very careful with this it has all rights to re this account no tags click through we're in review looks good and create that user now we have our access key in secret access key you can download the CSV if you want to keep it in your records I'm just going to use it real quick so cuz I really need to keep this account I'm actually gonna delete this so none of you can get sneaky alright so here we are in a terminal and I'm just going to use a AWS fault and DevOps so one it's gonna ask me for my access key so let's paste that in then let's go back and grab the secret key paste that in cool now we're in so now if I wanted to I could do and just to get in there now you know this vault exact DevOps at one and I mean so that's it so that's it it's pretty easy to do and now we are exact and in our terminal and we can create our terraform file and then you know start to make changes all right so picking up from where we had just left off we created our user we permissioned our user with a group that had administrative rights and we created and then we were able to set up our eight of us fault and exec in via our terminal so now what we're gonna do is we're going to get inside of our ec2 dashboard make sure everything's prepped to ready to go then we're gonna build our terraform files and create these two easy twos one for the web server and one for the Jenkins server so here in our dashboard what I want to do first is set up my key pair and I'm going to just call it DevOps one and I'm gonna make the pin because I'm Linux there it is it's in my downloads folder and I already have one in there so it's gonna rename it but actually we do that now so I'm gonna move from my downloads one then to my SSH folder and I'm gonna name it of course step op so one pepper and yes right and then what I need to do is I need to schmatta it at 400 SMUD 400 Sh DevOps at 110 and there we go so now it's a able to be able to exec into an instance if we needed to so the keys all set up and our instance is our AWS is ready to go so I'm gonna jump on over I'm actually gonna in from this terminal does open up so where I am right now I've created a folder called DevOps a1 it's a project folder nothing inside of it I'd recommend you to do the same thing and then you can push this up to a repo later if you want but we're gonna just do everything inside of here I'm gonna go to make a directory call it tear form to keep all my tear form files in let's get inside there and I'm gonna make two directories for Jenkins and our web so there we have our two directories inside of our terraform and then I'll go into my Jenkins and we're going to start here so I'm going to create a file called main TF that's our terraform file and here we are okay so from here what we're going to do to start is actually before we get too crazy I'm going to open up a getting Norah file because if you are going to push this up to github you're going to want to make sure you keep out I'm gonna make sure you keep out any large files ignore terraform so this here and I've linked this on the blog post so if you want to copy it you can but this is going this is basically the standard github get ignore for terraform projects you're going to want to use because otherwise you won't be able to push this up so there we go so that's they're all done with that now let's get started so I'm gonna use like I you know weird using AWS so I'm going to use provider AWS and then our default region it's gonna be us one okay so there's that and then save it and then what I'm gonna do is throw in a variable and you know I have to do this but I'd do it kind of by default because description because we will want to name our instance upon deploy so name the instance mod deploy okay and you'll see that in action here in a minute and see what I'm talking about and then we're going to create a new resource and this one's going to be an AWS instance and then the name this we can call it like you know DevOps one Jenkins okay and then from inside here we're gonna do an ami which let me show you there are there are ways to do this you do it from terraform but just so you understand what's going on with this we're going to go to our Amazon machine images and well sorry that is not right we're gonna go to launch instance and then use free tier and I'm gonna use a bunt 218 so I'm going to take this ami image right you can use anyone that you want that makes sense for you and then yeah so it's that simple just plug in that ami ID and then we're gonna want to again keep this in the free tier so instance type equals t2 micro and then the key name is what we've just created which I use DevOps Oh what okay so about easy and then the last thing I want to throw in here is the tags field and this is where our variable comes into play so the tag which is name is going to be bar dot right so we're done so that is our different file what I want to actually do is just inside there copy Jenkins main and I want to actually just put it into web and then I can go open that web mean what we'll do here is just name this swim so we're keeping those files separate and everything else is gonna remain the same because pretty much the same thing and then what we can do is I can get right into Jenkins and I'm going to do a tear form in it it's going to download are either best plug-in and will do the same thing for for web so I'll pause it you do that do a terraforming it in both and then come on back we're gonna go ahead and run a terraform plan so I'm going to do this and make sure you've got your AWS exec done so here we're gonna do terraform plan I'm going to name this instance Jenkins it's going to spit out what we're changing slash adding right so we have our ami that we've set T to micro our key name and then a lot of defaults as well plan 1 2 adds or the changes are to destroy so we're good with that so we're going to do apply to Jenkins it's gonna ask me to confirm yes and then what we can do is pop on over to our ec2 dashboard and we should see this spinning it yep there it is that easy so let's go ahead and do the same thing for the web and then come on back here and we'll continue moving all right and we're back so we have Jenkins and Webb setup if you have any question of the web just go back and repeat the Jenkins processes and name it Webb and do it from the web folder in our terraform directory and so that's it we are actually up and running we have two instances really with nothing installed on them but if you wanted to try to connect and you can connect pretty conveniently using that or you know you can always just do it yourself copy oops copy your link and then we can cut down to a terminal and I can do SSH and paste the bun to and because that's our user the abundance comes standard with the bun to a user and then designate our key which is that ssh devops or one PIM and yes and I mean so pretty cool stuff and that causes a default security group so we will likely make some changes to that and or not I mean it depends on what you guys want to do you can do it yourself you can put your IP on there so only you can log in from from your address or you can leave it open and you know tear this down whenever you're done so awesome so let's take a minute and review what we did so we created our user permission the user got into a device fault then we wrote our terraform file for both Jenkins and web and then we planned and applied those changes to our ec2 so the next step in the process is to build those ansible scripts and so we will do that in the next segment awesome and nice work all right so let's talk about instable so we're going to be using ansible to configure our servers remotely and in theory we could update many servers with the same configuration and only automate it right or only do it once and then a lot of maybe the rest of the time right so what we're going to do here is we're going to configure our Jenkins server and we are going to configure our web server with ansible playbooks and so let's start there show you what we need to do in regards to making those playbooks end updating our hosts file but if you haven't had this installed already check out the blog go to the documentation and install ansible on your machine so what we'll do is I'm going to start out we're going to create that ansible directory and then inside of that ansible directory we're going to have two files so let's start these from scratch here I'm going to open up provision where the Jenkins first write save it and then we can also open a provision web right so those are going to be our two playbooks and so before we kick that off let's do a sudo vim this is gonna be an Etsy ansible Oh put in my sudo password alright so here inside of our host file we're going to note there's a web server section and there's a Jenkins section and we put that in your file should just have comments in it if you're new to this and then we need to update our servers and keys in in these lines here right so if I go back to my instances I am going to grab that public address and come back to my file and so here this one was Jenkins so what we're gonna do is do a bunt to at so it because of on sues the user at our public address and then we're gonna do ansible SSH private key file and point that to our DevOps a one file and we're gonna do the same thing with the web server so let's do that face that in okay then save that so there's that and now let's go back to our files so let's start with the Jenkins these are a yam will file so if you don't know much about yam will be careful with your tabs or spacing because it's very dependent on that so we're going to name this entire playbook the configure Jenkins playbook and then we're gonna do a hosts is our Jenkins host that we denote it in our file lower case day and then the tasks we're going to do below so let's add a new task I will call it install well that's first Before we jump into that let's talk about what we're going to need because that is obviously a key requirement here so if we jump over to the install Jenkins page Jenkins it runs over the JVM so we know that we're gonna need Java and you know we'll need our web browser for the admin panel which we already have clearly and then we need to come down and we're gonna have to install and start Jenkins for a bun to Debian Ubuntu wherever that went there it is so here so we're gonna use these lines here to download Jenkins and then we're going to start it in an it D so let's do the first one we need Java right so let's do install Java requirements or something of the nature of that and then we're gonna do use app because it's a bunk do and within apt we're gonna update cache yes because we want the update of packages and then we're gonna do name of that is default - JDK right and then the last thing we need is become yes because we want to do su do the next that we need is to install Jenkins right so this one instead of app we're gonna use a shell we're into some multi-line and then below it we are gonna pop back over Jenkins grab the script as is and throw it in there watch our tabs and that's fine studios in there so we don't need to worry about to become user however we do need to add on the end of this - why and the - why so that those will auto confirm that we do want to install those packages and it won't hang so look for that if you have any issues it's often missed so then the next one we're going to run Jenkins and this is gonna be another shell look money this one was going to be a one-liner Etsy in it dot d and then Jenkins start right and we want to become yes cool that's that let's start on our web as well so what do we need for our web server we know that we're gonna do a Python app right that is doc arised that is well that's actually about it so really we just need to make sure that we have docker installed that we start docker and wouldn't hurt to have pip on the server because we're gonna have Python on already so let's add pip three in there so let's do name provision web servers and then host is from our hosts file web servers and then the tasks are left so name first thing we want to do is let's go ahead and install pip three and you may not need three for this but it is a helpful tools to get in the habit of so we're gonna use apt and then we're going to update our cache once and then when we are going to find the Python three - pip package and then we're going to become yes okay so that's pip name install same thing but for doctor doctor oops and then at we don't need to update the cache again name stocker dot IO and then become yes whoops took a screen shot for myself apparently and then the last one is going to be name start docker I've got really like wild fingers over here this one's gonna be shell right here it too on my keyboard it's a lot of fun we're gonna do system control start docker eyes get the two confused I always think it's docker start just control enable doctor right I think yep all right and then Oh oh my become yes okay let's go ahead and now that we have our playbook set up let's run ansible playbook provision Jenkins yes let's do the same thing in full playbook provision Webb okay and these are gonna take just a little bit of time so I'm gonna pause it you do the same thing and let's check back here in just a few minutes okay so it looks like our instable jobs finished up let me know if you have any issues and the slack channel or you can't yeah you know get it to work this also github repo it's got a full stack that you can go and get the code if you need to but try to figure out any errors on your own if you do get some so cool so our boxes are configured now let's just test this out by going to the web box and checking to see if we have docker running right cuz that's what was one of our steps so let's do SSH and we're gonna do a bunt to at air - I Sh DevOps pen okay so we're in and so we do docker PS and it's right so you can check images - docker images cool so that is that our servers are configured now what we can do is go into Jenkins admin and we can set Jenkins up we can get SSH configure it on there and then we can work on security groups and then docker is our application so let's jump into Genki now all right now let's try to get Jenkins up and running on our browser so we're gonna take that public address and we're going to go to 8080 and here we are so we got to our initial screen we're going to need to put in a passcode that Jenkins provides in our box so we're need that so let's go back get our public address again make sure you have it copied to your clipboard and then let's go to here we're gonna go ahead and SSH to a bun to at our docker address with our key and from in here now we're gonna go back and grab that Pat taste it we're going to do a pseudo cat right and grab that key pop on back here paste it and continue okay and then we're gonna go ahead and install these suggested plugins and it's going to take a minute so go ahead and get that done and then we will come back and now we can do is to set up our user and then get inside you can do this too with whatever you want okay save and continue that's fine and start using Jenkins now we're inside and really what we want to do at this point is just a couple of different things first of all we want to install a plug-in I installed it available this is H and did not search there we go this one install without restart and take just a couple seconds and voila let's go back to dashboard let's add some SSH credentials so from here Jenkins global credentials and do SSH and then username was a bunt two for our user and then what we're going to do is we're going to add in our private key which I'm gonna allow you to see mine because I delete these so once I've done so let's do a cat PEM then go copy that I'm gonna come back and then paste it okay so that's good so we have credentials now and yeah that's really bound it from the configuration side that we needed to set up right now and we will come back in here and actually nope sorry false go to configure system real quick just a very quick step we're gonna set up our SSH profile for our web server so once we're in configure I'm gonna search for SSH go down add the hostname is gonna be the web instance private IP and that's there we go go back 22 is the default SSH a bunt 2 is our credentials that we just made and then we can check the connection and we're gonna need a security group so let's hop on and over there so security groups what we're gonna do is open this in a new tab we're going to create a security group for our web server so I'm gonna create one I'm going to call this web the one or whatever you want to call it doesn't matter SSH for web we're gonna do an inbound rule add it and then we're going to do SSH and this is going to be specifically I don't want anybody SS aging in here so it shouldn't be right so I'm gonna use just my Jenkins box which I think I think that yeah private to private is fine right we'll find out I guess create actually when I add some work to that web one here let's edit our inbound rules let's add while we're at it let's do a faulty CP from anywhere and then let's do a HTTP from anywhere because we use this later okay and now let's go back to ec2 and check our instances go to web it's running and we're gonna change actions networking change security groups use I'm gonna use web server one the one I just made sign and there we go so now if I come back to Jenkins check it again and now we're successful because we've allowed Jenkins to go that IP address SSH to our web server so cool alright let's save it and we're good with Jenkins so awesome Jenkins setup is complete and in the next section we're gonna quickly build a Python app with docker and then build a job and push it to the web server right so now for the docker portion I am at my up account scotty full stack the folders devops out one this is where you can find everything it's all public for you what i'm gonna do is i'm going to copy this python small falcon app because we this is not the goal of the course is to learn Python right we we just want to quickly dock rise an app and piece it up all we're doing here is on the 200 response from the root route we're gonna print hello world and then from the page to route we're gonna print this is the second page on the 200 response so nothing fancy really not at all so I'm just gonna copy that and we're gonna come back to our a note pad slash IDE and get rid of that one and let's create a new folder inside of our DevOps o1 folder make sure docker and go inside Tucker and then open up a hello dot PI app okay and I'm gonna save it cool and if you haven't yet you can see I'm using am importing Falcon there we're also gonna use guna corn so pip3 installed unicorn and or will actually and not work Falcon okay so already have those installed but you can do that in a virtual environment where you can do that just on your local and cool so the one the two files that were going to need are we're going to need a requirement text file and a docker file so save it and save it and then so in the requirements that text file I'm gonna do a pip 3 freeze so I'm gonna do a grep for Falcon and then after I've installed it I'm gonna put that in requirements dot text file and then I'm gonna do another pip fries with a grep pretty guna corn and I'll put that in the requirements that text file and you can just you know paste it in however you want but this is just if you got a lot this is a really cool way to like you know add them to your requirements file so that is that now for the dockerfile so if you're not familiar with docker it is a container service so it's gonna what we do is we build this docker file this dr. Paul tells docker how to build our image that we're gonna then it becomes immutable and then we're gonna put that on to a server and we can assume that as long as the base level requirements on that server exist to run docker that this image is going to be the same no matter where you deploy it so it's a it's a safe way to deploy code to an environment and not tear down the environment or break the app you can easily roll back so this is a really cool tool that devops really everyone should be using so but to start this off we're going to import a base image from docker called Python in the Tagus 3 so we're using a Python 3 base image we're gonna run a pip install - - upgrade upgrade pip so to pick 3 right and then we're gonna make a work dirt it's just standard docker so you can always look up documentation if you're confused about anything specific I'm gonna call my work directory working directory hello world then I'm gonna copy our requirements requirements that text file to the working directory and then we're gonna run pimp 3 install - our requirements dot text right and it's going to install everything that we just put in that requirements file and then we're going to copy everything from our directory to the working directory and then we're going to CMD and we're gonna call guna corn and then we are going to bind unicorn - B - port 8000 okay and then we're going to initiate it initiated use call app okay so awesome that's our docker file now so you have one two now we'll do a sudo docker build this directory - t4 tag and then we're going to put in our repository and the tag name that we want to call it whoops so you do okay so successfully tag then built my image and then what I'm gonna do is su new docker push to dev ops o1 okay they already exist so obviously I've built this before on my machine and then I'm gonna come over here to docker and tada it's over there okay so we now have our tag so that's really it for the docker piece of this segment you know we are we can do more in-depth videos later on but that's really the base level of what you need to know for what docker is and then you can play around and do more with that on your own which I totally encourage but now what we want to do is jump over to our Jenkins portal for the last piece of this we're gonna create a new job called like DevOps whatever you want to name it I'm gonna deploy right gonna do that you might want to me like the play web or something something practical okay and then we will add a build step and this is going to be execute script our credentials that we entered earlier are already there the command that we're gonna run if you come down to the blog post you can copy this in or you can run your own but we're gonna name this hello world and then we're gonna run it and bind it to 8,000 and then say you know the image that we want now you can primer primer of ties Jenkins - you know swap out the tag if you want but I'm just gonna hard code it just for you know the intent of this video apply it save it and do a little dance cross your fingers build it check on the build and it's downloading the image all the dependencies need elevator music okay success and then we can jump over to our web public URL so you do even the IP or that DNS and paste it in port 8000 and there we go hello world if I did the slash page 2 this is the second page so congratulations guys you completed the first DevOps video and actually my first tutorial video ever so you know give me some feedback love it either in youtube or on the slack channel send me an email a team at Scottie full-stack calm anyway you guys can reach me I'm happy to help anytime we're all in this together and yeah anything I can do so congratulations nice work push this up to github and don't forget you're getting nor file for those massive terraform packages and yeah keep at it let's do this hope to see you guys again soon

Channel: Scottyfullstack

Views: 7,742

Rating: 4.9864864 out of 5

Keywords: Devops, learning, beginner, interview, job description, tech, portfolio, github, terraform, ansible, jenkins, docker, python, app, application, development, coding, IT, aws, amazon, employment, job search, career, career change, learn skills, skills, how to learn tech, how to get a tech job, cloud, tutorial, web tutorial, build an app, start a new career, how to change careers, free, learn to code free, operations, what is devops, make an app, learn docker, dockerfile, infrastructure as code, money, salary

Id: lFIlFedch3k

Length: 41min 33sec (2493 seconds)

Published: Sun Mar 01 2020