How To Automate Your Dev Setup

hey guys bg here and today in this devops deep dive i'm going to be showing you how you can take the first step of many into improving your development experience by automating your development environment if you think about it it's one of the first things in your development career that you don't actually consider could be later on technical debt for you and what i mean by that is when you start developing nine times out of ten you just start installing things now that doesn't mean that you can't take that process and grow from it but it definitely means that later on when you try and reproduce that process it's going to be very difficult this is where automation comes in automation doesn't just save time but it saves you time in the sense of how do i do this thing and how do i make sure that when i do it again later it's done the right way so that i'm guaranteeing that the setup is what it should be basically what we're talking about is automating all the things anytime you install a program automate it anytime you configure a file automate it anytime you have anything that you need for your dev environment that you want it to be elsewhere automate it never do the same thing twice because when you automate those things you don't have to be worried about doing them again later so let's start with the first thing the operating system i have a fresh manjaro with i3 installation here i chose manjaro with i3 for two reasons the first one being that manjaro actually has a very nice wizard for installing arch linux and on top of it manjaro also has a community edition with it that includes i3 gaps so when we're talking about automation and making it kind of very easy to set up a foundation for yourself having an operating system that you like as well as any other features that might be coupled with that operating system out of the box is super valuable i'm gonna go ahead and hit mod enter to open up my terminal and then i'm gonna type in sudo pac-man s-y-u whoa bg you're not automating this why why aren't you automating this i have found that system upgrades can be very very troublesome you'll find yourself having some type of broken driver or something else that happens and you really don't want that to happen in the middle of your workflow when you're developing something or working on something so in this case we're going to do a manual pacman upgrade the main thing here is we're upgrading the operating system manually and we're choosing not to put that in automation so it doesn't hurt us later on when we're trying to do something really quickly the first thing we want to think about when it comes to automation is how do we want to manage the things that we want to automate the things like configurations the things like package installations and other requirements that we need for our development environment but the main focus here is really how does it solve it the best for us there are some very simple things that we can use to start but then later on when we want to manage more complex things maybe rely on other tooling the tool tools i'm talking about in this case are going to be bash and ansible now bash is a very easy way of starting with automation but when you start talking about things like do i know that that change actually happened how can i confirm it as well as managing files properly doing things like templating bash kind of falls short so what i decided to do was lean on another automation tool called ansible i use it because in my opinion it does a good job of replicating almost what you would do in bash but translating that in a very simple form of yaml that you can just easily configure and then run playbooks one of the things i personally like is having a path in my home directory that i can easily access and reference at any time so this is going to create a directory in my home path called dot dot files we're actually going to create one more directory to create a very special file that we're going to use over and over again it's going to be what we call our dot file script now this script is a script that we're going to run every time we want to update our environment so to do that i'm going to create another directory in our dot files called bin and in that bin directory i'm going to create a file called dot files now if you don't know the bin folder is normally a folder that's added to your path to provide binaries or shell scripts that you can run anywhere on the operating system so we're creating a path that we're going to add later to reference so that we too can just run dot files anywhere on the operating system we don't have to worry about running it in the dot dot files slash bin path so once we create that script we also need to make sure that we can execute against that and to do that all you have to do is type in chmod 755 to make it executable if we try executing this it's fine we can just use the direct path to reference it and run it and it doesn't return anything but at the bare minimum this is a reproducible script we can now use now one other thing you might notice is you have no editor whatsoever to work with this is a fresh installation another reason why i chose manjaro with i3 is because it includes vim now i use neovim but i want neovim to be installed a very specific way with ansible that i don't really want to have to do in bash so let's do that let's create a shell script that is reusable that allows us to guarantee every time we run it we have the latest version of ansible now i want to write this as a bash script so i'm going to make sure that bash is executed now there's one other thing i want to do we're going to make sure that this script also returns errors as it needs to now what we really want to do is we want to check to see if the program is installed and if it's not then run the package manager and we're going to say if exclamation mark which means not and we're going to define a very simple if statement here now in this if statement we're going to do dash x with a double quote and that's going to check if a value is defined or not so what command are we checking for well inside of our code we're going to type in c-o-m-m-a-n-d-v and the command we want to check for and that's going to be ansible after that we're going to type then and then we're going to close our if statement with a fee what have we got we have an if statement that checks if something is not defined or essentially if we execute the command ansible and see that it's not defined then we're going to do something in this case we're going to install ansible so i'm going to do sudo pacman dash capital s ansible now when we run this script we should see ansible get checked for and if it's not there pacman will prompt me for my pseudo password if i do this you're gonna see just like i told you it's saying hey i need to be able to run the pacman installer because ansible's not there so let's go ahead and do that i'm gonna type in my pseudo password and i'm gonna hit enter now what do we notice we notice that it says hey ansible core and it gives us a whole bunch of prompting we wrote this script with the confidence that it should do what we're trying to tell it to so do we really need to prompt now that's up to you if you want it to prompt you can just continue writing your script but for me i want this to be a completely automated process so if we go back to the script we're gonna see that we have pseudo pac-man dash capital s ansible and in that we actually can add one more thing to it to make sure that we don't have to have any type of intervention and that's going to be the no confirm flag if we run that same command again what we're going to notice is it's going to go ahead and start installing ansible without asking us if i run it over and over and over it doesn't matter because now if we type in ansible we can see that ansible is available now that we have ansible installed we can now look at the next thing we need to tackle something that's becoming more common is not only using it against remote servers but also using it against your computer locally and so what you can do is you could set up ssh keys on your host and then allow yourself access with your own ssh key via ansible so the first thing you're going to notice is that we're following that same pattern of checking to see if something exists and if it exists then doing something with it for me it's pretty much always the same spot so we're going to do ssh underscore der equals and then we're gonna set it to home slash dot ssh this is our ssh key file this is our private key so it's important to check to see if this exists before we try to regenerate it the first thing we're gonna do is we're going to make sure that our ssh directory actually exists so we're going to create that directory just like we have before and then we're going to change the permissions on it just like we have before with 700. after we change the permissions on the directory we then generate our key dash f will determine the file that it's saved at in this case we're going to do the file that we're checking now there's a few things we want to do before we finish which is we want to make sure that the whole ssh authentication process also works because all we've done so far is just generate the key what we're actually going to do to solve that problem is we're going to copy the output of our public key which was generated with our ssh keygen and put the output of that contents into the authorized keys file the last thing we're going to do is we need to make sure that this file is protected and you're going to notice a very common pattern which is i normally set everything only accessible to me i don't set it to anything else unless it's something i know i want to run everywhere we're gonna go ahead and save that file and then after we do that we're gonna run our command one more time so when we run this you're gonna see oh look i didn't have an id rsa before and you can see even here hey the user as well as my hostname were added as a comment to that and the key has been now generated now that should also mean that if i check my authorized keys file that that exists there and if i do that you should see there it is there's my public key now my ansible can run against my own computer and automate and do things for me because i've put that ssh key in there i'm going to go ahead and reopen our dot files script and in here you're going to notice that i've gone ahead and added a new variable to the top of my script and that's the dot files underscore dur variable now that i know that path is a variable and accessible to me i'm going to actually go ahead and change into that directory that means we can run things like our ansible playbook and our ansible plugin updates so what i just added was our ansible install dash r requirements command what this is going to do is after we change it to our dot files directory it's going to look for a requirements.yaml file to see if there's anything it needs to install and if it's installed it needs to check to see if those need to be upgraded so now i'm going to write this file and we're going to run this command and now you'll notice that we get a command saying the requirements file home the alta for stream dot files requirements.yaml does not exist and that's because it doesn't our file is now created let's go ahead and run our dot file script one more time when you run this command you're going to get an error it's going to say no requirements found in the file that you just created that's because there are no requirements in that file and as a matter of fact we don't even need this command right now technically the point of me showing you this however was to show you that the really powerful thing about this dot file script is you can handle all of those things that would need to be ran over and over and over again before even running your playbook so if you look at this logic we'll see first and foremost it's checking to see if that file exists just like it was before except this time we're not including an exclamation mark that's because we don't want to check to see if it doesn't exist we're checking to see if it exists so this isn't going to run unless that requirements file has been created and is in that directory we're going to go ahead and close and save find our file and then run it again and when we do that you're going to notice we get the same error that's because that file still exists and we haven't removed it yet so let's go ahead and do that there's our file let's remove it and then let's run our script one more time remember how before when we didn't have to install pacman how it just ran instantly well it's doing that again and that's because now it's seeing hey i don't have any file that i need to run against meaning i don't have to update any plugins remember automation comes first you don't want to do everything else and do your automation last i'm going to open up my shell script and at the very bottom we're going to add a new command this is our playbook command this is what's going to run all the rest of the automation against our computer what the ansible playbook does is it looks for a main.yaml file on your system to check for all the automation you want it to run for us that's going to be the entry point to all the automation on our system so we're going to want to put that in the same directory as all of our other dot files stuff you can see here i add it to dot files underscore dir main dot yml the one last thing i'm going to show you here is that i'm including a dash dash diff this dash dash diff just shows you the differences and changes ansible makes when it runs the playbook let's go ahead and save this file and let's just run this and see what happens now you're going to notice when i run that script we actually get an error similar to the one that we got before that's right we haven't created the file yet so it's not going to run let's go ahead and create that file now you'll notice that you get different output this time as a matter of fact it actually does try running but it fails because it has nothing to do this is actually okay what this means now is that you have a shell script that is able to set up ansible for you as well as run the playbook it needs to so it's doing exactly what we want if we run it again you're going to notice that it says it every single time remember i said earlier in this video that the goal of this script was to make it so that you had a reproducible configuration management tool that you could run over and over and over again to set up your dev environment this error simply means that our journey's just begun and now we're going to write a whole bunch of automation to make our lives even easier so we've really got a lot of the main problems here that we needed solved at this point i say go automate enjoy yourselves and have fun any time that you find an opportunity to automate something do it and again automation should be simple if it's becoming too complicated or anything like that take a step back and rethink how you're solving the problem one more thing to note is i will include all the source code that we used in today's video as well as my own personal dot files that this was all inspired from i literally used this exact same flow every single day and i'm continually developing on it and adding new things that i like if you have any questions comments or anything like that feel free to share them i'm super interested in learning how other people like to automate their environments and i'm always interested in learning new things remember we stream weekly live on twitch we're a variety channel where we do a bunch of different stuff programming as well as gaming and that's co-hosted by myself bg as well as my good friend and wonderful co-host otota if you ever want to be a part of our community you're more than welcome to join the discord i hope you enjoy these devops deep dives and i'll see you on the next one

Channel: TheAltF4Stream

Views: 8,199

Keywords: devops, automation, programming, softwareengineering, ansible, bash, shell

Id: V_Cj_p6se3k

Length: 15min 6sec (906 seconds)

Published: Wed Mar 30 2022