Managing Environments with Terraform via Directories

okay so we just saw a video on using workspaces to segment our environments here's what i sort of don't like about them the first thing is that terraform cloud here is something you might end up using as a place to run terraform apply and plants and all that good stuff because that's the point of terraform cloud it'll do that stuff for you you give it a github repository provided some environment variables and it will run everything for you up in the cloud in their cloud now they have a concept of workspaces in terraform cloud as well which are actually a little bit different they are different than the local workspaces so that's a little bit confusing the other thing i'm a little bit iffy on the workspaces here is that you don't really know what workspace you're on easily so it's easier to have some human error there so we have the terraform workspace show command again but it's not really in your face it's kind of easy to miss what workspace you're in and the last thing the reason why i tend to avoid this is because i've actually had issues where when upgrading to newer versions of terraform and i was not in the default workspace my state actually got lost and i happened to not use a versioned s3 bucket in that case so i lost that state i had to figure out how to delete everything and start fresh which is really a big pain now remember how i emphasized in a previous video how important was to have bucket versioning on in your s3 bucket it's from hard experience from that use case combined with workspaces and updating to a major new version of terraform something happened i don't even know what still and i lost all my state so in this video we're gonna see a slightly different way to do this it's a little bit more manual i kind of like it a little better and it's one that you might see a bunch of other people used to this isn't just me making it up off the top of my head so let's go ahead and see what that looks like there's going to be a new directory structure here so in here i'll make a new directory so make der and one is going to be called production one is going to be called staging and of course you can see the pattern here will make a new directory per environment inside of each item here we're going to copy cloudcast and variables into them copy that into production and then staging we can delete them from here okay so let's close this let's go ahead and see what changes we're going to make for these files the variables files we already have these commented out so we're going to leave those commented out because we don't need to say hey we're in the staging directory let's make this a staging environment because that's pretty redundant but if you have other variables you want to add to this file here you can go ahead and do that inside of the cloudcast file per environment we're going to go ahead and get rid of local we're not going to use terraform workspace and we can bring back our infrastructure environment variable that we defined before in here i'm in the production directory of the cloudcast file so i'm going to default to production for infrastructure environment and in the staging environment we'll bring that back as well get rid of locals and we'll say the staging one of course has a default of staging for the environment now one more important change up here is our back end here so our back end here has some configuration and it's hard coded so here cloudcast terraform tf state we really do not want the same key per environment so we could probably go ahead and do something like staging here and in production we can change that to production but instead i'm going to go ahead and do something different here we're actually going to comment these out and delete them essentially but i'll comment them out here so i have them for reference let's go back to our original here for these and i'm also going to get rid of the dynavo db table locking in our case here because we probably need a dynamodb table per environment in our case i'm just not going to show that right for now but we did get rid of bucket and key and we're not going to define those here and that is because we can actually define some backend stuff when we call terraform so it's a little more dynamic so the terraform command can accept a back-end flag and we'll see that in a bit okay another change here is that we change the directory structure a little bit so our modules are now up a level so we can had a dot here to go up a level and then into the modules directory because the modules directory is not any longer in the same directory as our production and staging files and here of course i forgot to change this local infrastructure and will once again become var and for m we'll replace all for that and ec2 app ec2 worker and vpc will change the directory structure there to reflect reality okay let's close these we're going to make some new files here one is going to be backend production.tf and another is going to be backend staging.tf this is how i've decided to add some backend configuration per environment but what you do is going to be totally up to you now let's go ahead and just check out production cloud cast scroll back up to here and bucket and key are the two things we need for our back end state now so we're going to copy those in to each item here and we'll copy that here okay so back in production back end staging now i'm going to use the same bucket for each but one advantage of using separate files for the for the back end configuration is that you can separate these and use whatever configuration you want i'm going to keep the same bucket but you might want to use different buckets per environment totally up to you in my case i'm only changing the key so i'm in back in production this will be the terraform production.tf state file and this one will be the terraform staging tf state file you might even want to do a different directory right so we could say cloudcast's production terraform tf state and i think i actually like that a little better here it's going to be cloudcast staging terraform state okay so i'm going to choose to separate these environment state files in s3 using a directory so production directory staging directory here so i think i have enough changes here to go ahead and test this out i'm going to use terraform and we can't just go ahead and run terraform plan or anything like that it's really not going to know what to do plus we need to reinitialize everything here because we have a new structure so terraform in it is what we want to run but we have some flags to give it so the first i'm going to do is say change directory in to our various directories here so we're going to be managing a state per environment so production versus staging right so we're going to have different state files so we have to in it in each different state so we're going to in it in our production state and in it in our staging state so terraform change direct so we can tell it what directory to run out of we're going to go and run out of our staging directory we're going to in it and i'm going to say back end config equals i don't think we need equals here but that is going to be this uh file backend staging dot tf now notice um it's very not clear here because of my sizing error so we'll do terraform change dir is staging we're going to run the init command and a new line i'm going to say the back end config is up a level backend staging.tf now i'm up a level here because we're changing directory into staging and therefore relative to that directory the back-end staging tf file is up a level now you might want to put back in staging in the staging folder so it's not like that totally up to you so let's go ahead and init this oh that one wants an equal sign okay so initializing modules and error unreadable module directory unable to evaluate directory sim link or those does not exist okay so our file paths are wrong somewhere and the answer that is simple i changed the production environment but i didn't change this one so let's change local infrastructure environment and we can replace this with one that has two dots wherever that occurs and then we can try to re-init this stuff it looks like it's working okay great and then we can run terraform plan now plan is simpler and plan and apply we still need change stir and remember we need to say equals and relative to our current directory we're going to plan in the staging uh environment here so change stir staging plan that is planned out right so we have 34 to add zero to change zero to destroy so i destroyed infrastructure after the last video so it's going to recreate everything but it's recreating it in the staging environment which is nice right environment staging for the tags now let's go ahead and clear this out we can do another in it and instead of staging we can say production so the production stuff is going to get initialized and then for there we can say instead of staging production we can plan that out as well and of course we can apply but i'm not going to bother doing that this video just because it takes time so this is going to go ahead and do a plan for the production environment 34 to add 0 to change zero to destroy perfect okay last let's create a helper script because these commands are a little bit of a pain to type out so i'm gonna make a new file here i'm gonna call it run just because that's a favorite uh bash script file name of mine and that is going to be ugly to type out and we're in a code editor so let's actually use the code editor so new file run to sh i'll just do sh so um phpstorm here knows what kind of file it is we're actually going to run it as bash and i'll go ahead and paste in a helper script here so user bin and bash use bash the environment is going to be our first argument that we give so it'll be staging production etc and then we get the current directory of our script here so the run.sh script this is going to get that current directory the directory name of the bash source we're in a cd to that directory so we're always running relative to the location of this run.sh script so just a little bit of insurance there now if we have the number of arguments we pass this is greater than zero then go ahead and run the following if it's not it's just going to do nothing and head back to the original location so if our second argument is in it we run this extra command because the init command needs this and wants us to give it the backend config otherwise if it's not in it it's going to be plan or reply or something like that in which case just go ahead and change directory into our environment one so we need a directory that matches staging production etc like we have and then run the second command which is going to be something like plan and apply so this is a little easier over here let's do chmod plus x run so that's executable run the sh and then i'm going to say move run to sh run because i want to just call it run then i'm gonna do run here and it's gonna do nothing if i just run it without commands so i can say run staging in it um we already initialized so i don't need to do that i'm just gonna go ahead and say run staging plan okay and that did a plan it's going to try to create 34 things as usual so we have a helper script here that is nice it's going to give us a little bit of an easier time to run our terraform commands but the main point here is that it is much less likely to do some human error type thing because we have to type out the environment in order to run commands here and even if we did that by hand we have to change dir into some environment right staging production all the good stuff we have very segmented things visually by directory here right so we can see i'm in production or i'm in staging and then within production and staging we can do some different things if we want to right so i could say like the count of um ec2 application servers might be you know something large like five for our production environment and it might be one for our staging environment so you can make changes per environment in a little bit of an easier way using this method okay so we saw two different methods of changing your environments of working with multiple environments within the same terraform configuration i'm going to go ahead and move on to some more advanced topics in the next in the next module and this is going to be a different way to segment out your terraform configuration not necessarily by environment but we'll do that also but instead by the area of aws that you were working with because that might be very useful in larger teams where you might have one team working on network stuff and another team working on compute stuff like ec2 and lambda and all that good stuff this is just another way to organize your terraform configuration files that you might want to think about depending on your team size and if you have different teams working on this stuff

Channel: CloudCasts

Views: 12,814

Keywords: devops, aws, terraform, hashicorp

Id: hkoxlJGklaM

Length: 12min 12sec (732 seconds)

Published: Tue Apr 13 2021