Managing Multiple Environments with Terraform

one of the primary benefits of using infrastructures code is to produce code that is reusable across multiple environments but how do you actually create a workflow to support it that's what we'll be talking about [Music] [Music] today what's up everybody I'm Ned bellance Ned inthe cloud.com and welcome to another edition of terraform Tuesday today we're going to be talking about how to work with multiple environments in terraform this is a topic that I get asked about a lot and it's usually phrased something like this how do I use the same code to deploy across multiple environments that's actually two questions the first is how do I write code that is reusable across multiple environments and the second is how do I actually deploy to multiple environments we'll be tackling the second of those questions today meaning that I'm going to assume that your code is sufficiently abstracted that you can provide runtime values to adapt it for different environments this is going to be a long video so if you're curious about a specific example you can hit up the chapters or jump to that portion of the video I'll also include links to the repositories I'm using in the description below special thanks to spacelift for sponsoring this video more about them later let's start by exploring the various options you have available for managing multiple environments for each of these options I'm going to assume that you are checking your code into Source control and that your workflow is referencing that VCS platform it doesn't have to be a gitops workflow per se but at a bare minimum you are using a VCS platform to store your code I'm also going to assume that you're using a cicd platform to execute your terraform code again it doesn't have to be a gitops flow but you are using a cicd platform to execute your terraform code there are some unique challenges to running cicd with infrastructures code which I addressed in my video about spacelift I'm not going to rehash that now just suffice to say there are a lot of terraform automation platforms out there that can help you with your cicd workflow and all of them will support the options we're going to look at last disclaimer before we get into the options these are not the only ways to do it there are other options out there but these are the ones I'm most familiar with and the ones that I've seen have the most success if you have another option that you think is better please let me know in the comments below the big three options for working with multiple environments are using git branches this is a scenario where you have a branch for each environment and you deploy from that Branch many terraform automation platforms default to this workflow so I guess it's pretty popular using folder structure this is a scenario where you have a folder for each environment in a single repository and a single main branch you deploy from the folder using separate repositories this is a scenario where you have a separate repository for each environment each repository has its own main branch and you deploy from that repository I'm also going to include an honorable mention of ter grunt later on now let's take a look at these three options and once we know what's involved we can discuss the pros and cons of each on many of the terraform automation platforms that I've used this seems to be the default workflow let me set the stage you have a repository with your terraform code the main branch is considered the source of Truth the most definitive and most current stable version of your infrastructure new environments will be spawned from the main branch and created as their own branches from the main branch we could have a test Branch a development Branch Etc each of these branches would be used to deploy the corresponding environment so if I wanted to deploy to the test environment I would merge my code changes into the test branch and the cicd platform would execute the terraform code from that Branch configuration values for each environment can be stored on the cicd platform on the branch itself or in some other location pulled by the cicd platform the code itself is basically the same across all environment branches but the configuration values are different when you want to make a change to your infrastructure you start by creating a feature Branch from the main branch you make your changes test them and then create a pull request targeting one of the lower environments that you use for testing or development once the code is approved it's merged into the target branch and the cicd platform executes the code then you can perform whatever functional testing you have outlined to validate the lower environments once you're satisfied with your change you can promote the change by creating additional poll request s that Target the other environments so if you want to promote your change from test to development you create a PR from test to development once that PR is approved and merged the cicd platform will execute the code in the development branch and you can perform your functional testing there rinse and repeat the process until all running environments have been updated and then you can merge the code into the main branch making it the updated source of Truth for all environments at any given time you want to have only one feature branch in flight this is to prevent merge conflicts and to keep the code in a stable and consistent State across branches since infrastructure tends to change a lot less than application code this is usually not a problem generally the size and scope of Any Given repository should be relatively small you don't want to have a single repository with all of your infrastructure code in it that's a recipe for disaster instead you want to have a repository for each application or service and then repositories for the shared infrastructure components the shared infrastructure repositories would contain things like networking identity and access management and other infrastructure that's shared across multiple applications how granular you get with your repositories is a matter of some debate and would probably best served by its own video having smaller bound Ed repositories keeps the change rate for any given repository relatively low and allows for feature Development Across multiple repositories to happen in parallel the way in which you actually promote code changes across the branches it might vary slightly but the overall workflow is the same you create a feature branch make your changes test them and then promote them to the next environment for the next option we are going to to separate our environments into directories held within the same repository so instead of having a branch for each environment we now have a folder for each environment the main branch is still considered the source of truth and the cicd platform will execute the code from the main branch but each automation process is watching a different working directory the configuration information for each environment can be held in the directory of that environment assuming that it isn't sensitive information that shouldn't be in Source control to begin with sensitive values can be stored with the cicd platform or on an external data source that is referenced by the configuration values a new environment can be created from an existing environment by copying the folder or alternatively you can keep a canonical version of the code in the root of the repository or in its own directory then you can copy that code into your new environment directory and modify it as need when you want to make a change to one of the environments you create a feature Branch update the code test it and then create a poll request targeting the main branch once the pr is approved and merged the cicd platform will execute the code from the main branch and deploy the changes to just that environment each feature Branch should be only targeting a single environment when you want to promote changes from one environment to another you create a new feature Branch copy the codee from say the development environment directory to the staging environment directory and update the config values if needed then create a PR targeting the main branch and merge it once approved the cicd platform will execute the code from the main branch and deploy the changes to the staging environment our final option is to have a separate repository for each environment each repository has it own main branch and the cicd platform will execute the code from that Branch the configuration values for each environment can be stored in the repository itself in the cicd platform or on an external data source that is referenced by the configuration values when you want to make a change to one of the environments you create a feature Branch update the code test it and then create a PR targeting the main branch it's kind of like the folder flow once the pr is approved and merged the cicd platform will execute the code from the main branch and deploy the changes to the environment promoting a change from one environment to another requires making a feature branch on the target environment repository copying the code from The Source environment repository over to that Target environment repository and updating the configuration values if needed then you create a pull request targeting the main branch get it approved and then the C ICD platform executes the changes on the environment those are the big three workflows so why would you choose one over another that's what we'll get into next but first a word from our sponsor spacelift is an infrastructure as code platform that helps you automate your terraform workflows it's a hosted solution that provides a web UI for managing your terraform configurations and it integrates with your version control system system to automatically run terraform plan and apply when you push changes to your repository H that sounds pretty useful even more than that spacelift includes a policy engine that allows you to Define policies that will be enforced when running terraform plan and apply this allows you to enforce best practices and security policies across your entire organization and it can even prevent you from deploying infrastructure that violates those policies space lift also takes takes care of Hosting your state data includes a private registry for your modules and supports Dynamic credentials for the major cloud service providers managing multiple environments with terraform as we've seen is a complex task and spacelift helps you focus on managing your environments and not the platform if that sounds interesting to you head on over to spacelift doio to learn more and sign up for a free trial I've included a link in the description below and thank you to space facelift for sponsoring this video using a branching strategy with your code is a pretty common workflow but it can also be a little difficult to maintain if you have a lot of environments you need to be very disciplined about code updates and promotions across branches like I mentioned earlier you only want to have a single feature Branch inlight at any given time to avoid inconsistent environments the branch strategy also assumes that all of your environments are relatively the same except for configuration values and so the same infrastructure is code can be used across all environments if that's not the case then a branching strategy might not be a good fit using a single feature Branch at a time is pretty easy when you only have three or four environments like Dev staging and production you promote the change from feature to Dev to staging and then to production doing your testing at each step and then you merge the code into the main branch but what if you have 10 environments or 20 or 50 are you going to wait until every environment is updated before merging to main what if someone else wants to introduce a change while you're still in the promotion process what if you have a critical bug that needs to be fixed in production right now do you create a feature Branch from the main branch fix the bug and go through the full promotion process or just merge the fix into production and worry about the rest of the environments later these are important questions to ask during the planning phase the branching strategy also requires that each environment use the same state backend type as this will be defined in a backend block within the code through the magic of partial configurations it doesn't have to be the exact same backend but they will all be of the same backend type if you're using S3 they will all be S3 backends that's probably not a huge deal but it is something to be aware of visibility into the status of each environment and the code you're running can be become a little bit difficult and the whole process requires discipline and knowledge on the part of the maintainers overall I like this workflow but it can be difficult to maintain its scale using a folder structure keeps the branches down to a minimum you only have a main branch and short-lived feature branches there's a lot less to worry about such as merge conflicts branches that need to be updated and the overall status of the code you can quickly see how many environments you are managing and what code is running in each environment but you also need to be extremely diligent in promoting code changes from one environment to another and it's easy for two environments to get out of sync you're going to need some process that can easily easily do a diff between all the directories and find the differences and you need a source of truth that has a definitive way the code should be the solution here is to make sure the team is communicating when changes are occurring and that they are disciplined in their process there's also the issue of controlling access on most VCS platforms you can protect certain branches and decide who can commit to them but there's usually no option for controlling changes at a directory level so if you want to lock down access to the production environment you might not be able to do it with a folder structure if a team member can commit to main then they can alter production you would need to use separate branches or repositories to lock down access control for teams that aren't as familiar with using Source control the folder structure is a good option it's easy to understand and it's easy to see what code is running where but it does require a lot of discipline to maintain folder structure also allows you to Define different backend types for each environment unlike the branching strategy so you can use terraform Cloud for production and staging and then S3 for development and testing maybe that's useful I honestly don't know using separate repositories is the most flexible option but it also introduces a lot of complexity and administrative overhead in my experience the primary reason to run separate repositories is to apply stringent permissions on access and deployment your production environment exists in its own repository and you can lock down access to the repository to ensure there are no unauthorized changes you can also do this with branches in GitHub and other VCS platforms but it is a bit harder to manage and doesn't have the same level of Separation the downside is now changes need to be copied from one reposit repository to another and it's much harder to visualize how many environments there are the status of each environment and what code is running in each environment you can't just look at a single repository and see all the environments you need to look at each repository individually and if you have a lot of environments that can be a lot of repositories to manage just like the folder structure strategy you can have a different state backend defined for each environment again not sure if that's useful but I'm trying to be thorough if you're in a highly regulated industry and you need to lock down access to production this is a good option just be prepared for a lot of administrative overhead I've reviewed teror Grunt in the past and I found that it wasn't for me I felt that many of its features had been addressed by improvements to terraform itself or through automation platforms that had a solid understanding of the challenges of running IAC in a cicd workflow but one of the canonical examples of using terragrunt is for managing multiple environments so I wanted to at least mention it here with terragrunt you can define environment specifics and different backends for each environment ter grunt takes the folder per environment approach while adding the ability to define the environments in the code itself you can also add child modules easily use different versions of modules and terraform within each environment and deploy to multiple environments in parallel now like I said most of these features have been addressed by terraform itself or by other automation platforms but teror grunt is still a viable option for managing multiple environments and I know it has its fervent supporters out there so if it works for you then by all means use it I'm not going to go into a lot more detail here but if you're interested in terragrunt I'll leave a link to my review in the description below the big three options for managing multiple environments are using branches using folders and using separate repositories each has its own pros and cons and each has its own level of complexity there are also other options out there that might work better for you depending on your organization preferred workflow and requirements you want to structure your environment management strategy in a way that makes it easier to maintain your infrastructure not harder to that end I would recommend considering the following questions how many environments do you have how many people are working on the code what is the level of experience of the people working on the code how often do you expect to make changes to the infrastructure what are your security and compliance requirements what is your preferred cicd workflow where do you plan to store configuration values answering these questions will help you determine which option is best for you if you have another option you think is better please let me know in the comments below I'm always interested in learning new ways of doing things if you'd like to take one of these examples for a spin I've included links to the repositories down in the description each one has directions for use and leverages GitHub actions to keep things relatively self-contained a special thanks to spacelift for helping sponsor this video and thanks to you you dear viewer until next time stay healthy stay safe out there bye for now

Channel: Ned in the Cloud

Views: 4,475

Keywords: infrastructure as code, cloud computing, continuous integration, devops terraform, terraform tutorial, hashicorp terraform, terraform syntax explained, terraform tutorial for beginners azure, continuous integration and deployment, continuous integration github, continuous integration and delivery on azure databricks using azure devops, terraform tutorial azure, hashicorp terraform tutorial, devops terraform azure, devops terraform project, infrastructure as code azure devops

Id: YcfWKy8YiLo

Length: 20min 39sec (1239 seconds)

Published: Tue Oct 17 2023