Azure DevOps Pipelines with Terraform and Stages

in this video we create an azure devops pipeline using stages and a terraform [Music] hello everyone i'm travis and this is cerralto's this is another video in the series on azure devops pipelines in this video we create a pipeline with multiple stages similar to the last video we deploy a virtual network and bastion host in azure only this time with terraform before we get started please like subscribe to this channel click the bell icon for notifications and share with a friend become a member for early and ad free access to videos and check out my courses on azure 80 hybrid identities and azure virtual desktop at udemy.com the link is below back to it we'll get to the demo quickly i promise before that let's take a look at stages and why we want to use them we can create multiple stages and add jobs under each stage we covered jobs in the last video we can add conditional logic to stages and jobs this gives us the ability to create some complex pipeline deployment scenarios making a stage dependent on the value of a variable or the success of a previous stage all helpful for a variety of deployment scenarios for example we may have one stage that builds an app or infrastructure then another that runs a test on that app or infrastructure then one that deploys the code if that test is successful the example coming up starts with an existing azure devops project if you're just getting started check out my playlist on azure devops the code used in this example is available in github i'll leave a link below let's jump into azure devops to get started here we are in the azure devops repo i assume you have the basic understanding of terraform so i'm not going to go into detail on the actual terraform code if you're getting started with terraform check out my playlist on getting started with terraform the link will be on the screen or below or something i tested the code locally to verify the terraform deployment it sets up a virtual network and azure bastion host just like the last video only with terraform i'll have a link to the files below if you want to try it yourself the dot get ignore file prevents the dot terraform directory tf state tf vars and other files from getting replicated to the repo this prevents local variables and settings from getting replicated to the devops repo generally they don't need to be there i did create a new terraform.tf vars file in the repo yaml files don't handle complex variables such as terraform maps easily it's cleaner to create a dot tfrs file in the remote repo also i have a storage account that will be used for the remote backend data we use a centralized backend to prevent conflicts if others run code against the same infrastructure let's take a look at the storage account we have a storage account with a blob storage container called tf state and that blob container is empty let's go back to the azure devops project let's add a service connection next the service connection gives azure devops access into the azure subscription to deploy resources it will also be used to access the storage account for the backend configuration let's go into the project settings service connections create a service connection we'll select azure resource manager and if we scroll down we can go to next we'll leave it as service principle automatic next login with an account that has rights to add security principles into the subscription verify you have the correct subscription added leave resource group blank we need to scope this at the subscription we can give it a connection name i'll call this one devtest service connection and click save that finished now we have a service connection for our project let's go back to the repo and create the pipeline here we are in azure devops let's go to setup build select starter pipeline here we are in the pipeline and before we get any further i just want to show something if you go to show assistant and type in terraform nothing shows up if you get results you're okay but if you don't we need to add the terraform extension we do that by going back to the organization not the project go to organizational settings extensions we're going to browse the marketplace and search for terraform let's add the terraform extension the one published by microsoft make sure your organization is selected and click install we can close this now we go back into extensions we can see we have the terraform extension let's go back into the project here we are back in the azure devops repo let's go back to setup build select starter pipeline now we're back in that same project and if we go to show assistant search for terraform there they are that adds the terraform extension to our organization so we can use it for our terraform tasks let's get started with the pipeline by changing the trigger to none that prevents the pipeline from running every time we make a commit next delete steps and everything underneath it we need to add some variables these are for the backend storage account start with the word variables we'll go to the next line and add our variables we have a variable for the storage account resource group name the storage account name the backend container and the backend storage key don't confuse the back end storage key with a storage account access key the backend storage account key is the name of the configuration that will be added to the blob storage account this one is named devpipeline.terraform.tf state we can change the key name if we're using this in a different branch next up we'll add two stages the first stage will verify the config the second one will deploy the code if the verification passes start by adding stages we'll go to the last line add a line and add stages we'll add our first stage after the hyphen type stage give it a name we'll call this one tf validate next we'll add jobs this may be unnecessary we only have one job but it shows how jobs are nested in stages and tasks in jobs we'll add jobs and then create our first job so after the hyphen we'll type job colon and we'll call this validate go to the next line and under the j we'll type continue on air we'll set that to false with this configuration the job will quit if it fails next add steps backspace to remove the dash let's open the assistant if it's not already open and add a terraform install task search for terraform the image running the pipeline doesn't have terraform installed on it we have to add the install task to install terraform before we can run any other tasks select the terraform installer tool i'm going to use the latest version for this example you may want to use a specific version for production workloads so breaking changes with a version update won't impact existing pipelines we'll add that under task let's add a display name we'll call this tf install let's go to the next line and move the cursor under the dash remember formatting is important in yaml we'll add the init command next we need to do this before we can run any other commands go to tasks and search for terraform again add a terraform task not the installer task this time make sure the provider is set to azure rm and the command is init and go down to azure subscription select the service connection setup previously the next three items are for the back end storage we can add the information directly in each field but to make the pipeline more manageable and portable let's use variables instead let's add the variable for the backend storage resource group next add the variable for the storage account next is the blob container and finally the back end storage key and make sure your cursor is in the correct spot clicking add will insert the task wherever the cursor is located click add add a display name init for this example that adds the terraform init command go to the next line for the next task we'll move the cursor under the dash we'll add the verification task next this will verify our code is good before it's deployed search and add a terraform task leave the provider at azure rm and change the command to validate we don't have to add any back-end storage configuration because that's part of the init command click add and update the display name with validate okay looking good next we'll add the deploy stage let's go to the last line and two spaces from the left for our next stage there we go add dash stage and we'll call it tf deploy we want to create a condition next so this stage only runs if the previous one completes successfully under stage add condition the condition has succeeded and we'll specify tf validate this indicates this stage will only run if tf validate is successful next we also need the stage to run after the first to make sure they don't run in parallel we can add a depends on statement after the condition statement and we'll add tf validate for this as well now this stage will be conditional on the first stage completing successfully it also won't run until after the first one runs let's add a job next we'll go to jobs and add a job we'll call this one apply again we only have one job in this stage but it does demonstrate how to create multiple jobs in a stage let's add our steps next the s and steps will line up under the j and job okay we need to add the terraform installer job and the terraform init command next we have to run these two as part of every job that runs the terraform command we could enter them like we did in the previous step or we could copy and paste them i like that idea better we do need to be careful with formatting so keep that in mind let's copy the tf install and init task and paste it under steps just the tf install and init task we don't need validate watch for formatting the cursor should be under the ian steps when we paste it in now we have the install and init command let's add the plan command go to the next line and move the cursor under the other dashes to add a new task we'll search for terraform again change the command to plan set the subscription to the service connector we added previously and click add update the display name to plan let's go to the next line and move the cursor to add another task we'll search for terraform change the command to apply and set the subscription to our service connection and add let's update the display name we'll call this apply you may notice with inputs we've got some squiggly lines it's telling us it's missing some properties but as you may be noticed some of these properties are for aws and others are for gcp so we're fine that does it let's save and run the pipeline so we'll start by saving once it's saved let's click run the pipeline make sure we're set to the dev branch and it looks like i've got a error on line 11 column one unexpected value variables so let's go and see what i mistyped we'll edit that it looks like i added two colons so let's remove one of those and try it again we'll save this and run make sure the dev branch is selected and also before we run it if we go to stages to run we have the option to select one or all of the stages we'll leave them all selected and go back click run we can see the two stages the tf validate and the tf deploy if we click on one of them we can see the jobs running that's a good validation passed and now it's running apply tf install finished init finished and plan is running and now applies running it will take a few minutes to deploy the virtual network and bastion host i'll pause here and come back once it's done that finished and we've got no red x's so we should be okay let's take a look at the portal and see the changes here we are in the resource group it created and in that resource group we have a v-net and bastion host just like we'd expect let's look at the storage account next here's the storage account we'll go to containers go to tf state and there's our devpipeline.terraform.tf state state file in the backend a remote storage location so that was successful but how do we get rid of resources let's go back to the repo let's create another pipeline that will destroy the resources we just created before we create the second pipeline let's go into our pipeline go to the options of our existing pipeline and rename and we'll change the name to apply pipeline save this makes it easier to distinguish the apply and the destroy pipeline we're going to create we don't want to mix those up now let's go back to repos make sure you're in the correct branch for this we're going to copy a file we can't copy a file directly so we have to open the azure hyphen pipelines yaml file and highlight and copy the contents we're going to use this as the basis for our destroy pipeline so if you highlight control a for all and control c for copy and if you're on a mac you know what those commands translate to next we'll create a new file we'll call this azure hyphen pipelines hyphen destroy dot yml great and let's paste in the contents of the other yaml file we need to make a few changes to this file first we don't need the tf validate stage we already validated the code at this point we'll leave the stages in but we need to remove the first stage so everything under stages to the second hyphen stage i'll just scroll down a bit so stage's colon is in place but everything underneath that all the way to the hyphen stage tf deploy is selected and i'll delete we can also delete the blank line now we have stages but only one stage next change the stage name to tf destroy instead of deploy we also don't need the condition and the depends on statement since we got rid of what we've based that condition and depends on statement on that leaves us with jobs change the job name to destroy we still need the install task and the init task and we can leave the plan task in let's go all the way down to the last task apply change the display name to destroy and something to note we're not editing the file in the pipeline editor so autofill won't work but we're almost done under command in the destroy task change the command to destroy so we're not applying it anymore we're destroying so we've updated the names and that all looks good let's commit now we have the ammo file but not the pipeline yet go to pipelines new pipeline select azure repos we'll select our repo and instead of starter pipeline use existing azure pipeline yaml file set the branch to dev that's where we created the file and under path select the destroy yaml file and continue now we have the destroy pipeline click save under run don't run it yet though we could run it from here but let's go to pipeline and we don't see it here under recent let's go to all there it is now let's change the name to destroy pipeline and save now they're distinguishable between the two let's run the destroy pipeline next make sure that we have the correct branch selected and run we can view the jobs the job's running this will take a few minutes to finish i'll pause here and come back once it's done the deployment finished successfully and the resources have been removed that is how to create a terraform pipeline in azure devops with stages to validate apply and destroy resources in azure i hope this helps you better understand how to create stages in azure devops pipelines and use terraform with azure devops thanks for watching

Channel: Travis Roberts

Views: 39,091

Keywords: Terraform, Terraform init, terraform plan, terraform destroy, Extension, Repos, Repositories, Azure repos, DevOps repos, CI/CD, Continuous development, continuous integration, pipeline, DevOps Pipeline, DevOps, Azure DevOps, Git, Server, Code, Cloud, Cloud Services, Iot, IaC, Infrastructure as Code, azure devops tutorial for beginners, DevOps Job, job, dependency, dependencies, service connection

Id: d85-KD9stqc

Length: 22min 26sec (1346 seconds)

Published: Sun Jul 31 2022