Secure GitOps Workflows with GitHub Actions and HashiCorp Vault

all right so hi everybody my name is xiao and and i'm partner sc covering the asean region and today i'll be bringing you through these um you know how you can actually secure your git ops workflow with github actions and hashicorp vault so you know before we go into the details of how we can secure it right i think we first need to set some basis and understand what is a ci cd pipeline so a ci cd pipeline comprises of a series of steps that are going to be executed automatically every single time we make a change to the code and then push those changes up to github right so the goal of this ci cd pipeline is to really make sure that at any time we make a change to the code we have some processes in place that is going to automatically run the ci build of the package and then automatically attempt to deploy that change off to our infrastructure so how does github actions fit into this right so it is a platform to actually automate developer workflows and the ci cd pipeline is just one of the developer workflows that are available so essentially they are a small like bits of codes that will be automatically executed in response to something happening in or to your repository so the things that can happen in or to your repos are known as github events so for example you can configure code to automatically run anytime that you commit some code on your machine um and then push it to github another few examples are like maybe anytime a pool or merch request is created um or closed um anytime code gets merged into your primary branch so these are just some examples of github events so there are many many events that you could look out for right so um anytime one of these different events occur to our github repo we can choose to run a set of workflows but there's some problems when it comes to this or maybe some challenges when it comes to this so we frequently know that we need to like inject certain secrets into our city pipeline um and then github actions would be a great way to trigger events based on on how we want to build test or deploy our code so for example in terms of the challenges right in a cic pipeline maybe you would need a credential for a cloud provider um to trigger like an application deployment or maybe you are uploading an application binary into a storage bucket and you need some form of credential for that so but these are just some examples right but this definitely highlights the need for us to do some form of secret injection into these kind of workflows so there is some challenge right when without using um some sort of secret management solution so um because users will then end up hard coding these credentials into their pipeline or even their source codes right and it will actually be a disaster if these secrets end up in the wrong hands especially with github if you put it like a public repo um anybody can go in and see and and collect all these information all these secrets right and this will be a disaster so instead of hard coding credentials this is where hashicorp comes into the picture to enable teams to securely store and tightly control access to tokens passwords certificates encryption keys um to protect all our machines and applications um and we use this concept of the secret engine to do so so a secret engine um they are essentially components to store generate or encrypt data and they are extremely flexible right so it's very it's more easy to think of them in terms of their function so we have some secret engines that simply store and read data like your static kv store or maybe other secret engines that connect to other services to generate dynamic credentials on demand so now we have an official support for hashicorp vault github actions the vault github actions actually allows you to you know take advantage of your secret source from your vault infrastructure for things like maybe static or dynamic secrets and then inject these secrets into your github workflows um this integration also allows you to authenticate to vault using a various number of ways such as tokens add role github off method also so once you have authenticated to vault um you know then vote will then allow you to fetch those secrets based on what you need and just to read the rate again right um you know you probably heard this multiple times but um our our philosophy or our principle right is that we always put workflows over technology so what you will see in the demo later we are going to use the same workflow essentially the client in our case github will need to authenticate to a third-party source um and then once we have authenticated it we will be given a token we will use that token with a set of policy that is defined to it and then we can use that token to access to communicate revolt and access the secret right so um that is the base that i want to set and let's hit the demo now so let me just show my screen desktop tool um just want to check uh this is my demo of um visible yes okay thank you all right so um this is essentially the um you know this is a very simple code right an application code that i have downloaded already um on the side of my visual studio you can see i have four types of um terminals running here all to serve different purpose right so um first we're gonna just use the github one um and we're gonna just deploy the application right so once this application is deployed um we can actually see that this application is also using a hard-coded secret um this is not what we want right so um we can see that it's going to we can run a command line to echo the hard code receiver right so if i do this we can see this is the hard coded secret so now um you know we are showing a default hardcore secret we don't want that to happen i'm going to start up a vault server it's in dev mode um having the root token as default so before we can go into configuring our vault server we of course first need to you know export our variables once it's done i'm going to configure default server right so the github workflow later it will actually be a secret define at a path secret slash data ci so we're going to create now a secret aesthetic secret define a policy to access the secret and also generate a token um that can that has um attached the policy to it and access the secret right so now we are creating the secret we're going to test whether the secret is retrievable yes it is secret from vote right um and then this is the policy that we're going to create so essentially this is the policy name ci secret reader the policy path enables us to read at the path secret secret slash data slash ci so once we see that the policy has been successfully uploaded as ci secret um reader we're going to create now a variable token to capture you know uh this token with the creator policy so as you can see we create a token here for token create and we attach we specify the specific policy that we want to attach to this token so to test right of course everything we need to do we need to test whether it works first so we're going to test with this we have indicated that our token is going to be that github repo token and we're going to do a vault kv get off that secret path and okay so we can use this token to retrieve the secret right so we have determined that it's usable now our next step is to create our github repo so i'm just going to create one here vault action snapshot so now um we're going to set up our github repo to host our code um in our case so i'm going to do a commit and after that we're going to use the push existing repo from the command line so if i do a refresh of this page we can see that our code has been uploaded into our github repo um our next step right um is the step of authentication because uh our runner in so our runner is our workflow um like worker right they will need to authenticate to the vault server and to do that we will need to authenticate with our github token that we have already created so um what we're going to do now is we're going to copy the github repo token first and we're going to set up a secret in within our repository itself so if i go to the repo i click a secrets tab over here i'm going to add in a new repository secret i'm going to name it token and then the token value is here right so once this is added yeah once this is added our next step is to set up our runner so um to run github actions right you need to set up runners um and in our case we're going to set up a self-hosted runner you can actually also um utilize the github runner but that is like on a pay-per-use usage or you know it's a sas offering from github itself but instead of doing that i'm gonna use a self-hosted runner means you're running the runner on your own platform so i'm running now a mac os right so i'm just easily you can just follow all the instructions that are available over here so we're gonna we are going to use this this here download the latest um runners essentially this one is just um following the instructions right here but nothing extraordinary needs to be done um so we're just going to extract the installer so once that is done it's going to use our github runner tab so just yes okay uh okay this one so we got now we're going to start the configuration of the runner itself right so once i execute this later shortly we will see like um something pop up which is this the self-hosted runner registration so it's not authenticating you know to github once this step has passed we're going to begin the configuration of the runner so we're just going to use the name of the runner default group so just gonna hit enter i'm gonna name this like snapshot the runner um it's vault extra labels you see that the runner has been successfully added um and then after which i'm gonna use the default work folder so once this is done you know we see all these check boxes you can then start to run this runner and once this is activated this runner is currently listening out for jobs right so this step is done our next step now is to configure our workflow so um github actions right they express operations that they want to accomplish through all these workflows then these workflows right as mentioned at start can be triggered based off of different events that take place during your entire lifecycle um in the repository so now we're going to create this hidden directory so it must always follow this directory github workflows um and then i'm going to create a workflow file named image builder and i will run through the code in a bit once i copied them over all right so moving to this step essentially this is a very very simple um github workflow um the name of the workflow determines how it will actually appear within the action interface of github itself which you will see in a bit once i've activated the workflow um the on specify right on part of the code um specifies you know the when this workflow will actually take place you know so in our case it's going to run every time we push a commit to our repository and then at a high level this image builder workloading it needs to you know check out the source code import the secret build the image with the new secret and this is essentially it so we have also added a job to the workflow named build so our job is named build and it's running on a self-hosted runner on the build job right can you know you can define multiple steps um of it right so it always follows this flow you have the name the job and the steps that you want to run um and we can see this is we are using the default um you know the core github checkout action so after the source code has been checked out um we need to fetch the secrets from both so our second step will be the import secrets and we're using the hashcode vault action a version 2.4.0 and this part is where your configuration comes in right the width so we are using this action with this set of configuration so in our case we are using you know we're defining the url of our local hosted volt server we indicate to skip the verify of tls and we have also put in the token right so this is the authentication engine um the secrets dot vote underscore token is what we have defined earlier in github it will always follow this um this uh uh variable first right so it's secrets dot whatever so in our case it's secret stop token and after which this is the main part right where we are following the you know where we are defining the secrets to be retrieved from so we are we are retrieving the secret from secret slash data slash ci um and the key is at underscore secret and the next step of our um you know process of our workflow is to build the docker image right so um that's why it's named build docker image and then we're going to see a run docker build and we're going to build it with the new secret so it's going to be envi underscore secret so this variable environment variable will follow the same name that you have indicated here there are ways also to change the variable name i'll show you later while we have a bit of downtime when i'm uploading this workflow to the um uh this to trigger the runner right so now that i've ran through the workflow we're going to you know do it to do a push and try and and we want to activate this uh github runner right so while that is happening so this step may take a while so what that's happening i'm just going to switch so this is the essentially the github actions interface later when we see the you know runner running this will change um and it will update to show the workflow um activated so while we are waiting for that to happen let me just share with you this page right so this is the marketplace where the where you can find default github actions so we are running the latest version which is 2.4.0 um and this is essentially the place that you can see right um and get examples on how to use it um see what kind of authentication methods you can utilize so that's why we are using the default one now which is token you could use adroll you could even use like the github token for it as well and one thing that i want to show is in terms of your like um key syntax right so this is what i was talking about earlier essentially if you name it like this npm token it will take this as the environment variable so that's why we have defined it on our right here env underscore secret so it will always follow this name that you have indicated but if you want to set the output name if you want to define a specific variable name you could also do it as such so you this is the secret um the key right and then this part is where you can specify the specific variable name that you want so i think this is done okay so if you actually there are multiple ways that you can see whether you know your build job or your workflow is actually completed so i switched over to my runner tab over here we can see that it was listening out for jobs and then it's running the job the build job right this job code built and the job built was completed with the results successfully um successfully done so i'm going to refresh this page okay so we can see this is the workflow we can even over here we can see what had happened so we had set up the job right and then after setting up the job we run the actions the checkout actions um then our next step was to import the secrets so we get the vault secrets um you know with our workflow configuration over here then we built the docker image and essentially that was it so post the run what other actions he needs to do and then complete the job so now we're just going to test to see whether our you know our application our docker image has been configured with our new secret and essentially this is it right so we see that instead of using this is a hard-coded secret we are seeing a secret that has been retrieved from both so that comes to the end of the demo um but before that i also want to show you some resources that you can leverage so you can you know we have a learn guide right um the hashcog tutorial it is up to date it is um it brings you through the entire process of how you can get this up and running um and of course you know if you want to go into details of the specific actions you can these are some um urls that you can also go to so this marketplace was what i showed you earlier and if you want a you know a deep dive into the into like this specific github um workflow so there is a one hour youtube webinar um from uh i believe the the github partner engineer as well as our vault technical marketing they were speaking it was a co-host together they were speaking more in depth of the of this integration so if you want to spend more time to learn these this thing more you can go into this webinar and see more information

Channel: HashiCorp

Views: 134

Keywords: HashiCorp, Vault

Id: rhB72KFiL48

Length: 20min 44sec (1244 seconds)

Published: Mon Dec 13 2021