HashiCorp Live Codes Vault & CircleCI, Part 2: Using GCP Secrets Engine in a Pipeline

well hi everyone welcome I'm gonna make sure that we have our audio correct so if someone in chat could just let us know if our audio is good that's test yeah we're I know we're on a little bit of a lag in the twitch stream so feel free if you yeah I can hear you now okay thank you thank you much appreciated all right well we're back for part two we're back with angel and I'm rosemary all right yeah do you want to do your your intro as the second time for those who haven't joined us in the previous one yeah sure so my name is angel Rivera I'm a developer advocate for circle CI and I spent a lot of my time well usually spend a lot of my time at conferences and meetups and hackathons and all these tech conferences and events basically you know speaking to the developer community and learning how you all are using software and getting ideas on how I can you know help you use your software better and technology better as well as bringing some ideas back and feedback back to the development teams and product teams that circle CI so that we can build useful features and our products and make your life easier we like that yeah I like my life being easier when I build things yeah we all do right no and for those who have not seen me here or have just joined us on Hoshi Corp live before I'm rosemary Wong and was one of the developer advocates at Hashi Corp I primarily do work with terraform but I have worked on and worked with a number of the Hashi crypt tools today we're going to look at vault so if you're interested in vault in particular we're sort of we're sort of taking this from a problem statement of well angel has a pipeline that he'd like to configure to interface with vault in various ways shapes and forms so last week what we did was we injected a static secret which was your docker hub user name and password yes so what let's just I guess maybe run through the pipeline real quick kind of get people who were involved yes so what I have here is it is a project it's called a lot of weird things here I had to reinstall a vigil city code like literally ten minutes before this thing because of these Mac updates I don't know if something's weird have it happens so yeah basically have this simple project here it's just a simple node app that kind of displays almost it's a static web page and what it is is very like I have some tests that run right so when your pipelines obviously you want to test your code so I use this app is a real simple kind of bug you know how before my how to's I don't want to focus on the app because you know I could get multiple distributed systems and micro services and a lot of folks when I'm trying to teach them something about pipelines they kind of focus in on the apps and how they're written versus like no you need to focus on the pipeline and the automation in that space so what we're doing is yeah we we're leveraging vault to store and retrieve and protect basically app secrets like API tokens my username and password to docker hub so that I can push you know new versions of my docker images so this pipeline here this is a configuration file so if you use circle CI or any kind of CI tool modern CI tool I should say you're probably using some some form of yeah mo and at circle CI we use the ml for the configuration file and this particular one is like the config dot UML file right so if you're using circle CI this will probably look really familiar to you if you're not then you need to create a folder within your you know repo or your project and have a hidden folder called circle CI and within that you would have a config gamma file which is what we have here and I can run through this real quick just to kind of briefly recap in circles CI you know it's the ml so these are keys here and this is the jobs is just a list of jobs which are a grouping of commands that you want to run in the a certain way so what we've done here is the run test right so that's just a label for a grouping of commands in the job so we have a jobs list and this run test is a job and when you want to run and within jobs you need to identify where you're gonna run time you're gonna run your your builds on and this one we're using docker right docker image and since this is a new project we're gonna use this no 10 and in this phase right we're literally just testing the application so once you have that you would create steps and these are the actual commands that you're going to run within that job so right we're gonna install our dependencies well the first thing is this is just a basically a git clone right right from from github so when you push up a commit upstream to your branch upstream circle see ID text and change and then runs this pipeline right against against those changes in that branch so the next step down here is be to install NPM dependencies and then install them it will and then the next one would be run the test so I used mocha as a test and with some reporters called mocha awesome so that I can create test results and then export them into a nice HTML file which is really cool so you know it pins your test results to the build in the circle CI dash bar which will show you a little bit later and then I'm storing these results test results here and this is how you would store the test results or pin them to your to your bills right and then finally this is where we kind of left off last time we got together and it generated this episode two which is I'm really excited about so this is where we're gonna be focusing on and here is basically what I'm doing is uh in this step is where you know we're actually the job is called fault fault off so that you know you can name that whatever you like we just use fault off as the name for this this particular job and as you can see here the docker image is a little different we're using Rosemary's kind of she baked an all-in-one image that has all the toys but yeah it's all good I mean that's how that's how we roll ready that's how docker rolls actually it's not your it's not any different right yeah my images are just as big how do you and I have nothing but at the end of the day this image here has all the tooling so although it's kind of pre baked right and she installed I believe you can I believe you installed the the vault agents right console you probably have terraform on all the all the cool has she Corp yeah well not every has she cooked so that's the thing it's a vault and terraform primarily and there are some other little little things thrown in there docker is also thrown in there g-cloud g-cloud command line and cube CTL so all of those are thrown in there and then I just build it every couple all the all the cool new toys right yeah you need to run in your pipelines especially if you run a ton of infrastructure pipelines right yeah so once you have that image right so the this this step will take and basically grab this image and then again right we're gonna check out some code and then we get to the interesting parts so oh yeah let me jump on this real quick so since in this step we're gonna build a docker image right a circle CI has this feature where you can tell it to kind of set up a remote docker image and then from there are remote yeah a remote container like a sidecar so that you can then build docker within docker if that makes sense right so the only requirement this is that in your image like up here it needs docker the docker client install which rosemary has so once we get there or once we get past all that and the runtime setup what we're gonna do here is basically just set up a couple named environment variables for the name so that when I push up to docker hub you need kind of a unique name you want to tag them with like latest and then whatever other you know you kind of number or whatever so this is what I'm using the circle CI it's an environment variable that's kind of floated down into the runtime from the host automation or the host system which is circles together but this is where it gets interesting right here are some environment variables that will be will we'll have to set them up again right and or have you set them yeah they've been preset so okay cool yeah the good news is that you know you can when you get a roll ID so this is a call to vol april for those who are curious about the last week replay what we covered and i put the link in the chat so you can go back and look at all a parole and some of the information there but the idea is this is a roll that's associated specifically with this pipeline and it means that you have to reissue the secret for a certain cadence I have already reissued the secret and I've already put it into the vault secret ID environment variable and vault roll ID environment variables so both of those have already been added let me let me just grab just I'm gonna have to do this anyway so I'll do it now let's grab it let's just take a look at the project within circles TI's so that folks are because we are recapping so it can get confusing there and yes there's a video of us the previous steps we took but just to kind of make it a little bit clearer and then we'll just jump right back in I still need to configure my box a little bit but that's okay we'll figure this out so yeah so yeah that's not it I think I'm in yeah and this is so by the way this is the circle CI dashboard for those of you that don't know and I need to find yet so right now we're in the middle of upgrading our UI so when I click on this failed build it's gonna change the UI to the new yeah there you go so the new version so we're like in this weird flux state so yeah as you can see there's an old experience button what you saw was the old experience which is fine because again we're in the state of flux but this was a bill that guess you did yeah pushing something I really make any changes I just did a little toggle right and it failed because it couldn't authenticate to fault Wow yeah oh wow see look you got it it just it just hung for five hours we should have paused it Orkut canceled it but I was like yeah that's not good I get I get unlimited I can get unlimited can build minutes anyway so so right so what rosemary when we're talking about when we're talking this these two like environment variables right bolts roll ID and volt secret ID we've set them up in this project which if you so this is basically kind of the the admin page for the actual project right so your repo so every project will have well could have different environment variables of really affiliated with it and this is where you would set all that stuff up right so you can go to the Advanced section and it tells you like hey you can control how you know the circle CI trigger is triggered to two things on your to perform CI on your on your repos but this is where we're actually gonna be playing in and as you can see here rosemary already set up the new role ID and in volt secret here so that's that's where those variables are coming into my pipeline just to give everyone an idea she's also set up the vault address to her instance the vault running in on the internet somewhere and I'm glad it's protected and by the way this is where you would enter so environment variables are inherently protected in circles see as you can see you they give you a little bit of a hints of what the values are and that's okay but it doesn't give you the full value so this was we had a question in chat how do you control timeouts in circle CI where you know what we were seeing before right there was an authentication failure and yeah how do you control this so I just want to be clear here and let's see there is a timeout key set you can assign that I just wanted to show everyone let's just I want to make sure that I give you the right information because I don't use it and I probably should normally you wouldn't have to and rosemary and I encountered that last time and I'm definitely gonna investigate this a little bit better about seeing how you know I understand why bolt is doing it because bolt will just keep trying to keep trying to keep trying but and I think I asked you right is there a timeout we could set and right and but that might be a feature you know that could be added because at the end of the day like you're just basically you know you're DDoS in future is not intended to be used you know in that manner I guess and I think part of the config maybe we need to look at and an update but yeah I mean these things captain right so when you're developing software that's how it goes and you won't encounter these things until you watch me are you you encounter them when you're doing so yeah so let me see I believe there was a timeout somewhere did you see it no just bump up the font a bit oh sorry yeah here we go how's that better oh sorry there's a bit of a lag yeah that's a little bit better thanks okay let me do it ctrl F in and time out yeah here we go so there is a no-no output timeout the last time command can run without output so yeah I believe this is what you would use so 10 minutes was yeah that smells about right so there is a default on circle but if you know you're paying for build minutes you would definitely need to if you're gonna run vault and you know yeah I would I would recommend that and maybe we could stick one in ours yeah maybe but again you know that's okay will but I just want to let people know that there is a mechanism there I barely use it and probably shouldn't but you can also do a sleep command so we went and you know you could even write bash to do that as well so if you know there's there's a couple ways to skin that cat but there are facilities within the yeah mall that you could use like this guy here yeah what were you doing though it's something called vol agent and that is actually the way that while agent was intended to be configured was as a sidecar so it's continuously running so part of this is the side-effect of vault agent itself which is going to continue retrying if you do a vault retreat you know you've all read from the KB store which is not using mall agent you know you could very well say that this isn't retrying it can't reach the host and it stops so you know this might be just a side effect of the configuration we're using which is vault agent and you can see here I mean plus it up a little bit just in case somebody can't see it no I just lost everything there it is so yeah as you can see here this is this is how you would implement this in your this time out rule inside of AAA runs step right so so you know what let me just grab this and say how long does it off take like 30 seconds or something yeah less than that I would say even but a 30 seconds is a good fail 30 second is a pretty good failover yeah so so let's just implement that right I mean we can just say here like we would do it before is it in the right oh yeah wait a minute so now this is four so I said I'll tell you what let's let's just give it a five minute that should be enough to build them and push no I mean 30 seconds is I think sufficient the problem is we're building we're building a docker container oh I see we've seen without right right yeah it is outputting right so it's still outputting so I just think what this line is next to like is under run or something it might not be as part of the command output oh I see what you're saying I got you I got you all right well we'll leave we'll leave it at two minutes and see but basically if you're interested in timeouts for circle see I will just will go with this that there is a new output timeout so that means you know your your build isn't it's just basically hanging but I don't know if this would solve oh you're right this won't solve that problem yeah yeah because it is outputting yeah we just do kV grab you know you do okay well kV reads or read KT so you wouldn't use of all agent per se the reason why we're using about agent here for those who are curious it's because it renders it to a file automatically for you and and it's very useful for larger pieces of metadata right so you don't have to write you know echo into this file etc right yeah so the next thing I'm gonna do is before we get started with rosemary that's let's get that build running so I need to go into the site is this okay to show that yeah it's you know that this is a this is a vault instance on a kubernetes cluster it's you know it'll be destroyed after this so if you claim to use it it will probably disappear is it working it is working I just tried it it's not the right vault address I'm looking at our slack I think you said it to me few days ago can you send me it again let me see yeah that's not the right one okay but yeah if you could send a new one over that'd be great awesome she has all the cool toys everyone I don't even know what important stuff I have above it like just it just appears some of this if you sorry everybody that I just switch screens by accident for those who are more or less curious about how I deployed this I just have a vault on gke repository that just deployed this I'm gonna put the token all right it's protected I just don't all right so this is what it looks like never this is what it looks like right yeah yeah this is the vault dashboard were you I'd further for their service and this is the the I guess what would you call this a secret that's the key value engine specifically for new Jas circle CI so that's a path that is an engine that's got key value backing to it and it's right good except this she set this up pre-staged of course to help with getting it set up and then pipeline right yep and here I would just create a new secret is that right yes that we should keep it all right yeah we can create paths for the secret what did we call the secret I don't remember I think we need to refer to the vault agent yes yes we refer to the agent HCl right yeah okay I believe is just soccer or yeah we'll figure out where is that ah there it is so this is some patchy court what is it called caching Corp language right yeah so Hoshi curve configuration language yep yeah which is basically a DSL domain-specific language that works with the hatchet Corp ecosystem it's just an easy way for you to kind of or yeah for them to give you the ability to write the same kind of data structures and consume them in all of their products so if you're writing terraformed this should look very familiar right no man jobs yeah right so yeah so what we would yeah we called it just us RA the key was us R and then the other key with a password but the we have to label it as docker hub dr. hood right there you go that's okay great so what I'm gonna do is set up that path which would be up here right mm-hmm yep yep docker hub ah card hub and then here just my user another one called PWD and I'm not going to show you all their password do we spell I think it's PWD I never I never take that uh oh there we go yeah I'm that I if I set it up as good I never typed the full thing I hate that Oh anyway so alright let me go I'm gonna again okay I'm gonna change my virtual background because this thing is just flickering too much okay sorry everybody you're gonna see the green screen yeah can I remember it yeah I can so the reason why we're doing this is obviously right this is one way to keep my keep passwords and secrets stored securely in a secrets management tool and it's a one-stop shop as well right so you no longer need to like have spreadsheets or post-its you know with your passwords on it and sharing maybe even in Word documents I've seen that too before yeah yeah that do that yeah so this is one of the other tools or one of the other use cases for a tool like balls right you can just keep and then the cool thing is you can access this stuff via API which is really neat yeah so I've added my user key and then password key and then the values are obviously I'm not going to show you that but we're gonna save it and now we have it yeah yep in the system and now I'm gonna go away from it rerun our pipeline so in theory this should all work again okay yeah all right so let's let's yeah jump back over to let me grab this and put it in put it right here I'll pin it so we're probably going to refer to it again indeed and yeah and then yeah here we go sir ci so let's go back to our project project all right that's good just jump back this way quickly the new UI yeah I know you know it'll I hope it'll jump into know there's the old one that's okay but what we could do is encircle yeah so we have this this failed table so let's try to run it so normally this would happen you know this disree engagement would happen if if you push some code up but what we're gonna do just rerun the failed bill which which should work so we're gonna say rerun the workflow so the workflow is basically real quickly yeah you can see here we have a workflow which looks like this when you click into it and we have two jobs so if we're running if we're going back to the config file yeah we have two jobs and if you look down here the workflow is kind of an aggregator of your jobs right so you can orchestrate how to to run jobs meaning you know we have the run test job and then we have the vault off job right now because I don't have a dependency on one another they're running in parallel and that's why it looks like this there's there's no you know connection but if we were to run them sequentially then this vault would be connected and it would run one thing first right so it's almost like it's blocking yourself right now since we already know the tests are going to pass right and all that good stuff then we can just run it in parallel so let's jump into the vault so this is what's happening and that vault off job which is not letting us log in so secret ID invalid state ID all right we're gonna reissue a secret ID back yeah you can you want to do that in the backend there yeah I can reissue it that way we do not expose lest we expose everybody to our vault secret yeah so what I'll show them what you're doing why well yeah by just jumping into the hall yeah organization and I think their own thinking yeah so going back to those environment variables the reason is they expired right so we need to jump back into that project variable setting and rosemary issue can you make them a little longer I could I mean I would redeploy it okay I could make it longer no worries no worries you already did it no not yet hold on okay yeah so so what she's doing is basically you know repopulating these so that we have some valid keys in the runtime and that the agent can then call back into the vault system and get the credits for docker hub which are these yeah I'm basically getting it for this role right Oh all right Mariette sorry so that's that would be policies right yeah this one yes that's it cool so once we have that we can rerun and let me cancel that one before I run out of credits yeah well okay I know the owner uh yeah about that all right we're gonna try this one more time I'm gonna actually cute yes so that's yes so I'm gonna cancel this run so finicky and by the way it's it's all about right the access and then the tokens which is what we're trying to illustrate so once this is all up and running you know you should be able to and you understand how these secret magics always work you should be able to not have these problems as far as I think the nodes are failing it's an automation kind of glue so that's where you would usually after you issue this you would after you issue a role ID for example you would actually want to use a you know like a trust what they call a trust or Orchestrator so this could be terraform etc to push directly to the circle CI environment variables that's one way you would do it but in this case we are not unless you can you like Crom that too right or is that weird saying that kept some sort of yeah you could also cron it too and you know so there are a couple ways that you could organize the trusted Orchestrator in this case we're just opting to use a kind of this manual approach but you would not tend to use this manual approach we are doing this for the sake of not keeping your stuff up too long Plus this is cool for folks to kind of see the inner workings very good at least the reasons yeah all the reasons are educational yeah let me i'm putting the new one in come on okay all right all right it should be updated if you want I can kick it out or you can kick it off I can kick it off no problem all right yeah so let's do this again I've got to go in it and of course so yeah we're gonna rerun this from the start and then from here right the one we're interested in is this so you see they're both they should both kick off at the same time I'm sure the test is already you jump in there it this one should kick off rather quickly yeah so yeah so that's the beauty of circle is circle CI is that you know you don't have to worry about any infrastructure like normally if you were running like a jenkins box or something like that you would have to stand that up and and take care of it we provision all the resources for you in the back end so all your compute is kind of managed through us and then all you have to worry about is your code and then it triggers with every commit which is simulated by me clicking that rerun button so let's just jump into the vault off you seeing anything yeah there you go it's pulling down my it's building that building yeah it's building the image now so let's jump over to there's the environment variables that we were talking about seeing there they're protected because the system this agent went and called back over to the vault system and brought that in right and then populated without having to expose anything so that's what's really cool about this yeah and yeah so what happened here was yeah so it ran this is where the the vault server logged in got got what it needed and and you're still not exposing it even the in the output thankfully and here we go it's building my image and oh wait I guess it submitted it so let's check on docker hub we should have have a freshman image sitting up there oh if I spilled Dockery yeah it's Friday no I'm the worst like I admit it it was a lot someone actually heard my people I heard the request for pizza last week so you know we're just blaming it all on like pizza for Friday Friday brain and I was like I you know whatever it is a minute again we have yeah we have an image here right pushed a minute ago which is really cool so let me see if I can refresh them hopefully they'll show up here to docker hub sometimes I don't know there might be some late lag yeah see so build 47 so if we go back to this thing let me just so yeah build number 47 and that's the number I use the build number for it from right from my config file so that's great we have that's working again I guess the next step is gonna be thinking about the piece where I'll let you talk about so I think angels request was I want to learn about dynamic secrets right because what we just did was push static secrets to a key value store and we configured our pipeline to retrieve from that so that's one learning experience that you would get from vault but the real value from vault is the dynamic secrets and basically what it does is that vault has a ton of automation that you can extend it with and it will handle the leasing revocation creation of any you know generally most secrets associated with certain providers so one good example of this is like a cloud provider and what you'll be able to do is revoke or generate new keys on demand with certain times to live time to lives you know so you get to choose how you how long you want those credentials out there but the idea of that is a question about you talked about time to live maybe some folks don't understand what that is yes so time to live is how long a secret is allowed to be used very very you know very straight in some ways very straightforward kind of concept but is that very nuanced right so time to live is very tunable it's very dependent on the type of secret that you have so if you let's say half maybe a secret that's really difficult to change let's say this might be your you know your home Wi-Fi password for example that kind of stuff you might have a time two-way of three four months five months maybe two years if you decide to yeah right I don't think I've changed mine in ten years okay best practices at all right but then sometimes you would have secrets that you only want for the lifetime of a container that has come up and after the container is done you shouldn't have those credentials so what vault will do is basically help to in those help well it will enable you to tune those time-to-live parameters depending on the situation circumstance and type of secret that you have and so that's where most people enjoy using vault if you use vault on kubernetes for example your you can set up vault integration with kubernetes in the way that your secrets are limited to the lifetime of the kubernetes pod so if the pod dies that credential is no longer valid no longer available so it's very powerful yeah it's really cool we're not going to do that today what we're going to do is actually set up GCP something called GCP secrets engine for vault and it's going to issue a new surface account every time we want to deploy to kubernetes so every time we want to deploy this circle CI no js' application to kubernetes we get a new Google service account every time nice yeah exactly well yeah we'll see I think if we can configure it oh we can do it yeah yeah it's not it's not tricky but to to a certain degree but it is sometimes you know again it's the time to live you even saw this today when we were trying right now but when we were trying to do the secret ID I tuned it just a little too short so that means that we have to get it revoked etc so what this will do is actually it will it in a way that is automated so I don't have to go and get it reissued like I was just doing the vault so we're gonna start first I guess with configuring a vault to do this right so it's not like if you suddenly do a vault command you suddenly get the GCP secret engine there's a little bit of stuff you have to configure first before you get around to you know getting those secrets dynamically so give you access to my room to my desktop so you can drive or yeah remote control oh we're gonna try this yeah all right yeah why not okay can you oh wait I can give it to you sorry Rick I am clicking okay women yay all right now try this at home everybody yeah yep good there we go icon things all right um so the first bit is that you'll notice that I did a lot of kind of like initial creation and configuration of alt and mostly this is because well it gets a little bit exhausting configuring vault by hand and you could bash script it you could go vault you know enable this Cavey engine right but it sometimes gets a little tedious then you forget which engines you enabled which engines you didn't enable where you put things which policies you put it in so I just do it all through terraform so if you're curious and you're digging through this repository what you'll actually see is a GCP terraform file under vault and terraform what this will actually do is configure the vault GCP secret back-end so this is equivalent to vault enable as you know gee I think GCP Secrets something like that it's the same as if you logged into vault and typed it into the CLI in this case what we're doing is we're saying I'm just going to enable it with this resource this is using the vault provider for terraform so I have as you notice a time to live and the time to live is about 10 minutes so the service account the reason why we're doing a service account for this instance is that it enables us access to the kubernetes cluster you can use tokens and in this case and in that case it would be really nice because the tokens are unlimited you can just keep issuing new tokens however in GCP service accounts are limited or not service accounts service account keys are limited so you can only issue 10 for every service account yes you have to kind of look at your use case right and then kind of identify alright do I want to use tokens or and and I think everyone's obviously everyone's setups different everyone's regular Parmenter different policies are different so make sense so basically if I set this to 10 minutes you know and and with circle CI your builders run pretty quickly pending any anything that we've done that to configure the pipeline you know that that it's expanded or it runs it for like 20 minutes or five hours in this case but the idea is that the you know this ten minute window should be sufficient for this particular pipeline I mean we can even make it smaller if you want I don't know if you want make it smaller the question about that so with a CL and the whole framework we have the capabilities of I agree right here so let me variable file right so let's say we we could we could kick off maybe create a variable here that says like what was it the name of that CPT yeah so this this value here right you can create a is it this one no this one both of them okay so let's say you know you know your builds are on average like two minutes or three minutes you know you can query circle CI but they could be a step in your config file that can query circle CI and you could literally when you run this through terraform it learn to terraform or now yeah it is vote so running so it's not running as part of this pipeline right now it's running in terraform cloud so it's leaning in circles di the the terraform piece configuring vaults is actually running into right but what I'm saying is in this pipeline I could have a line that can call it like it you know there could be a step that calls back to Circle C and say sorry what was the the last amount of time for billed for that specific job and then you could automatically populate that right and then you can use a variable right from and then terraforming yeah that makes sense yeah so you that you you you don't even have to worry about it you could kind of go off of what you did before exactly I don't know just an idea sorry I'm sorry it's really cool I mean we I mean maybe that's like part seven yeah can you get it you can I think it could it's actually very it's very possible to do that because then you get a better sense on average of what kind of time to live that you're looking for right sorry I just saw no sense yeah yeah I just I just wanted to like show people as well that you know this is a very dynamic and flexible system this the the HCL in the terraform system like if you have there's a variable component to it so if you have anything that's you know obviously very varied you can leverage that rate when you're building out your terraform because when I first started using terraform I was a I'll just use these to define some values meanwhile I didn't realize that in the CLI when you're crawling terraform you could actually Thoreau you know dynamic values here so yeah so continuum sorry no no no no problem I was like addressing in the chat hilariously me that we I was already there was something wrong with the stream because our mod was like is there something wrong with the stream that's fine it's fine everything is fine okay so back to this the idea here is like you know you can tune this time to live however you didn't you know whatever design you choose the ideas it has to be appropriate for the use case and in this case you know you could get a lot more you know interesting information you know from circle CI which kind of neat and you can actually tune it through here so that's actually beautiful we need to try that note to self oh I'm gonna try that later maybe we get together again no I know the next time hackery write some rules but I'd like to yeah let's get this thing tightened up a little keep going at it right exactly we'll get to that point yeah so what I do actually is besides turn on secret backend right so secret back end is GCP what this will do is set a default time to live for that secret beckoned you can also have a difference you can name the secret backend rosemary and it could be specific to the rosemary application the idea is that you know if you have multiple secret backends you can configure that the multiple secret backends can also be multiple GCP secret backends the reason why you might do this is that maybe you have different sets of credentials for different accounts yeah yeah right there you go exactly so the important thing is that maybe you want to minimize the blast radius you don't want to have credentials that give it to every single GCP project or every single GCP account that you have you only want to scope it to a very specific project very specific place so that's why you can configure multiple GCP secrets back-end so long as they have different credentials and you can configure them with different time to lives so that's where this is coming in and by the way the GCP credentials Angele you don't have access to these credentials right like you can't actually see them you'd like they're just like somewhere no no no they're not and yeah they're not I just have the repo so if you if you didn't publish them to a public repo and github then I don't see them yeah so that's the idea right if you're a ball operator and you're configuring this for a development team the development team doesn't really need to know the GCP credentials it's right that you're using to create these service accounts all they really need to do is authenticate a vault and that's a nice cute question question about the counts variable here so like what is this appended piece I know I have a feeling I know what it is but I just wanted to clarify yeah so you'll notice that I have these little count count meta arguments so I use a form of feature toggling that I call a feature toggling it's not really a toggle I mean it is a toggle okay this is a toggle it's a toggle because I do a lot of demos I also tend to push to production infrastructure not production but production for the sake of the demo right and and I don't really feel like going in and editing terraform to say like enable it disable it comment it out it just seemed to me it just seems kind of extraneous to comment this whole file out when really I could just toggle and say let me just create a variable this is a terraform variable called enable JP secrets engine and as per true feature application feature toggling right it defaults to false right I don't expect to run it and then when I'm ready I just toggle it to true and then it creates these two resources for me so you just taught me something that like I say now you yeah you convinced me I'm going to use this so again this is part of a CL though right so like you know giving that the the capabilities to be super flexible and yeah this is awesome so yeah thanks for that I like using it some people are like why would you ever do this why don't you just make the edits and then get revert and I'm like well I don't really have time for that and honestly the way it works is you don't want to be put polluting branches with like useless information I don't know you can do like you know you can prune that as well but yeah this is so much better and I agree yeah so there you go we just learned how to toggle set in feature flag sets in the in terraform yeah there you go you can use it you know be careful with it I just say be careful with it but I use it here because I didn't want to create the GCP secrets backend I wanted to walk through this config first before we actually created it but the second part of the GCP secret backend is not saying I need to reference GCP the second part of this is actually called roll set so what this does is that you'll notice that when you create GCP service accounts you attach a roll to it right you don't want the service account with admin for everything project admin for everything what the GCPD secret roll set describes is what kind of roll should this particular service account have in this case I am actually binding it to the project but I'm also saying that it should only have a container developer role so it's only got the responsibility to deploy to the kubernetes cluster and not do any cluster administration to it so can I ask you another question so this binding piece right you're binding yet this this description of the role right - something that lives in Google's cloud and then that's where you're getting let me just close this up for a sec that's where you're getting the call to your you're calling out to Google and that's added this data a bit it's here right yep so terraform or yeah it's here for me yes it has a provider that knows how to talk to the Google API and then you can literally making an API call here and saying give me this project ID right based on the credentials that the system has from terraform is that right yeah that's exactly right yeah okay cool yeah what's really cool about this is that you mentioned that this is called a provider in terraform right so there's a provider that calls out to different api's we have a Volta provider calls up to the vault API there gives a Google provider calls out to the Google API in itself right the secret back-end that you see here acts a lot like a provider right so you can think of it as in the vault space a secret back-end is a bit like the terraform provider world nice yeah yeah yeah and I just wanted to make that clear for folks as well as myself yeah yes this is why you have someone ask questions while you're on streams because you know what yeah and then this bit here right is also on Google cloud or is that a role that we're pulling from vault that is a role on Google cloud so it is indeed a static role that's predefined in Google cloud you can use a role that you've already defined with the tariff you know with the Google provider for terraform somewhere you know if it's a custom role in this case we're just using the GCP out-of-the-box roles container developer yeah and then I mean that's that's what they're there for for quick right you don't you already kind of know they're pre baked and that's cool all right I just wanted to be clear on that as well yeah so all of these are yes so that makes sense of so this binding bit is literally just saying hey you know attach yourself to or use this information and then you're using this data bit here to call out and get that project ID that's cool so try to automate as much as possible that's the two of them but basically once you set this up anytime someone reads from this secret back-end right anytime someone says GCP read give me a service account it issues and creates an entirely new service account and after this 10-minute interval the idea is that it voids the credentials so it says this is no longer valid you can't just log into this a revoke say it cleans it up so that's what the kind of the interaction is right and that's where you're calling this back-end piece here right and then it's it's basically course correlating to this or yeah so and then index is this like an index thing right it is an index unfortunately the downside of the feature toggle if you use it this way the downside of count is that there's an expectation that it is indexed so if I took out the count I wouldn't need this index but because I ride the countin it does require the index that is the know it's easy to dipose so yeah okay so okay right right and then right because the return where I got you I got you it it doesn't know how to map it exactly exactly I mean it's an object structure so you could just say give me everything so I could put a star in a wild-card it but I you know I only know there's never going to really be one of these so I just yeah all right that's always asking those like wait okay but there's got to be a reason and thank you you are you logged in to terraform cloud on this computer no but I can be okay so what we're gonna do is we're gonna toggle this on so it creates it all for us because I don't really feel like going into the vault CLI and then typing typing typing today let me log in real quick and you can just use my account so it's app dot terraformed is and this is another cool thing that you you know the company is put out so before you you would have to manage your state either through some form that I oh right yeah I got calm here so before I write if Fury is a user of terraform like I used to be when I actually had an infrastructure I was responsible for and yeah so before that I would have to write providers and then backends to store the state tear the Hachiko company has like literally given free access to their yeah era form cloud now which is awesome and yeah I I started using it when I think I got an email from someone long a while back and and I started you know beta testing it was really cool and I think you know this is the way to go or I save your state here and then you can also invite other people like rosemary had invited me to this just today or yesterday so that I could you know you show this ends when they were beta testing I got this really cool like Yeti mug yeah and they use it every day I got it like right before Kovac started that's great yeah it's so I don't know if they still giving them out but I don't know maybe there's still Yeti if I can ask yeah so basically the reason why this is easy it's just so it's easy State management I just at some point I just got get really tired of setting up GCSE storage buckets for this stuff so yeah yeah I mean if you click on that workspace you'll see here it's just a vault configuration and so if you just click you know that oh yeah I forgot I can click on this yeah you're alright so basically there's things that you can queue manually there are things that you would queue automatically as you can see here this was already pre queued so I'm not really I'm just gonna discard it there's no reason to but so it works similar to terraform open-source plan and run plan and apply I'm not going to go too deeply into it because that's not necessarily the point of this live stream okay you probably do a different one right yeah exactly we should probably do a different one on it but as you can see here the reason why Angela can't actually access any of the credentials is that they're alright only so even if he tried to like go in and edit them he couldn't edit them yeah they're alright only so all of these things you know I store I make sure they're not there and so what I'm actually going to do is I'm going to enable the toggle I think we called it enable what do we what did I call it jump back over free yes thank you enable GCP secrets engine let me copy that jump back over I don't think I actually have keyboard access to do that yeah so I'm going to enable the toggle this is the reason why I do it because again don't really feel like typing and so I'll just enable it just so it creates it oh yeah there you go yeah yeah so you're basically pre-baking the command run right in this yeah so what this does is that it just executes it for me so I don't have to sit here and cook up state and do all sorts of weird things it's just a little bit easier for me I just learned something new the other neat part about terraform cloud is that it has Sentinel so if you have a bunch of vault policies that you're not sure if they're good bad you know if maybe they're not compliant you can go ahead and actually tell them like hey yeah you know let me actually scan it and that's what Sentinel will doing this it would be policy yeah so Sentinel is that like a compliance tool like similar to like chefs inspect or something like that yes okay it's me it actually kind of looks at your terraform configuration it looks at vault configuration basically it's you it's a way for you to declare whether or not something is compliant or not compliant so right yeah very binary test yeah thanks yeah so here is just applying it but as you can see and part of the plan you can tell that it's actually going to create this GCP we're all set now so I'm sorry I'm keep touching the mouse I'm sorry okay nothing messed up that's fine yeah it went GCP roll set so now it's just going to create the GCP roll set it's going to create you know the secret back-end that you can see here so this is this credential etc notice it's suppressing all the sensitive value so you don't see it and now the apply has finished yay does that give you a yeah like a any elapsed time or something yes it well I think it does it just doesn't it just says yeah finished a few seconds ago but gotcha yeah yeah that's cool anyway nice beyond that uh let's go back to fault the vault UI oh that's just a tad oh wait I mean let me bring that back over I think yes yeah I pinned it for you oh thank you yes it's right next to it I'll pin them there you can just yeah okay so the important bit of this is that after we've run the terraform you can now see that there's actually a new back-end right it's this GCP secrets engine you'll also notice that I can't really do much with it you know there's nothing it's been configured right doesn't see six hundred eighteen hundred but that was configured using your ACL and right when we created it in the yes and the agent this is not the agent and when we created it it's created as part of AGL yes as part of this terraform configuration if you go to GC PDF as part of this repository there you go alright cool yep so you'll notice there's not much information in there doesn't expose credentials or anything right so you won't be able to see anything you also notice that kind of the key here is that I've actually already enabled this policy but anything under the GCP / key / nodejs circle CI I should be able to read I should be able to retrieve that information and some of the service accounts for example oh is that what you wanted yeah actually I was gonna go into the CLI and actually see if I can retrieve it from this one are you one of these this is this is a really cool thing I discovered the first time we or you showed me the first time you can run CLI commands through the the the UI against the the host vault system right so instead of you having to you know jump into terminal you can just do this exactly it's kind of nice you know if you have access to the UI you know this is a good way to kind of learn it so as you can see it's thinking as you can see there's a ton of private key data whoa yeah yeah it's only some of it so you're good exactly but you know what you'll see basically is that it's a service account it'll continue to you know read that so you'll be able to see that kind of basic thing every time you create every time you ask for it it will give it to you it will give you a new one basically you have a plus that'll that'll die anyway right so yes it will die in ten minutes yeah so you're talking yeah exactly like let's have at it you get ten minutes of compute and rosemary yeah I know all right so now that you've confirmed that there is actually indeed a key but you won't be able to see us because angel does not have access to my GCP project but what you'll actually see is that there is actually a service account that's created it's like got a key to it etc and then if I do another read it will actually create it alright so yes start hacking hey someone in the chat would like start hacking all right so let's go come on we gotta lay the foundation now I just destroy this vault instance basically I am no commitment to this vault instance all right so let's go to let's go to the vs code and let's actually configure our pipeline right to read the service account so we can deploy to Cabrillo yeah let's jump in there so we have yeah yeah I guess is this the job that you made cube config is a shoulder that's one initial that was an initial kind of like let's get started sort of thing yeah why don't we should we just change the name of it to do something we could say yeah we could say deploy to kubernetes well just do the whole workflow why not deploy to we'll just do the the shortened version of Kate's you know I got you I got you not yelled at but told by the CNCs whatever cloud FBI cloud native computing foundation computing there you go so they did a quick webinar with them and they were like I had a slide I always use lower case because that's just my style and they're like no it has to be a capital K you know branding you know that's fine it's it's legit but I'm gonna in my code it's gonna be lower key okay I'm a rebel now that way yeah in a pipeline you know when everything else is lower case then you standardize on lower case we're not gonna just create you know we're gonna cut the snake cake yeah I just had to share that little antigen obviously it stuck with me alright so what we're going to need to do is basically get the service account key out of vault and into here so we can either follow the same pattern we had before with Vall agent or we persist to work spaces or I don't know how we want to approach this so persisting their workspace I think we talked about that but that would expose you could then gain access to the key right what would what would you recommend then so just sticking it well you can't stick it in an environment very well it has to be a file yeah it does have to be a file yeah we'll just pull it out and then make a file and put it in the what is that they're protected I think if you stick it in what does it config it what kind of image is this is it at one two or Debian this is a Alpine image who Alpine I don't you yeah it's Alpine that that's an Alpine image and it's one gig well that's because there's vault in terraform which it's not even the light version of all tearful oh I see yeah yeah it's the full version of outlines like a slimline alright that's cool so yeah I mean I just pump it out wherever right and then cuz it's gonna it's it's ten minutes and gone right yeah well I mean really what we could probably do is just reissue vault agent again we just have to write it out to the file or something right so we have to tell okay some to do something you pick I'm new so I mean we already have the vault agent stuff we don't have to persist it we could just read you know reopen decade again I guess okay how's that works too okay yeah basically we will take whatever is above line 38 yeah you still have access so you can drive if you like hey I'll let you if anybody has any other ideas in the chat by the way we're you know you're more than welcome to to you know to actually put those ideas but just for the sake of uh you know what we'll just persist it because you know what we'll we'll give it a try good different pattern sure so we're using this persist to workspace command which basically I cannot I can't type yes I think this kind of worked yeah I can I can income it one comment that yeah yes so basically what this will do is take the file that we've created for said you know face on ball agent you know multi control so basically it takes this and it mounts that file and persists it across the workspace and I believe it's for the lifetime of the workspace correct yes yeah yeah these are these are temporary right so they are a right as the pipeline so the entire pipeline so this is a way for your different jobs to purchase data and have access to data that was maybe created in a previous step or right some other outside now outside but some other X I would say maybe they're they're running in parallel or something so if you want to share data workspaces are one way to do that temporarily built thanks alright so basically what we want to do is we mentioned before in this previous step here that we were using vault agent agent what it was doing was writing the secrets to a file in this case one of those files was vault docker hub and this stored the docker hub credentials specifically but there are other temp there are other files that vault agent could also be writing out so one of these right for example is let's say no js' circle see I mean some of these might be kubernetes so there's a lot of information that you could also write in addition to everything else right is there one that we need to uncomment and that file real quick yeah I think everything needs to be ok I can do that for you so that you don't need to struggle yeah cuz we did we uncomment this last week to get it going but move uncomment and save you're good to go thank you yeah so this is the reason why using of all agent is kind nice is that if you have a large number of Secrets you're trying to retrieve from you know from your vault you know whatever your vault path is it will template it out in the way that you're expecting to consume it so in this case I'm just writing them out writing the values out you know very simply and you know to each different file type right so a route terraform for example for tech connecting to terraform cloud for one example you might have kubernetes cluster zone temp kubernetes cluster Temkin organized cluster zone you know there's a whole variety of things basically that you would write out to these different files and you know one of them here is the service account key nice mm-hmm so basically we really only need anything probably under temp GCP project for now correct and any of these kubernetes one yeah yeah we're just gonna tell it right to the employee something yeah so let's just do all temp can I tell it all temp yeah yes yeah or last time what did we do and where did docker hub go so temp vault token maybe we'll just change all this to vault right yeah you could do that yeah cuz it saves it inside of that root the folder within the projects right so that's where it was the you can put it wherever you like that's basically yeah we'll just say mount everything involved which seems I don't know I feel like it needs to have a subdirectory somewhere can I do that I don't know it seems seems like if I persist the entire vault thing then if I check it out again it'll just override it or it doesn't matter hmm you can change this thing yeah change it so wherever you like temp temp it's fine yeah for consistency we'll just do bolt yeah well as long as you don't add it right it won't matter you know I mean yeah it's not it's not it's not it's not it's not watching for it right okay got it yeah you have to commit it right and add it a candidate yeah so these are just like whatever ignores them until the adage is mount external source with credential that is also true you could mount an external source with the credential as well so yeah you could use yeah you could use a cache for that but then right then those credentials are sitting in cash and yeah you know if they if you're using terraform and you have a ten minute time to live who cares but there's still 10 minutes of action yeah is there some anything in particular I mean do you know if you were using secrets right in you know in circle CI right would you go mount and like some kind of external source into something one of the builders is that a capability yeah you can do pretty much anything as far as that goes like but I would yeah anything with secrets definitely I would keep them locked up inside of an environment variable so one of the ways that I before our vaults or if I'm not using vaults I would just take a the GCP you know JSON whatever security key and base64 encode it into a value and then create an environment variable in the circle CIA dashboard with and then that in base64 encoded value would be living in there and then when you have a step where you need to you know configure the G cloud run CLI to do something and then you would just call that environment variable and decode it right so then and then pump that out into a file so it's similar what you're doing and it's protected you know inside a circle yeah the only difference is it's the static key and right you can't really change that whereas you can use vault here too to get a temporary token so yeah there's multiple ways to skin this cat that's why I wanted was interested in learning a little bit more about Bolton and trying to you know make this a very temporary value or token whatever so I basically decided let's just mount temp just for the sake of sure touching too much but what happened so how do I get the information from the persist workspace to the next job well good question so there's an attached attached to workspace okay and then yeah that's gonna say we could also look up just to be sure yeah so for those who are curious I'm going to drop this in the chat but there's an attached workspace tree there alright uh so I need to attach this somewhere in this case I'm just gonna attach it to temp because everything else is attached to temp right the other question that I have is that can I persist everything if I say it's a persist workspace does that mean it persists everything in that directory yes great alright I'm sure that this is the right command yeah unfortunately yes I believe everything in there is it's persisted yeah that might be better to star everything in there does that work that doesn't seem to like it no worries I don't know see I'm looking at the something else free and that's me nice cool well hopefully that works is there a way to verify this locally like if I have a good config or not so good config like there a way for me to ascertain if this is good or bad oh yeah you can do here let's see if you pull up if you pull up to the terminal which I think let me get up for you you can type I think I have yeah look at that anyway you can clear that out and you can you can use reflective inside of you okay you can use we have a CLI circle CI and then I think it's validate let me make sure yeah so so yeah so we have a circle see I CLI tool and you can do the did it to do or is it it's a good diagnostic no do you see it I think yeah yeah yeah yeah okay so right so I don't obviously do this at all okay alright so yeah so you can do validate there you go so if you wanted to do validate it is valid okay okay so what that does is Lin seer yeah Moe and yeah checks against those keys and make sure that you know nice so you actually remembered the attached workspace right no no I didn't I looked it up and then pasted it I said of course no no let's be honest the whole point of this show that all of us are copy and pasting from somewhere else oh yeah I've been doing this a long time yeah it never it didn't used to be that way by the way the internet changed things yeah I used to have to actually think it out or actually you know bug like a mentor or something like and then yeah they would get annoyed and yeah you know that now the internet you know that's pretty cool yeah it's still pretty good I mean you know yeah all right I'm sure do a cube CTL version just to check if this pipeline runs correctly because in theory if all of these are retrieved right so what we're doing is we wrote it out to file we mounted the files we're attaching it at the workspace now we're reading from the files so as you can see this is using g-cloud CLI the CLI is looking for the temp file it's looking for the tcp project file kubernetes file etc and if all of this works we should get a cube CTL version at the end of it that actually shows us the server version I guess we could do cluster info but I feel like cluster info might be dumping things that I don't really want on the screen yeah yeah let's not do that I got some smart people watching I'm sure I know yeah don't mess with us a little we're not gonna try to we're not gonna try to Tempest I I love the community but they are a set of pranksters for sure yeah all right so we're gonna see if this works shall we just uh oh we don't need to authenticate a vault I don't know why we kept this I can't just do this all right so you persist it up top right yeah they persisted up here yeah is that gonna work though that that star with persist let me take a look real quick I'm not sure if that's gonna fly let me see your did I mean we could be really specific I could be really specific about which ones I persist so I know which ones that I need so we could actually ya know so you could do no that's fine I think I think let's try it yeah you can do that I think you can do yeah you can do that so you can do yeah yeah you're fine you're good you're good let's try that yes temp forward slash star you could do I think okay or is that no way cuz you're mounting temp yep let's try the the star what the hell all right can you push because I did not know I can't see thee yeah I can do that you know jump over to this and then here we go oh no I didn't push it yet 200 you want me to push sorry no no we we it won't even run right because we didn't even put it in the workflow good can we put it after the vault off sure okay oh wait so you want to do we can do like requires in the workflow I think that'll be there pretty cool yeah let me do that so that we have a sequential thing right yeah exactly cuz you have to build the image first and then yeah so that what that looks like is basically here let me grab so we can just do this well leave those in parallel since we know the tests are gonna pass but we can do rayray okay so it's um what's this called again deployed eggs or something yeah there you go right there you go and then and then we have to say here so we're gonna do right decoy and then requires is it a - I forget you know it's a straight up all right here you go requires there you go and then from here we're gonna say vaults off there you go and I believe that's all we need I'll save it and then we can push it up yes and we'll get ready shot all right great so right so we made some changes let's jump over here we'll just use the I normally do this CLI wise but so we make changes here right we need to push those yeah yeah cuz we uncommented right okay so we have uncommon uncommented that the changes in circle or are basically these things actually I can see something already wrong with it okay go back to the agent HCl I think we moved everything under pipeline right so we would isolate pipeline versus kubernetes versus like application side secrets right so you wouldn't so like that's that's important for 38 for example line 38 yeah there we go so did that is that right I don't think that's right okay can you go to volt actually let's take a look your pins over there and oh there we go yes my pin okay mm-hmm so let's actually take a look at the secrets in which path etc just make sure we have it in the right path right because this was no js' pipeline and then kubernetes so there was no kubernetes as far as I can tell there was no kubernetes path so I think it is not quite right no all right so let's go back to the s code all right so this this does not seem quite right so pipeline pipeline slash kubernetes let's actually see is it in here cluster project sown okay so let's go back so we have project zone we have cluster so that's good okay I think that's mostly fixed then all right yeah yeah I think that's the right path all right that's Commission this vault doctor who can stay right her yeah well doctor who can stay we still need it all right so we're gonna jump oh yeah we're gonna commit is that right yes all right cool yes so I'm gonna use the GUI version and then we're gonna say awesome fingers giggles all right so we're gonna do the push and once that's up in the branch you can jump over too and just go back to the bolts hey it started running all right let's see what its gonna do it's a repo stuff right yeah so basically we hopefully these two are cool but you can see here I added requires right so that's how you add a dependency so like you know I mean if you don't want to run things in parallel then right you would do normally this run test would also be over here and it would be its sequence right and this is a way so parallelism is a way just like concurrency right to speed things up so if you have like this is really good for if you're building multiple versions I can node and you want to test your application and different versions of node you know you could run what we call a matrix job but what that does is runs all of that stuff in parallel at the same time to speed things up look the other day see ICD is supposed to be fast right you want to get the feedback loop and know what's going on with your app as quickly as possible yeah so let's jump in here and see I don't think it's oh yeah it's doing it is it yeah look at all those secrets that you're not seeing yeah it's just it's just rendering all of them out there you're not seeing them but they're in the files yeah yeah you exactly so right so yes so this is throwing up the image into docker hub and hopefully will soon jump back over to build test which is the workflow and yeah this this should either it's kicked off sometimes the UI takes a minute to sync up let's see or not it's thinking it's thinking I think it was the persistent workspace there you go we got it so now this is the the gold here the one thing though I don't have access to here your what do you call it the GCP cloud yeah to show that oh yeah sure we can show it all I'll just share my all attempt to share my screen if if this work okay yeah yeah at least you'll you'll be able to see what's going on in my GCP yeah by hand since I don't have your creds and then I want no and we did not grant you okay all right an account is required when using p12 keys why does it think it's a p12 key alright let's jump back to the yeah do you need the vault you I know the value I should be okay I'm just wondering why it thinks that did a g-cloud off service account isn't quite right okay so I'll let you drive you tell you okay well let's go actually search for g-cloud off activate service account and see what the heck look in here yeah it's I think we're missing a email address I think that is probably part of the problem well yeah because the g-cloud off activate service account requires an email address in the middle of it oh you're right so you need to go back to the code I think we probably need to go back to hmm-hmm actually let's look at some I do have actually access to this so let me just do this let's actually do this probably like the better the more a better way so to speak um so the vault provider for terror forum let's actually see it the GCP roll set outputs a an email and that way we can store that as well all right so let's see we just need to get the email cheese peds peaches peaches be secret back-end secret we'll set so let's actually take a look and let's see if we can extract the email account associated with it so service account email nice yeah so what we're doing yes yes if you could go back to vs code yep yes ctrl F for search I think we're gonna have to control that first search for word what do you oh no no it's fine it's just it's a very dense page at times all right so what we might do is actually put this into somewhere in 2kv somewhere yes so what we're going to do is just pop this into kV somewhere you can possibly retrieve it for a vault but if it's necessary for this pipeline and we've already committed to this pattern we might as well you know thank you continue with along this route so do you need to copy and paste something yeah well actually so far it's ok so we'll just say it's kubernetes we'll just say this is email Oh pipeline kubernetes sorry so that is pipeline kubernetes right and the data JSON we'll call it email actually why do when I put this up here because yeah exactly I don't need this actually I could just do this I can't so it's a data jail yeah so all of these are gonna be keys and values and that you don't have to worry about it and hello is that was they on their house I think that's Jose in the chat I can't actually tell people this are you on the chat - you are on the ok yeah there we go ok so let's actually do this roll set here right so GCP roll set dot roll set mm-hmm this is using a templating like so the JSON it's Hara doc temple ink template ting so you actually do have to interpolate within here so the roll set dot what was the the parameter service account email was that what it was yeah yeah service account underscore yeah all that good stuff yeah oh I just did not copy that correctly great go back - yes go please there you got it we had you know we have to get this live share thing working correctly anyway so we've got the email now in theory this should run so let's actually commit again oh I can get to it okay yeah git status fault terraform I could do it from here you get that as well yeah this will run our pipeline again though right so that's kind of not so great oh wait fault yeah it's okay I mean I got minutes you got minutes well we can add that as one commit and then the second commit we can own oh then it's like this whole chicken or egg issue right okay yeah thank you okay yeah good get it actually if I add see I ignore does that mean like that gets ignored and circle CI but then pushed in no if you do that with like you get ignored on that file sorry see I skip oh yeah I wonder if terraform cloud will run this still it probably does cuz I don't believe this see I skip is the trick is a trigger in which case we might be able to hack it so see I skip basically ducked it basically stops a CI from running the pipeline oh okay cool yeah so we're going to see if terraform cloud will still run it they may or may not run it I jump over to yes our alright so in theory circle CI picks us up and doesn't in theory should not be running this there you go cross fingers that was the original yeah did it pick up there we go yeah it didn't run right so meanwhile this should be running in terraform come out okay yeah Alec it didn't like something instance key oh oh it has count okay yeah let's go back to vs code this is what happens when you have toggles everybody so maybe I won't use them as kidding hey uh you know let's just see did this thing right Oh yours in KB this one right yeah okay so roll set zero zero and I think this GCP similarly right GCP not zero so just so we have that it add it come in it skip because we apparently apparently this now works now that I know this yeah well this is a nice yeah kind of flag so like what you're doing if you're doing especially integrations right and you don't want to consume your your CI bills but you need to pretend like it ran in a in the system yeah exactly all right so we're gonna go back ahead you can jump over it so let's see if this picks it up yeah all right so there we go it is planning and by the way what would Rosemary's doing here is which I learned the hard way was you're you're actually running these builds inside of the terraform cloud right so you know you're not running in locally when I first started using this yeah outside of like normal usage just trying to run things locally but it was running in the cloud and was like whoa what's going on and they don't we found out that and I was like it couldn't find a binary or something right it was a little tick box in this in that in that UI that says run local and then that when that happened boom it things started happening right okay that's available now which is good all right can you go back to be his code really someone needs to like fix this whole thing where I can actually scroll yeah well I think if I gave you access to vias code you'd be able to do all this right it doesn't matter yeah okay so what we were doing was now I need the other let's just create another file for this why am I in putting them in individual files I mean we could render them as environment variables I mean there are a couple different ways you can do this they're just it's just all whatever you decide hosted it was or popipo and I was saying coincidence I was just watching video oh yeah so yeah that's a talk I get I gave it a few places actually so that's cool Jose nice to meet you by the way yeah there are a number of I think there are a number of talks out there about you know secure a developer workflows and such but he's talking about so I gave a talk about like shiny object syndrome and how to avoid it but it's object syndrome yeah it's more of a culture talk okay yeah what was the shiny object in question for me yes I tell story about how I implemented MongoDB and like this is a while back like a couple years ten years ago and yeah it didn't end well I won't ruin this story spoiler alert so wait you have to post a link to it though you should go to the okay yeah since we're talking about it I could do that that's all right I'm on your computer hold on okay you know I got my laptop here you know I'll another link and then you can you push you push make sure that looks okay and then I'll let me post the link last one I gave was I mean there's no PHP UK 2020 just look for that Rivera seven hours ago was that lead dev of the dev I didn't want to leave that recently - yeah Oh see someone posted it as well thank you oh nice so alright I need to push this alright let's go I'll just do it the old-fashioned way I was going I was going all CLI cuz I don't actually know how to proper yeah it's all good it's all good I'm the you I were on the IDE I don't I just I can't I can't with this yeah I'm just trying to show everyone how to use it I hate I'm like you but I don't like to use this the terminal inside this thing it's it's lame I jumped between the what is it whatever that's called the I term 2 or whatever yeah so it looks like we got an attempt to running let's see this will work yeah so yeah we should hopefully have something here soon yeah all right shiny objects I need to watch this now yeah yeah that's it it's it's I've been giving that talk dude I got a lot of mileage out of it I enjoy giving it because it tells a story you know basically failure and those are fun Wow this where ever you almost 20 minutes out I know we might be cutting it down to the wire again yeah is there a part three in this or what no I don't know so next week there may be a maybe a special audition I don't know it may be a special edition we're trying to figure it out so well you know I'm always game to hang out with you I know I well you know mid-may it's like yes well we'll figure it probably yeah I know whenever yeah yeah I think it can't authenticate yeah oh no yep yeah dang it did it you're getting some bad I mean is that you and me to look at the logger yeah no no I think it just means that we timed out I should have just expanded yeah Thunder I don't know why I didn't do that I should have do it now right like I could just you know hold on a second top part you know part seven this keeps basically we timed out again right so I should have set the secret to like ad infinitum but it didn't quite work that way hey so you gotta like what we're doing thank you we're trying to make it useful and and realistic I mean at least yeah that's obvious you know we're not we're a reality TV show rosemary I know well this is a reality TV show I should Oh vote vote within the audience but like I am I've been seriously reconsidering renaming this stream or this session on Fridays to don't push on Fridays because I feel like it just constantly this is just the theme that none of us really correctly configure anything on a Friday so well yeah I mean I used to have a roll-back like years ago like never deploy anything on Monday or Friday but that wasn't mainly because I didn't want to be stuck in the office right those days all right let's rerun this and see what happens all right it just needs a rerun right yeah yeah just rerun yeah I got to go out I'm sorry about that no wait okay cool so yeah let's jump in I need to get the API going to do these things for demos I know like run the script to do this all right cool so what's my everything now by the way you'll notice everything that I've done is in terraform because I got tired of repeating and keep keep react sensing and keep doing this I should have just set this up somebody said just crash everything up Friday yeah I have thought that that has crossed my mind I'm done yeah I know right you think but something just keeps stopping it I'm happening yeah like you need money you need a job this is doing it looks good all right well it looks like next time I'm just not gonna bother with the secret ID to set to time to live it's clear it's clear that I should just set it to a really really arbitrary law infinite settings so that it doesn't time out and and revoke it but that's proof that it is working yeah yeah for sure no that's the point right like and then of course we already kind of caveated it with like you need to figure out what you know you know how you're gonna operate in from there like like I said earlier in the in the previous session that we did you know security is not easy this is super complex I mean you know and it's pretty cool I actually did a talk at the did you give a talk at the all all the talks from sneak no no I didn't so I gave I was sitting on the panel screen you still in the security panel and yeah we were talking about so they were gassing questions about security and you know how it swung left recently and you know kind of they were asking questions about all the stages between like you know developer through operations and then maybe security and yeah I was talking about like look when I was developing code we didn't give a about security to be fair like it was not even something I thought about but it's changed so much now because there's more tools available right there's a lot of more automation and it's understandable before you would like literally sit and look at just you know like a spreadsheet and manually test things so yeah it's it's advanced so much and and this is not easy a matter of fact in in oh we got our failure here authenticate oh kubernetes service kubernetes email kubernetes email email what that explains the problem yeah I did the cat kubernetes service and so you can every Nettie's email yeah sorry Friday bringing today really friend you have them wrong email do need a jump anywhere yeah I go back to the vs code yep okay yeah I accidentally you can drive you got it click well click you bring a tease email there we are oh yeah I named it not correctly oh yeah it's all good so you're a way better typists and I am I would have been fumbling through that now I'm just like used to this at this point right but also I was I was I have the you know for better for worse I had the the teacher who sat there and we had like these paper keyboards oh yeah like even though we didn't necessarily have the computer we have the paper keyboard and so you would have to like practice on this favorite keep child abuse to get you in the thought process for like typing I don't know why I was like you know I got time I didn't really care but yeah I feel you on that but I I actually when I was younger I was in high school you know and by the way I was typing typewriters and yeah it was I didn't I didn't join typing to be learning how to learn how to type that way in high school in the 90s early nineties so different hold that you know I mean there was really no computers in my high school yeah I don't think really the high schools at least the school I went to didn't have them so yeah yeah and it's not even today it's not common to find you know a full fledged computer lab and every and every high school or every school either right so I mean it as part of this I think was like it was the coming of of like a an age where I guess the school district thought that there was a reason for kids to learn it and they didn't have the computers but they decided we're gonna print out paper keyboards and have them pretend to type from the paper keywords so hey but look you're you're skilled yeah you know I again a lot of practice yeah yeah I remember okay so we're gonna shop and developers are like I could type they never say here we go good okay so okay let's see this top this out real quick and see what happens when it like does this like obscure there you go oh okay that's fine morning air hi OpenSSL not available yeah you have Python installed oh you know but this is the G cloud this is G clouds uh internal there this is their image so got you yeah fascinating have you seen this before no but I feel like we have a JSON private key that's what I don't understand we definitely have a JSON private key oh we could do this um here oh we're gonna do the dangerous thing oh I won't show anything right just yeah are you worried about me no no no it's fine it's fine why not well I won't do that SSH into it yeah and then you can we can you know debug yeah yeah somebody asks how old are you guys we're gonna infer that answer I'm I'll give you a clue I'm definitely in my 40s I [Music] don't mind sharing that all right is this thing like come on all right it's gonna do all that good stuff download that beefy image you guys know I know yeah there we go let me let me let me put this in you want to use the terminal or you want to use well I will give you a terminal yeah near the terminal yeah all right there you go gives you some access Sega oh that's not what I wanted hey we eat no sorry what did I oh okay let's try this again then there you go it's all yours hey all right so so what we just did what I just did was in in circle CI if you have a problem with a specific part of a build you can SSH in as a developer right to the to the resource where it failed in the environment and failed so you can debug live without having to bother anybody so it's kind of like self-serve so yeah I used to use Jenkins and couldn't do this then why does it say g-cloud does not exist mm-hmm try to try to do local hmm okay let me jump off so we can do Hey yeah it's so weird so we can do which just in case no so I don't think it's there aha see probably why or at least it's not finding it so that's a quick find and what's the name of it G cloud yeah G cloud there you go okay so it's in it is there maybe it's not that's what yeah let's see huh serve it activate activate service account sorry for those who are wondering why I'm peering around it because my mic is in the way I just need to move this mic somehow you don't have one of those fancy arms like me I do have the fancy arm but the fancy arm is like in the way which doesn't rhyme anything I'm like why are you like this fancy arm maybe this is not right can't open cube root Oh temp let's see so apparently this was the problem but we are not using a p12 key I believe it as a JSON yeah there you go but that's the thing it should be a JSON private key what oh if you were using to temp service account that JSON because it is a private key cannot read a JSON file so apparently something is a little funky hey can you do the G cloud component maybe they install some no well you know what we're gonna just dump we're gonna dump out the contents of the file alright because like I said we're reissuing it anyway all the time yeah alright so let's see what's in this oh it's base64 decoded AHA is that is that something we did no I think vault that that must be part of the secret engine so you have to decode it yeah so let's go back yes code hey we're gonna get this I know it oh so it does the same thing we do yeah nice very yeah all right so let's just base64 decode it finally we're aged we gave up just I'll just do this uh you know there's there's a bit you know what I did I missed a crucial step and by the way everyone this this repo and this branch will be available so like oh we'll figure out a way that I'll keep this up I won't bother like updating this I think it's this thing this thing okay so vault agent uses this thing called vault sorry called a console template under the hood so if you want there's like a set of functions that come out of it that are actually kind of neat so one of the built-in functions is actually decode you can just say let me just base64 decode this so yeah that way you don't have to actually you know sit there and is that is that like component built in to terraform um it isn't yeah so what's interesting is that it's vault agent is now built into vault or you know kind of like the the code itself is now bundled as part of vault right but the decode can these functional commands all of this kind of is based on a tool called console template which is not baked into parts of the codes our code is based is basically baked into the vaulty right well the reason why I ask is because you know the the different operating systems have different encoding writing capabilities or at least the tools yeah they're a little bit different then you know sometimes you get into these weird but if I would assume they would inject or at least come compose it yeah I think include should we shut down the SSH otherwise uh yeah could do that I can just stop it yeah so by the way if you want to you know ever need to use SSH it will timeout after like two hours once you're in it or you can just cancel the job yeah and if you don't use it it will stay up for what is it ten minutes I believe yeah so yeah you have about a two-hour window it's a play and in the ten minute window nice if it's not active yeah we don't want to leave people aging there yeah and well while that's running what I can show ya I can actually show the I mean how do I stop remote control oh there we go um what I can show is I can show GCP so what it actually looks like so let me stop remote control which will let me share my screen which will stop sharing your screen for a second yeah it was um which one is it I think it's this one sure okay so what you'll see sorry everybody on Twitch but what you'll see actually is I am here and if I actually refresh this it'll show everything but in my project what you'll actually see is that vault every time you read from vault it will actually reissue a new key so as you can see here this is actually the service account so if I go to the service account right um you'll notice here the nodejs service account here they have this key that has been reissued so this is since the last time we run it but okay as it continues to reissue you'll see a new one come in but this is an old one so you'll see that that was previous so I'm gonna stop share for a second any angel you can share again yeah it looks okay share my screen desktop one how's that everybody can see that yeah I can see it and it pests or at least it authenticated so if used if you drop down the authenticate to GCP you'll see that now there's a kubernetes version and the server version is now pointing to a cheeky gke cluster did it it didn't deploy anything though right no it didn't deploy anything so now what you can do is a cube CTL apply if you've got you know if you've got a kubernetes manifest or something there's I don't there is some terraform kubernetes but it does require some state and I don't know where we're going to put two said state yeah this is this is actually cool I think you know maybe we do a part and deploy to kubernetes I think that was our three things is docker hub GCP credentials and then maybe some kubernetes action yeah I don't know yeah if you had a database that you could connect your application to we can also do kubernetes vaults you know secrets injection as well so we can we can continue to get fancier so all of this pipeline stuff that we talked about you know you want your pipeline secrets away from the secrets that your application is really yeah right so sure you know this is a good way doesn't kind of separate the responsibility of it so let's ask the folks in the twitch are you interested in a part three dance i'm kubernetes database fun stuff if you are you know message on twitch let us know message you know through twitter etc yeah so you've got some people who are interested so let's actually do that I think this is a pretty good stopping point basically we got the authentication rotated we were able to authenticate to kubernetes there are other ways you can authenticate to Google and access the kubernetes cluster but this was just a good way to demonstrate the GCP secrets engine and vault and we had some interesting pitfalls along the way yes finally week we figured out it was basics t4 coded well but you know I didn't even know that that was an issue with that so like I would have been wracking my brain so yeah I know thanks for I really enjoyed just you know this stuff crashing on us and and because I believe you know you know the system really well and you know the pitfalls but until you encounter them I can ask like I asked about Circle C I stuff what it what makes sense to use what doesn't make sense yeah in the circle ci-flow and it's kind of cool because it's like how much how much can we not give access to people and they can you know you can still do you can still work on the system still deploy still be changing code so cool all right so I guess it's cutting it's time for a bureau clock right my time here some who are waking up you know thank you for joining us thank you for staying on for so long and hopefully we'll get part three up keep an eye out for a recording of this and if you know you want to play back certain sections for reference and we'll see you next time I appreciate everybody showing up I mean this is cool and yeah I think we should do a part three I mean if you want they're awesome okay thank you everybody she have a good weekend yeah have a great weekend

Channel: HashiCorp

Views: 677

Keywords: HashiCorp

Id: 4wa-25Sy0w8

Length: 119min 39sec (7179 seconds)

Published: Tue Apr 21 2020