Using Pulumi and Jenkins to Manage Your Infrastructure As Code

using pollumi and jenkins to manage your infrastructure as code [Music] as an application developer that also has to manage your own infrastructure you may be used to doing click ops you know what i mean by click ops right you log into a console you point and click and save and pray that your infrastructure works as expected you may also be aware of declarative ways to define your infrastructure as code such as using terraform or cloud formation however you just don't have the time to learn these new declarative ways to you to define your infrastructure the other thing is you're also really tired of following the click ops methodology of pointing and clicking and hoping that your infrastructure works as you expect when you're using click ops that can be very error prone and can be very complex to audit this is where pollumi can help out pollumi gives you the ability to define your infrastructure as code using languages such as typescript python and many others in this video we're going to look at how you can integrate polumi with jenkins to more easily manage your infrastructure as code here's today's starting point i have a jenkins lts controller it's version 2.303.2 attached to this controller i have an agent and on that agent i have the paloomi binary and i also have node.js 17. we're also using a sample repository in this video the link to that repository is down in the description i've already cloned the repository locally so let's take a look at what's in there to begin with so what we have are just a handful of jenkins files we're just going to start with jenkins file one we'll take a look at the others as we move through the video so what we're going to do first is we're going to verify the tooling that exists on our agent and we're going to check the version of node npm paloomi and we're also going to print out our environment variables so let's go and create a job here we go inside our controller so we're going to say new item and we're going to call this jenkins-example dash pollumi and it's going to be pipeline i'm going to click ok and i need to go pull it up so let's go take a look at that repository so that's jenkins example plumi and what we have here is this go back over here scroll down click on that there and here the branch that we're using is main and the jenkins file we're going to use first is jenkson's file dash one let's click on save and click on build now we take a look at the output what we should see is we have node 17 we have npm81 we have pollumi316 and then the output of our environment variables that we'll be using a little bit later on so now that we have verified that the tooling that we're going to need node.js and the pollumi binary exist on our agent now let's go and create the code that we need in order to provision the infrastructure that we're going to want to create so if we take a look at the pollumi documentation i have chosen today to use typescript you could have used javascript typescript python a few flavors of.net and also golang again i'm using typescript because my example is really short and this was the simplest thing for me to use to be able to explain it to you you can choose to do whatever you want as long as it's within that language set so following this example we can see here that we need to make dur my project and we're just creating a project and then we're going to say paloomy new typescript and that's going to create all the artifacts that we need in order to get a project bootstrapped so what i'm going to do is locally so i'm doing all of this locally at this point in my shell i'm going to create my project just inside of a directory now it says github here but this is just a plain directory at this point and then what i'm going to run is plummy new typescript whoops come on all right and then what we have here is we're going to see that oh we actually need to log in before we can create this let me size this up here just a little bit it's telling us that before you can even create the new you first have to log in well let's say that i don't want to log in what are my options well here let's do a control c and let's go back over to our browser and let's go ahead and log in to paloomi i'm going to click on get started actually i'm going to open a new tab there we go and click on sign in which i guess was above i could have scrolled up but that's okay i've already set up my account so i'm going to use github and i'm going to go ahead and zoom in as well and we can see right now the dashboard is empty i have no projects no stacks and then i have my settings well instead of going through the login process what i can do is create an access token and this is what we're going to do so i'm going to create a token i'm going to name the token jenkins click on create token i'm going to copy this and first off save it off to the side so i don't forget it then what we're going to do is we're going to go into our controller and create a credential specifically a secret text credential for this token so we'll say secret text there's the value and the value i'm going to use is polumi access token whoops and that will also be the description so that's set up okay now let's go back over to visual studio code which was right here the other thing that i need to do is i need to export an environment variable so that the pollumi binary knows what the access token is so i'm just going to say export polumi underscore access underscore token equals the token that i just created okay so i've done that now that i've done that let's go ahead and run the paloomy new type script one more time so now it's logging in using the access token from bloomy access token that's the reason why i named it the way i did now for the project i'm going to name it jenkins example whoops pollumi there we go the project description is going to be my lenode project what we're going to be doing is we're going to be creating a object bucket inside of linux that's the purpose of this project you could make up whatever you want this is what mine is the stack name think of stack roughly as an environment so in my case if i had dev stage and prod i would have stacks for dev stage and prod in my case i'm going to name mine main and you'll see why in a few moments so we're creating the stack main we're installing the dependencies so it's doing whatever javascript stuff that has to be done under the hood to get the project ready now the project is all ready to go and if i want to do an initial deployment i could run pollumi up now i'm not going to do that just yet as we went through this process to do the pollumi new remember i was doing this inside of my project but that's not actually inside of the git repository that i want so what i want to do is go ahead and move over all of the files that i just created inside of my project and put it into jenkins example pollumi so i have one line there which brings over the files we'll take a look at those in just a moment and then there's also one hidden file the git ignore file that is now there as well so if we were to come back over here and go to jenkins example pollumi now i'm back where i want to be let's go ahead and slide that down we have a paloomyamal that has the name the runtime nodejs and the description so we entered in name and description but when we said paloomi new typescript that created the runtime for us for node.js since it is a node.js project we have a package.json and we also have a package lock json just goes along with the territory we have a ts config json which covers all of the specific typescript configurations then finally other than the git ignore we have an index.ts all of the work that we're going to be doing is going to be inside of the index.tx at this point our index.ts is effectively empty it's just importing polumi but there's nothing going on so what we want to do now is go ahead and commit these changes and set up our job to actually run our pollumi up so let's go back over here and commit all of these changes so we have ts config.json pollumi ammo package.json package lock json index ts and get ignore and we'll say initial commit which if you look at this repository it's nowhere near the initial commit but that's okay let's go ahead and sync our changes so now everything is pushed up and in fact we can verify that if we go back over to our repository and we were to refresh here now we can see our git ignorer and all the other files that we just committed and pushed so one more time before we run the job let's go back over to our console look at the projects tab and we can see that the jenkins example pollumi has already been created because when we said palumi new typescript this project was created because we had to set up the access token you know rewind back just for a few moments think about all the things that we just did to get to this point and if we take a look at main what we have is oh there's no updates i'm needing to run a pollumi up but notice these commands i have a paloomi stack select we'll see this in just a moment and we have pollumi up let's go back over to our vs code and take a look at our jenkinsfile2 now if we take a look at the top and let me get rid of the terminal for now if we take a look at the top of the jenkins file 2 we have a shared library the link to the shared library is also down in the description my controller already has the shared library defined we'll look and see where that's being used in just a moment we're setting up an environment variable this is our paloomi access token that we set up on our command line and we're pulling it in from credentials remember that we just created the paloomi access token secret text credential so that's going to be loaded into here we still have our verify tools we just left those in place we did take the environment block off of there and now in our up stage i'm setting a local environment variable to this stage called pollumi stack and i'm calling get polumi stack get paloomy stack is a step inside of our shared library now what does it do so let's go take a look at that step and what we have here is get palomi stack and what we're doing is we're taking two environment variables the get url and the get branch and if you were to take a look at the output forget url it would be https colon slash and then the full url and the branch is origin slash in our case main then we want to get just the repository name because when we split it out from url it has a dot get on it so we're trying to split it before the dot get so therefore what we're building is a stack name that matches and is derived from our get repository again our git repository is jenkins example polumi part of the darin pope organization and we're working with the main branch so if we were to take a look at this example here this maps directly back to the main branch on the jenkins example pollumi repository inside of the darin pope organization this is an opinionation that i created that way i don't have to keep up with names i just define based on branch name what i want my stack to be that's why i picked main as my branch when i set up pollumi new so let's go back into our controller let's make a change to the job and change configure to jenkinsfile 2. and let's watch what happens we'll click on save and build now as it starts up what we should see here is the clone of the shared library that comes in no big deal we see the output from our verifying of tools we need to run an npm install because the agent is unaware of all the packages that are necessary in order to run so we first have to run an npm install we did that found no problems then we set our paloomy stack select it to darren pope jenkins example pollumi main this value here is the string that is created when we call get pollumi stack which is back over here this return value so let's go back to our controller then we run pollumi up and we give it a dash dash yes that's effectively saying just go ahead don't make it interactive just go so we're saying pollumi up dash dash yes so what happens and it gives us links that we can go live and take a look at them it's creating the stack which had already the stack had already existed because we did pollumi new and we created one new resource okay well what does that mean let's go back over here and let's refresh this page and when we refresh it what we're going to see is that we see the initial commit it gives us the commit hash and if we were to take a look at the details well there's really nothing here there's a timeline there's a configuration which doesn't exist in an environment which says hudson cli and then all the other information about our commit now what we want to do is we want to go ahead and set up our code to create a bucket within lynode so what we can see here from our documentation from paloomi registry is that there is a linux package that's available to us and there is an example for creating an object storage bucket and we're going to be using the typescript example and we're going to just be copying this over now i already have a cleaned up version of this that i'm going to bring in because i had to set a couple of specific items and i'll call those out here real quick so let's go back over to our index ts and i'm going to replace that because it comes in again so what we're doing is we're getting a handle to an object storage item in us east one which is a region within linode and then once i have primary which is the value of where that's going to exist then i create a new object storage bucket with the name of this and then i'm passing into a cluster of primary which gives me the primary id off primary so i'm passing that into cluster and then the label is going to be the same value as my name i just chose to make it the same both cluster and label are required parameters if we were to take a look at the documentation what we're going to see when we take a look at inputs you're going to see cluster and label are required fields there are other fields that you could set if you wanted to but for simplicity sake i'm just picking cluster and label so let's go back over to vs code you'll also notice here that it's complaining cannot find the module for polumi lenode okay well that's actually fairly simple we need to set up our import for this just to make sure that it's going to be clean when we push it up so the command that i need to run for that is npm install plumi lenode so now that i have a terminal here let's go ahead and paste that in so i have npm install pollumi linode hit enter it does whatever it's going to do and when we take a look here plummy linode is all cleaned up in fact our whole index ts looks clean let's take a quick look here at our get changes let me go ahead and save this because it hasn't been saved yet so we have three changes going in i have index ts which now has all of our code specifically for creating the bucket inside of lynode and then also we have the changes adding in the plumi linode import for both package json and package lock json so all of that is now in place and ready to go so let's go ahead and say added lynode okay and we need to do that we'll do that and we're going to sync those changes up now that we've done that let's go back over to our controller and we're going to run this one one more time so we're going to say build now and what we're going to see here as it runs is we're saying okay yep we want to go back to that same stack yes and we're trying to run this but it fails and why is that well if you think about it we never created any kind of credential or access token to be able to interact with linode so now what we need to do is go and create a token in lynnode and define that over in our controller so how do we know how to handle the token from linode let's go back over to the object storage and let's click on overview and it talks about oh we can provision any of the resources available in linux that's great here are the libraries that are available take a look at installation and configuration we're looking for things uh configuring credentials what we need is an environment variable named linode underscore token now we have two choices we could either set the environment variable which is what we're going to do inside of our jenkins file or you could say paloomi config set linode token and that would push it up to polumi and they would then have the credential there so since we know that we need linode token let's go back over to our vs code and take a look at jenkins file 3. so we're adding in a new linux token value we're going to create a credential called leno token which is also going to be a secret text and then that should be the only thing that we have to do so let's go over to linode and i've got one here let's go ahead and log in and let's log in here so the way to create a token on lynode zoom in here and go up to my name api tokens i'm going to create a personal access token i'm going to call this jenkins and since this credential only needs access to create buckets i'm going to create a very specific bucket in fact i'm going to set it to expire in one month and then the object storage has access to read and write that's the only token i'm going to create so if i click on create token i'm going to copy this personal access token okay got that let's go back over to our controller let's create our new credential i'm going to name it secret text the secret's going to go in there first make sure i don't forget that okay and the id is going to be lenode token because it's the id is going to be read into let's go back up here that's the identifier so that's what we want to name it and that's going to be loaded into the environment variable linode underscore token so we have that let's click on ok and then finally what we want to do is we want to make a change to our job and instead of calling jenkinsfile2 we want to call chicken's file 3. click save and now build now so as this one runs what we're going to see is our pollumi up we're doing linode so we have a plus here so we can see one to create then we can see that the object storage bucket is being created one was created we can see that it was creating and then created and success so first off let's go to lynode and let's see if that was really created we'll click on object storage and sure enough the buckets created exactly what i wanted but now if we go back into paloomi what we're going to see if i click on main can see that the update succeeded in a few seconds in fact it even maps back to the jenkins build number so i could track back to that here are all the different tags that are here if we take a look now at resources we can see here that we have a linux provider and then we are managing the object storage bucket with the name of darin dash and sumstream so why should you use jenkins to manage running your paloomi scripts first off it's going to allow you to do code reviews for any of the pull requests that come in for your infrastructure changes secondly since you are using a real programming language in my case i was using typescript you're able to use linters and static analysis tools to make sure everything is conforming to whatever rules that you want finally since you are using real programming languages like typescript python or any of the other ones that are supported by pollumi that gives you the ability to run unit tests integration tests or any kind of tests that you want if you have any questions or comments you can reach out to us on twitter at cloudbeesdevs if this video was helpful to you give us a thumbs up and if you haven't subscribed to cloudbees tv yet why not take a moment click on that subscribe button and then ring that bell and you'll be notified anytime there's new content available on cloudbees tv thanks for watching and we will see you in the next video

Channel: CloudBeesTV

Views: 369

Keywords: darin pope, jenkins, jenkins tutorial, pulumi, pulumi examples, linode object storage, pulumi typescript, infrastructure as code, pulumi tutorial, infrastructure as code tutorial

Id: CSmTOAr7ACo

Length: 25min 15sec (1515 seconds)

Published: Tue Nov 02 2021