Pulumi - IaC in your favorite programming language!

in this video we're going to talk about an infrastructure as code tool which is gaining popularity in the devops world and that is pollumi first we will see what is polumi and what makes it different from other popular infrastructures code tools also what are its main benefits and characteristics and then we will see how polymer actually works in practice by writing a simple pull me project to first create an s3 bucket in an aws account and then a more realistic example of provisioning an eks cluster when writing infrastructure as code scripts using all these other tools you usually have to use a domain specific language of that tool or yemo or json format which is something that a lot of tools use and that is great because it's easy to read and easy to define resources you want to create etc but what if you need to add some more logic in there like conditionals for example saying create these ec2 instance if this condition is met or use loops like i want to create 20 ec2 instances with the same configuration or what about reusing the logic like all 20 instances with this shared configuration plus some specific configuration for each one of them so if you want to use this kind of logic it gets actually pretty complex in such formats now all these logical operations and loops reusing blocks of code etc is a natural part of any programming language right so writing the same infrastructure as code logic in a programming language would obviously make writing more complex code easier and cleaner and that's exactly what pollumi did polomi allows you to write infrastructure's code in a programming language now which programming language is that well you can actually choose whichever language you want to use with polumi you can write it in node.js in python.net etc so instead of having to learn a new syntax or a dsl or new language you can use the language you are most proficient in and use it to also write infrastructure as code okay now you may be thinking cool i can use a programming language to write infrastructures code but what other clear advantages do you actually get by being able to write infrastructures code in a programming language of your choice well if you are a python developer for example you're probably already using the most fitting ide for python like pycharm maybe you have installed all the plugins for that language that makes it easier for you to write python code you have configured linters that tell you immediately if you have errors in your code or give you warnings about your code quality plus you know all the main libraries you can use in your code from python so basically you know the whole ecosystem of that language and the tools that you can use with it and that means you can use all that same ecosystem and the tool set as well as your expertise in that language to now write infrastructure as code and this can make you work much more efficient another thing that we see emerging and becoming a standard in devops is treating our infrastructure's code exactly the same way as our application code and that includes testing the configuration code having unit tests integration tests etc so with yaml and json format or a domain specific language like hcl that terraform uses it's pretty difficult to write tests for these scripts with polomi obviously you have the advantage of simply writing tests for your infrastructure as code as for your application code using the same test libraries same programming language same editor and so on now this shift of using programming languages instead of yaml or json format allows developers to now also write infrastructures code which usually is seen as primarily a devops or operations team task so it results in operations and developers working more closely together where the operations team brings the know-how on the infrastructure specifics and what needs to be configured in terms of infrastructure components while developers bring know-how on how to write the code most efficiently to configure the infrastructure in the desired way while also being able to properly test the code and use the existing tools for writing and managing this code now that you know what differentiates polumi from other infrastructures code tools let's actually see how it works in practice so in this part of the video we're gonna create a simple plumi project in node.js that provisions resources in our aws account first we're going to create an s3 bucket as a simple example and then we will provision an eks cluster using polumi as a more realistic use case so let's get into it first we need to install polo me which is super easy you have detailed guides here for each environment so on mac i'm just gonna do brew install polomi once polami is installed we'll create a new folder in this folder we're going to create a plumi project and to bootstrap the project plummy actually gives us a boilerplate code to get started with and we can do that using simple polumi new command so if i execute this in this new folder that we created i have a choice of technology or programming language and platform combination right so we have the platforms like aws azure google cloud kubernetes and then we have different programming languages that you can choose from since we're going to be creating resources on aws using node.js or javascript i'm going to go with this option here and enter some project information project name description stack name aws region this is important because we're going to be creating resources in a specific region i'm going to choose my region in europe and our project is ready to go now if i do ls here this folder was empty and now i see a lot of files that pull me actually generated and if i open the project in visual studio code you see that polumi initiated a node.js project so we have package.json with polumi libraries as dependencies and they have been automatically downloaded into node modules folder and you have index.js the starting code where all libraries are imported already we have polumi core code as well as aws libraries since that's what we selected and you already have a simple example code that creates an s3 bucket in an aws account and as you see this is simple node.js syntax or javascript syntax we are creating an object this is the name we can reference it with in polumi and then saving a resulting bucket id which is the bucket name that we will see in aws and we can actually execute this code just as it is and this will create a bucket in our aws account now how does polomi know how to connect to the aws account or which region to create resources in well plummy takes the aws credentials from the default dot aws folder in your users directory so if you have that set already you don't need to do anything but alternatively you can also set them as environment variables or configure them using polymer itself and that will work too and you can actually check the plumi documentation for all the different ways for setting up the platform configuration that polumi connects to and this one is specifically for aws so in my case i have this dot aws folder already configured and that means that we're actually all set and ready to execute this now to execute polami project we're going to need a simple polumi up command so if i run this first of all you see that we get a preview so nothing gets executed automatically it actually just gives us a preview of what's going to happen and then we have to confirm that we want that change to happen either saying yes or no or asking polomi to give some more details so we can make a decision and the preview before executing it is actually important so that you don't accidentally destroy the existing infrastructure so i'm gonna confirm we see a successful result and right here we have output the bucket name that we exported right here and this is a convenient way of exporting all the outputs that we want to see in the console right after the execution now obviously that's a super simple use case so let's do some more stuff with it like creating multiple buckets in a loop so i'm going to define three bucket names and then for each bucket name i want to create a new bucket with that specific name and again a simple javascript syntax for creating loops and executing loops and we can also export the bucket name for each of the buckets simply like this using our familiar javascript array and then just export the whole array here instead of just one id and now this will give us three buckets with three different names and we're gonna do polo me up and in preview we see again polomi's plan of what needs to happen in the infrastructure when we execute the program so the way it works is that polomi has a concept of state plummy stores the up-to-date information about your infrastructure in a state file in a simple json format and using that state plummy knows what needs to be created deleted or changed in your infrastructure so on every change in the infrastructure the state will get updated by default that state file is managed by pollumi itself it is stored on a hosted polumi service so it's completely managed for you but you can configure polomi to store this data file in a different location if you want to like s3 bucket or google cloud storage or also on your local file system and then you have to manage it yourself so using that state polomi decided what changes need to be made in the infrastructure by comparing that current state the most up-to-date state of the infrastructure with the desired state that we define in our program or in our script and plans how to get us from the current state to the desired state and this is basically the plan so we're going to confirm this the old bucket will be deleted and three new buckets will be created and you see the output now is an array of the bucket names that were created you can also print the output separately with pollumi stack output command like this now another cool thing that polumi gives you is polymer console so at the beginning of each execution you have this link here and if we open that link it will show you the overview of that specific execution in this nice ui and you see it's hosted by polum itself on your account and you have these activities as a list here as a history and for each activity you have the changes in your infrastructure as well as changes in your code that updated the infrastructure so you have this super nice overview of history of your infrastructure changes so you can track exactly what changes happened when and by which team member now this is obviously a simple s3 bucket infrastructure so let's do something more useful and create an eks cluster with polumi so first of all i'm going to delete all this code and do pull me up and plummy will clean up everything so as you see polymer is declarative which means we just say what we want as a result which in our case is we want no buckets instead of telling pulumi what to do which in this case would be please delete three buckets so clear advantage of that is that we don't have to keep track of the infrastructure state to be able to tell polumi what needs to be done or changed in the infrastructure because pollumi does all that for us so i'm going to confirm and now that buckets are gone let's create an eks cluster and first we would need an eks polomi library for that and we're going to define that in package.json as a dependency and as you see again advantage of using the ecosystem of a programming language in this case editor is auto suggesting the version of this eks library without having to look it up so i'm going to select it do npm install to download the library and we can now use it in our code now to create an eks resource we first need to create a vpc for it which is pretty easy we just say new vpc and the name of that vpc and you see an other suggestion here as we know them from programming languages which i personally think is so helpful when writing code because you don't have to remember all those function names attribute names etc so that's a simple vpc constructor and we need to set a cider block here like this which is going to be an ip address range for that vpc so vpc definition is done now let's create an eks cluster that uses that vpc so we have the cluster name let's call it cluster and referencing other resources is super easy just like we know it from programming languages name of the object and attribute vpc id let's also set the subnet ids from the vpc and set the instance type to t2 medium so generally as you see for each resource we want to create after the resource name we can pass these configuration options to override the defaults like ciderblock vpc id instance type etc and finally define the output we would need from the eks which is the cubeconfig file to connect to the cluster so exports and we can call it cubeconfig and this will be the cluster dot cube config attribute and this should actually be called cluster that's our cluster variable name and now this simple program is actually ready to be executed and with polumi up we'll get a preview of all the resources that this program will create let's confirm and we'll have a fresh new eks cluster in about 10 minutes eks cluster was created now going back to the polumi console if you click in the resources tab here you will see a list of resources in your infrastructure so that's your current infrastructure state and with the prefixes you can differentiate actually which platform or which infrastructure part these individual resources belong to so we have polomi resources here we have aws and then sub resources like ec2 or iam and then you also have these kubernetes resources and next to some of the resources you have these convenient links directly to that resource in aws console so you can view them in aws directly and next to the list view of resources you have a graph view that will show you the connections between those resources as well as which components actually belong to or depend on the others which is super cool because in our program in our code we just have these high level components that we created like vpc and cluster and then all the stuff that gets created in the background underneath the vpc and eks cluster is displayed in this overview so you can see all the components that were actually created in the background in this hierarchical view so as you see polumi console gives you this simple ui with the history of your infrastructure changes as well as a nice overview of your current infrastructure state and finally to clean all these up to delete all the resources that you created you can actually execute polumi destroy command and it will basically just reverse the pollumi up execution by destroying and cleaning up all the resources so i hope you learned a lot in this video let me know in the comments what you think about polumi and generally what is your experience with infrastructure as code tools and with that thank you for watching and see you in the next video

Channel: TechWorld with Nana

Views: 73,112

Keywords: pulumi, pulumi tutorial, pulumi vs terraform, infrastructure as code, iac, infrastructure as code example, techworld with nana, what is pulumi, terraform, cloud engineering, pulumi node, pulumi aws, pulumi kubernetes, terraform vs pulumi, pulumi python, pulumi explained, devops, devops tool

Id: vIjeiDcsR3Q

Length: 19min 40sec (1180 seconds)

Published: Fri May 28 2021