Vault on Kubernetes with PostgreSQL

[Music] what is up youtube and welcome to another super exciting video on secret management this is the future so if you guys are new to the series please check out the links down below i have made a getting started guide on vault on kubernetes on basically i'm spinning up a vault how to configure it all the basic concepts the algorithm around the encryption key and how all of that works i've also created a video on end-to-end encryption for tls to make sure our vault connectivity inside our cluster is fully encrypted and then i've also made a video on injecting basic or static secrets from vault automatically into our application so in this video we're going to solve a very common problem that i'm sure most of you have you have a database and you have an application how do we get vault to automatically create users in our database and then assign the password and inject the password into the application automatically without the application even being aware it's being done and also without the database being aware so we can also control or get vault to control the credential fully end to end the whole life cycle of the credentials so if an application dies vault will revoke the username and password we can also put a ttl on the credentials if the credential expires vault will update generate a new credential and inject it into our application without the application even knowing so in this example i have a postgres db running in the cloud you can have that postgresdb running in a cluster or anywhere i'm just using postgres as this example and then in the future we'll take a look at other databases and other examples so without further ado let's go [Music] so just so you guys know everything in this video is on github it's on the docker development youtube series in the hashicorp fault folder you will see a readme guide so everything i'm doing in this video is in this guide you can follow along as well [Music] now in this example we're going to need a kubernetes 1.17 cluster because we need the the admission controllers enabled by default i'm going to use kind to provision a cluster if you haven't checked out the video on kind check out the link down below i'll include a video there where i basically create kubernetes cluster within docker in like a few seconds and you can run multiple versions different versions of kubernetes especially if you want to throw away the cluster afterwards so once i have a kind cluster up and running i'm going to go ahead and create a new namespace called vault example and then what i'm going to do is i'm going to apply my hashicorp vault inside of that namespace and we can just see the vault is coming up and ready if you haven't seen the video on vault check the link down below i go through all the concepts of how volts creates storage its configuration the algorithm and everything you need to get up and running now that we have a new vault up and running i'm going to go ahead and execute a command in the vault which is vault operator init that'll initialize the vault for the first time give me all the unsealed keys and the root token so i want to go ahead and save that somewhere and then what i want to do is go go ahead and unseal the vault so i'm going to run vault operator and i want to pass an unseal and i'm going to use the keys to do this and i have to do this three times to get the vault unlocked and after the vault has been unsealed if we do a get pods we'll see our readiness probe will pass and our vault is now ready to be used so let's firstly take a look at our use case we have a postgres database up and running and then we have a kubernetes cluster with a pod running now this it can be a legacy php app can be any kind of uh application that needs to talk to the database so what we need to do the first thing is we need to run this application as a service account on kubernetes this will allow us to authenticate and authorize and have some kind of identity for this application so our vault will then be able to authenticate with kubernetes to basically verify that this pop pod is allowed access to the secret so the first thing we're going to need to do next is enable the um the kubernetes auth inside of vault to allow vault to authenticate with kubernetes once we have that in place we can then apply a policy for the pod so we can use the policy to say this service account has access to that postgres database and then automatically go and fetch and create a credential which will be created in that database automatically for us and then inject it to the pot as a secret and the cool thing is that pod will then run with that secret and if the pod dies will go back and clean it up automatically so to enable this authentication mechanism to allow volt to talk to the kubernetes api we're going to go ahead and run this command inside of our vault called fault auth enable kubernetes once we have this enabled and we're going to go ahead and we're going to jump into our vault and we're going to write a config and the config will allow us to authenticate with with kubernetes so we're going to create a vault right off kubernetes config so this is a config file within our vault and what we do is we pass a token which is the kubernetes service account that the vault runs as and we point it to where the kubernetes api is and we also pass it the ca certificate once we have this in place our vault can now talk to kubernetes api now i have a sql database in postgres right here in the cloud and part of the operational um pain of this in terms of secrets is i have to create service accounts or user accounts for applications to connect so here i have a postgres users obviously the root user and the admin super user now how do we get vault to automatically create and provision users for us so that's the first step okay so for this to work we actually have to tell vault about our database and we have to authorize vault to act on our behalf now for this you probably won't use your master password but you could create a user account with permissions to basically delegate the creation of user accounts on your behalf so to do that we're going to log into vault and we're going to give it our root password so now we're in vault and what we're going to need to do is enable the database engine so i'm going to say vault secrets enabled database and this will allow us now to configure our database credential creation so the next step i'm going to do is i'm going to get a terminal into our vault and now i want to write a postgres database configuration this will basically tell vault about our database and now you can use a specific user account that you want to use to act on your behalf so in this example i'm just going to use my um my default one so i'm saying vault write and i'm creating a database config for my post screen so this is the name of my database called postgresdb um this is the plugin name we want to use allowed roles you can define a role i'm just calling it sqlroll and then here is my connection url format that connects to my postgres so i have a default database in there called postgresdb i'm just turning ssl off for now for this database that's the url and the port of my database and then the username and password so we go ahead now and we tell postgres about our database with this configuration the next step what we have to do is we have to define a role here so we're going to say vault right and we create our sql role what we also tell vault is how to create these users so we put in the sql statement of basically defining the type of user we want to create now in mind i'm just saying create a role with the name and the password and i'm putting an expiration on it with a default expiration of one hour and then i'm granting select on all tables so you can kind of make this a little bit more granular depending on your needs so i'm going to go ahead and create that and that now creates a roles called sequel roll so you can basically define multiple different roles and then what we're going to do is we're going to map the roles to a kubernetes service account so any kind of microservice app job workload that runs in our cluster that is part of that service account list will automatically be able to use this feature all right so now we've told vault about our database and we've delegated the user creation to vault itself so we've eliminated the need for humans to create passwords and pass them insecurely between people and applications so now that we've got vault automatically creating a username and password for us how do we get this to automatically inject into our application now in vault everything is controlled with a policy so what we're going to do is we're going to create a policy and bind it to this role that we've created and then we can map kubernetes service accounts to this policy and we can start up example applications and they should automatically get the credentials injected so what we want to go ahead and do now is jump back into our vault and log in if you're not logged in and we're going to create a policy so i create this file called postgres app policy which is a vault policy file and i give it access to a credential path now everything in vault is kind of like a folder structure and you can create your own structure we've enabled the database engine so it will fall under database and then we have creds which is the credential and this is our sql role that we created earlier so anything that kind of falls under this policy will get read access so they'll be able to create credentials on the fly so we want to go ahead and create that policy file and then the next step we're going to do is we're going to say vault policy right and we're going to write that file into vault and we're going to call this policy postgres app policy so go ahead and apply that and now that i have created a policy the last bit that we want to do is kind of bind that policy to a service account so because we have a we're going to have a service account for our pod and we want to go ahead and say vault right and it'll be a kubernetes auth with a with this role so we created the sql role you can give it any name we bind our service account i have a service account called dynamic postgres i'll show you guys in a second that's going to be the service account that runs our pod our basically our php application and it's going to run in vault example namespace so you have to tell it which namespace to look into and then we're going to apply our policy so this is our policy that we created earlier and we're going to give it a ttl of one hour [Music] now the moment of truth to test this we can run a command called vault read and we can read that that role and as soon as we do that we can see that vault has gone and created a user account so it created this user and this is the password so if we jump back to our postgresdb and we do this command again we can see a new user has appeared and this user will only live until this time which is an hour from now an hour from now this credential will expire and vault will come in and delete that so this is just a normal application can be a php app legacy app can be any kind of a micro service that you're running i'm just calling this one dynamic postgres isn't the name of it just because this is dynamically this is part of the example giving it an app label i'm running a service account so this is the key part we run a service we create a service account called dynamic postgres and we run this deployment or this pod as that service account so now kubernetes will know that this service account has access will automatically be able to authorize through the kubernetes api and we'll be able to get our secret and the special part or the magic all lives in this annotation everything else is standard kubernetes syntax the annotation is what hal's vault or the injector to inject so we say agent inject is true we tell it to skip verify um tls because we're running self-signed certificates in this example and then here we say agent inject secret and this is the the secret name we say sql role and this is the full secret name in vault which is database create sql role and then we pass in a template to use now we also specify here this the the secret that we want to apply for in this template you can have multiple secrets with multiple templates if you want to if you're if your application is expecting multiple secrets from the vault and here we're using a normal go template syntax so we say with this database create sql role generate me this json file with db connection and then this is going to be the connection string that it will generate this allows you to kind of take a credential like a username and a password and then turn it into something that the application expects like a config file or some kind of uh formatted credential file and then finally last but not least we have the role which is our sql role this is the role that we want to use um in order for the vault to do all the authentication mechanism and give us access to the secret that we need now for um secret injection to work we also have to deploy the injector the injector is key because that's kind of the service that injects the secret into our pod so i'm going to go ahead and deploy that injector and then we can do a cube ctl get pods and we can see our injector is now running so the injector basically talks to the vault on the pods behalf it also injects it basically takes in the role of injecting the secret and does all the authentication for us so now with everything ready we have a vault up and running the vault has been configured to talk and verify with kubernetes about service accounts that run we also have a policy that binds to a role which binds to a service account so now vault has the ability to to talk to our postgres db it's a it has the ability to generate credentials so now what we're going to do is the moment of truth we're going to go ahead and apply an example application so we want to go ahead and run our example app this is our deployment file for our application workload that's going to be expecting a secret to be dynamically created and injected and if we do cube ctl get pods we can see our application has started up and let's go inside and see if our credential is there so we do exec it bash run sh now we're inside and if we cut out vault secrets sql roll we can see our db connection string was automatically generated we can also see that a new user was created and injected here with a password and if we hop back into our postgres db and we look at the user accounts we can see a new user account has come up so now fault will automatically manage these accounts on our behalf we don't need a user to come and manage the accounts anymore we don't need users to create secrets we basically don't have to insecurely pass secrets around people and this all happens automatically behind the scenes so that is as simple as it gets super secure dynamic sql credential creation and injection into your application with no humans involved whatsoever and this is not just limited to postgres sql this is can be used for other sql databases as well i'm actually quite keen to jump into some of the cloud integrations we can basically interact with different cloud providers storage so let me know down in the comments what sort of integration you'd like to see and remember to like and subscribe and until next time [Music] peace

Channel: That DevOps Guy

Views: 4,781

Rating: 4.9463086 out of 5

Keywords: devops, infrastructure, as, code, azure, aks, kubernetes, k8s, cloud, training, course, cloudnative, az, github, development, deployment, containers, docker, rabbitmq, messagequeues, messagebroker, messge, broker, queues, servicebus, aws, amazon, web, services, google, gcp, redis, vault, kms, security, hashicorp

Id: IWCOptiCKqI

Length: 15min 37sec (937 seconds)

Published: Wed Aug 12 2020