- Hi everyone, my name's Ryan H. Lewis. And today I wanna talk about
one of the most difficult or tricky problems in cloud software development. And that is where to put your passwords Not your password to your email account or your bank or anything. But when you have an application, often it uses usernames and passwords to log into things like databases. And so where's the best
place to put those? Not in your code, if you answered that,
I'm very upset at you. Definitely not anywhere that's gonna go into source control. Although I have done
that a few times myself. But really when you're in the
cloud, where do you put it? And so that answer has kind
of changed over the years and it actually kind of depends on the cloud provider you're using. But specifically for AWS,
where would you put it? Well, in the past, there
was a couple of ways. One thing you could do, is you could put it into
your cloud formation, not the templates but when you execute a
CloudFormation template, you could pass it in as a parameter. You could set that
perimeter to be protected, so you wouldn't be able
to see it in the console. And then you would
basically set that as maybe like an environment variable
somewhere, that's not secure. Don't, like don't do that. I have done that in the past but that's not the good way to do it. Then there's another service
called Systems Manager. And Systems Manager has
gone through a few changes. In the past it's actually quite good now. But with Systems Manager they had a thing called parameter store, which was a secure place that you could store basically parameters. And you could get them
out with the AWS CLI and also from your application. So this was a good place that you could store
username and a password, and you could retrieve
them in your app as needed but they weren't being
stored on the EC2 instance. The reason you don't wanna
store on the EC2 instance or like a Lambda as an
environment variable, is because if that were
to be compromised somehow then people like an intruder or a a malicious attacker would be able to read the username and
password for your database. And then it's just like it's just an easy step to
get access to your database. So you don't want that. So this was a problem. There were some solutions but they actually weren't that great. And so AWS actually provided a solution. They built something almost specifically for this called Secrets Manager. And Secrets Manager is it was released a couple of years ago. I think at a re-invent
is when it was announced. And I've started using it and it honestly solves
the problem perfectly. Basically what Secrets Manager
is, does a couple of things. But one of the main things it does is it stores sensitive data such as username and password, and allows you to retrieve
it with the AWS SDK or the AWS CLI. So you can have your application
running on EC2 instance. You can use IAM permissions
to define who can have access to the secret and Secrets Manager. In your application you can build a policy so that it gives access to
whatever role is working in your EC2 instances running on. And then when your application needs to get a username password for a database, or for like a third party
service or something, it just reaches out to Secrets Manager, pulls, gives it the key of the
value that it's looking for. And then Secrets Manager gives
it the username and password or really whatever's stored in there. You can also store like
keys, like access keys, and things like that for
third-party services, tokens, session tokens, things like that. And then your application can use it to do whatever it needed to do. So that's really cool and it works. It's very simple process and it's really custom built
just for that use case. It does do something else though which is actually really interesting. And I haven't gotten to play with as much as I would like to. That is automatic rotation of username and password credentials for databases. So if you're hosting your database in RDS, using one of its many database engines, Secrets Manager is pre-built to connect to that database in RDS and rotate the username and password as often as you want it
to, which is really cool. You don't have to, you just set up the automated rotation and it does it for you. And so this means, you may think, okay, well, if it rotates it how's my application gonna
know username and password. And, you know, that's kind
of one of the, you know based on previous cloud development or even before the cloud development that's kind of something
you had to deal with is how does your application know that? But what's awesome is the key for that username password stays the same. So your application goes up and you know, on Monday it goes up and says, give me the
username and password. And it's like admin password. And then it uses that and
connects to the database. Then it's rotated on Tuesday. Then Wednesday, it goes
to ask the same thing and it's admin three, password four, and then it connects to the database because your application never knows what the username and password is except when it needs to know it. When it rotates, when the Secrets Manager rotates
the username and password, it updates it in the database, it updates it in Secrets Manager and then your application
just gets the new thing every single time. So it's really cool. It's a very efficient process. And it's one of my favorite services because it's so targeted
on a use case problem that I have pretty much always had when I've been developing in the cloud. With Secrets Manager you can
store usernames and passwords, you can store access keys, session tokens. You can really store anything. There's a plain text mode that
you can just, I don't know. You could put, you could put a novel end. Well, probably not a long novel, I'm pretty sure there's
a character count limit but you can put kind of
anything in which is really cool but I'm gonna show you that today we're gonna create a secret
username and password and then we're gonna get
it in code using the SDK. And then we're also gonna
get it using the CLI and we're gonna parse it. So there's gonna be a couple
of things we're gonna do today. So let's take a look. Here we are in the AWS console and Secrets Manager is a service. So you can just go to Secrets Manager. I don't have any secrets in here yet. So let's store a new one. Actually here's one thing, you'll notice it does tell you pricing. So it's not free, it's
40 cents per secret. And then, you know, if you
are having a ton of API calls, it charges you per 10,000 API calls. It's not that bad. Honestly I think it's
actually pretty good. So let's store a new secret. So here's all the options, you can do credentials for RDS which will let you rotate them. You can do credentials for document DB which is a Mongo DB compatible
document database in AWS. You can do Redshift. You can do another database. I don't actually know
what other database means. Well, I guess, oh, I guess
you can pick these engines and then you can connect
some, that's cool. Other types of secrets is
actually what I use the most because a lot of times, you know, if I'm doing something with CircleCI or some sort of third-party service, a lot of times that's when I
need a username and password. A lot of times what
these apps that I've used that are using RDS, typically the username
password is so baked in and I'm coming so late to the game that converting over to Secrets
Manager would be, you know it's never a priority. But anyways, I'm usually using it this way and it works really good with this way. So let me show you what this looks like. So I'll say here. I'll make a username
and we'll call it Ryan. So there's two ways to do this. You can do key value or
you can do plain text. It's just JSON, you're gonna
be processing this as JSON on the other end, almost always. So this is just a pretty way
to build the plain text JSON, it's just the same thing. So we'll do password and we'll say, you know, fancy password, obsolete. Cool, and then you name the secret. And so this is the key that you use when you're getting it out. So we'll say test_userpass,
you can give it some tags. This is probably a good place if you have an application that it's specifically associated with. A tag is a good way just
to keep it organized. And then you can set up
the resource permissions if you need to access it
from different accounts which is something that
sometimes you may need to do but we don't have to know. So here's where you set
up automatic rotation with databases that it can connect to. This is where you would set that up. I don't know what it's doing with the arbitrary kind of key and value pass. I don't know if rotation
would really do anything. So it gives you some sample
of code that's so nice. Let's see what it looks
like in JavaScript, yap. All right so then we'll store it. So, okay before I was
messing around with this and I use the same thing, you have to schedule these for deletion. I'm gonna change the name. I already had one that named. You have to schedule these for deletion. You can't just delete them outright. That really does that so you'll go in and
you'll change your code. So we've got test_usepass2 in here. If you click on it, it
doesn't give you the, it doesn't let you see the value here. You kind of have to go an extra step to retrieve the username password. The secret value is what it's called. And this is nice because if you're showing
somebody else this thing, they don't get to see it. You can also set up IAM
permissions around this secret that can manage who can actually see it and do things with it. So let's look at some code. I've got code here with JavaScript and it is pulling in the AWS SDK. Let me see if there's a good Javascript. Set my region using the AWS
Secrets Manager SDK object. I make a function here that takes the key and then it sets it as a secret ID. And then I use this Secrets
Manager getSecretValue function to actually go and get the
value and bring it back. Passing in that params object. And then I've been using
this all time recently, the AWS SDK promise function, which means I can pass it and I sync away and do things nice here. Here it's just gonna console log it. And then I put some fancy no
JavaScript stuff around here. So I can pass it in as a
argument in the console. So here I am in this exercise
folder with Secrets Manager and I can run my JavaScript
file with note index.js and then I'll pass in the secret ID. So this is one that I
have the secret name. And if I do this, hopefully it'll work. Yes, it works. So here's what it gets back when you call getSecreValue. You get the ARN which I don't need. You get the name, which I know, you can actually version these
secrets which is really nice. With this, I only need one version, which is why it's set at AWSCURRENT. And here's the secret strings. So this is a value that you
want to get out of here. Then you can, in JavaScript, you can, JSON parse this and you can get the values and actually do the thing you wanna do, which is really cool. So that's one way that you can get the secret out of secrets manager in
JavaScript with the AWS SDK. Then we also may want to
get it in the console. This is something that like a lot of times if you're running AWS CodeBuild, you'll want to get the, you'll wanna use Secrets Manager. Maybe you're just running some cron jobs. You'd like to use Secrets Manager. This is how you would do that. So I've got a bunch of
different things in here. I've got the, getting the entire secret. So let me a comment out this and I'll run this, let's do. So if I run it's using the AWS CLI it's running the Secrets
Manager getSecretValue and then it's passing
in the secret ID here. So you'll see you basically
get the same thing that you got in the AWS SDK. You get just a whole bunch of stuff you probably don't need when you really just want the value. And so you can add, I've got a
little multi-part thing here. You can add the query secret string and that will just get you the, let's see if I say I set that, I didn't. That'll just get you the
secret string property off of that JSON response. So here's, the part that we really want. And then if you want to remove the quotes from around that,
you can use the output text. I don't need to show you that. But one of the things
that I liked the most is when you get that,
it's just JSON, right? What's your application
gonna do with JSON? Probably nothing, if it's coming in the command line like that. And so what I like to do
is actually parse that JSON and get properties off of it. On Mac and easy way to do
that in the command line is using JQ. And you can use Homebrew to
install it with brew install jq. And basically what you do
is you, you can run that, you can get that text output, so it doesn't have the quotes around it, still JSON structure. And then you can pipe that
into JQ using this format which I had to look up on the internet and you can pass in the property
you wanna get off of that, that JSON object. And so here I run it
and I get the username, then I run it and get password, there's probably a way to concat that. So it's just one command. But let me show you what that looks like. If I run it again, you'll
see I get my username and my password plain text. If I needed to do anything with that at a later point than I totally could and I could use those in
my command line utility. So those are the different ways to use Secrets Manager in AWS. It's going to be the best way for you to manage passwords when you're developing in the cloud. So definitely try it out
and get comfortable with it and make sure that you
aren't making the mistake of creating your application and building out all of
these username passwords, putting them in your code, putting them as environment variables. There's all sorts of ways that
you could do crazy things. I've probably done 95%
of those crazy things. So thanks so much for watching this video. If you liked it, consider giving it a like or subscribing to my channel for more. I'll see you next time. (soft music)
