How to Connect to RDS MySQL using AWS Secrets Manager from EC2 and Lambda by AWS Avinash Reddy

hi everyone welcome to my YouTube channel my name is ainash and in this video I'm going to talk about uh a uh Secrets manager service so basically the secrets manager is one of the um you know uh important uh security service so the secrets manager actually designed to store lot of uh sensitive information right for example you may have your uh database username and passwords right or any other like you know your application username and passwords that you want to uh encrypt and you don't want to hard code this credentials in your application right but you want to use that um uh sensitive information from your application so then you can uh create a secret in your uh Secrets manager so then you can retrive that uh from uh application where you want to connect or from EC to instance where you want to connect to the databases okay to demonstrate the secrets manager I'm going to use this database okay I just created this database even you can observe I just initiated this database creation and it success and here is this database uh information so this is my database endpoint so this is the master usern name and this is a master password all right so basically this database is running in one of the private subnet okay so I want to get connected to it so generally if you want to get connected to it um like you know there is Command right MySQL iph h iph p capital P small P right iph you for username so there we are going to give that username and password that is just connectivity process if you want to connect by using an application you have to hard code that credentials right so instead of hard coding how we can get connected to this database by use by um the regular command without any giving any uh password without entering any password okay so I have an E2 instance here so this instance also running in same VPC so first be without using Secrets manager how we can get connected uh to the database that process I'm going to explain later I'm going to modify this dat database to use Secrets manager currently under configuration You observe here Master username master password right so we have we are using this specific usern and password okay so let me connect to this uh E2 instance first I already have my skill package inside the ec2 instance I elevated my privileges to route so now okay so yeah I want to get connected to the database I have adjusted the security groups accordingly MySQL iph H to which instance you want to connect I'm taking this database uh endo and I'm giving that here okay then ion U admin is a username iph P the port number is a default port number 306 iph small p is to enter the password you can give the password here without any spaces or if you click on enter it will ask for the password so I'm entering the password my current password is Avinash 1 2 3 45 I have entered the password and I'm able to get connected to my SQL okay so this is a regular process this command or even if you want to enter you can give a one 1 2 3 4 5 then click on enter we got connected so within this command itself I'm passing my password okay so as I told you this is regular process now how we can use a secret manager to get connect but before that we have to enforce this database to use a secret right so currently as I have shown it is using a master username and password so I'm going to quickly modify this database instance and currently it is with uh username password where is it yeah currently it is with u um this right uh so the username and uh here we can reset the password but I'm going to enable this option manage Master credentials in a Secrets manager and also whatever the secrets we going to get I'm going to um encrypt by using default encryption key so then scroll down click on continue whatever the changes we are doing right so this uh like you know changing from hardcoded passport to Secrets manager password so this change need to apply immediately so click on modify DB instance before clicking on apply do I have any secrets here in Secrets manager no I don't have any secrets now let's go to RDS click on modify DB instance so immediately the database uh modifications will happen and under configuration tab earlier we have username and password right but this time we have just username and the password we got this Secrets manager Arn so the secret creation is in progress and the secret also encrypted by using a d default KMS encryption key that's what mentioned here all right um let's observe Secrets manager console you see here one secret um created here so this secret was created by Amazon RS so generally what we will get whenever you are creating a secret from ads console you will get two options one is username second one is password so the username is same admin but instead of the D 1234 Pi password a new password generated here and this is going to set as a password for the admin user you see here current status is resetting Master credentials that reset credentials are going to stored in the secrets manager so this is like you know a basic thing if you want to create a secret from Secrets manager screen you click on store a new secret credentials for Amazon Audi s what is a username what is a password you have set up for that so then you select the database then you can configure the rotation whatever the username and password we got here for how many days these are going to be valid so or we can enable automatic rotation also if you go to this rotation screen you see here rotation status is enable so rotation schedule is 7 days for every 7 days this password is going to change whatever password we got it will change for every seven days if you want to rotate immediately you do have an option to rotate it immediately or you want to rotate in a specific time then you can choose this edit rotation then you can choose okay I want to rotate it after like you know a week or two so again we need to uh have a Lambda function in back end to perform this automatic rotations all right so the database status also it's an available State and the secret actually created here so now earlier I have used this command to get connected right I have given password as a 1 2 3 4 5 if I'm trying to use same password what it's saying it's saying access denied for that user why because we enforce this password change and new password is actually storing in Secrets manager now how we can um retrive the secret from Secrets manager and how we can get connected to it so instead of hard coding how we can do that so for that I have a very simple code a a small script so let me show you that script so this is a script so we are retriving right we are running a command called a Secrets manager get secret value so we have a secret whenever we are creating a secret we are getting the secret name right so we have to modify that secret name here right once you modify that secret name here so then we are storing that information we are reading username and password and that username and password information or value we are using here in this command so we I I made it as a simple uh script so when I'm running the script so regardless of like you know manually going and checking and using that password this script itself will go to Secrets manager will check for the username and password it apply that values to this command so then it allow us to get connected to the database so let's modify this with uh actual database oops something wrong here with this editor now okay so here is a Secrets manager so first we need to grab the secret name this is what the secret name let's go and uh replace it so the existing one I I created this for testing purpose okay so let me replace this first go to insert mode all right so I have given the secret name here and also I'm going to modify this database host name also I want to get connected to this specific database so let me copy that and let me replace that here again this last command is purely optional one one okay so you can manually also use this command okay so we are giving this yeah so what's again what's happening this command is going to run my ec2 instance is going to uh read the username and password from the secret and that it is using here in this uh MySQL connect Command right col WQ right so now I want to uh connect to my database now uh let's run this you see here automatically I got connected so within the script I have not hardcoded okay so my request like you know from this ec2 instance whatever the request generated it went to the secrets manager it get the required value and and um we got connected now what happens if I rotate the secret so this is my uh current secret right retrive secret value this is my current password so let me write it down here in a document so this is current secret so I'm going to rotate the secret for that navigating to rotation rotate secret immediately then I'm going to rotate so now within a minute or two okay um the new password is going to set up here if You observe the status resetting Master credentials so whenever you are resetting or whenever you're rotating password in Secrets manager automatically a new password generates and the new password automatically set up here for this RS okay so this is going to take uh just a minute or two and you can just retrive the password this is a password is it same as a previous one no okay it's same it's same it's not like you know updated yet so as I told we have to wait for a minute or two e all right my database status is uh available so that means we should get a new password so go here retrive secret password You observe here so this is a existing one this existing password is overwritten with a new one again do do I need to update that uh here and all so no I already have this small script right so when I'm running the script again I'm connecting to the database automatically so the SSM a get operation is happening against at secret so we are getting the new secret and it is allowing us to get connected to it so this is the process from an easy to instance again so we created the secret uh from um RDS console if you want to create a secret from this you can store a new secret you can create a secret for RDS database or you can use for Amazon Dynamo DB or you can store credentials for red shift or any other database or you can even give in key and value pair also or you can give it in Json format as well all right so so for example I want to store something like a where username is equal to uh ainash and the password is going to be ainash 1 2 3 45 and uh I have a host entry host is going to be 10.0.0 do5 I gave some random one so whatever the information you're storing do you want to encrypt it if your answer is yes choose an encryption key then click on next now what is the name you want to give so YT IUN demo is a name I'm giving you give a description so then click on next so whatever the secret we are creating for high availability purpose if you want to replicate for backup purpose if you want to replicate to some another region you can enable this option as well so now do you want to enable automatic rotation instead of manually rotating this password you can automatically rotate so uh for that you click on schedule expression or you go with a Chron expression for every 7 days I want to then I can give seven days rotate immediately when the secret is stored and which Lambda function you want to so if it is a first time you are creating any secret click on this create function when you click on create function you give a name so remaining leave everything to default create a function so automatically a Lambda function creates so this is a Lambda function it created in my account yesterday I tried this okay so yeah so this is this Lambda function help us to rotate right so I'm going to choose that Lambda function then click on next it's a review screen right to retrive the secret we can use this code if you if you read this code we have this secret ID right we have the uh like you know where is the secret ID in which which region it stored so we have that information again a small thing how this E2 instance is communicating with the secrets manager so I have Associated an appropriate role uh to this ec2 instance here I have a role with a name called Secrets role that role is allowing uh the zc2 instance to retrive the secret from Secrets manager service right so along with this ec2 instance if you want to retrive or if you want to get connected to your database by using a Lambda function so you can still do that so but uh like you know when coming to this Lambda when coming to this Lambda so I have a um small Lambda function so basically this Lambda function is going to um connect to Secrets manager it is going to retrive the secrets so if You observe here what secret name you want to and in which region you have and to get connected to your database what is your DB host name what is your database name inside that what is a p pass port number okay so the password U and username it's actually comes from the secret so from the secret we are reading this username and password option we are using that to get connected to it so I'll make this um uh code available from our U GitHub link I'll give that GitHub Link in our video description so for now let me grab this database um endpoint name let's edit uh that here so I'm giving my data database endpoint name here and DB name is my DB only if you have any uh other DB you can use that so you can even verify that if you go to configuration you can observe that information somewhere here DB name is my DB so we are good with it and um port number 3306 and secret name so we have a secret here right let's grab that secret name let's give that secret here in this Lambda function and the secret actually storing in AP South 1 now deploy okay so I'm using this um Pi myql package so the required packages is here and this is a Lambda function okay so now I want to test this so when I'm testing what happened if my Lambda function is able to get connected to my database right it is going to return with um right uh connected to database successfully if there is an issue it give that error it's going to give like a uh status code error status code again where this Lambda function is running so this basically this Lambda function is running within a VPC so my database and this Lambda function within a VPC and it is running in a private subnet also how this Lambda can communicate with Secrets manager okay so this Lambda is running in a private subnet and my secrets manager right uh if if this Lambda want to access any secret so it definitely required communication for that communication you need either a ad Gateway so you should have a ad Gateway in your VPC for your secrets manager otherwise you should have an endpoint Secrets manager interface endpoint so I have a Secrets manager interface endpoint so that is running in uh this subnets and appropriate secrity group also Associated that is making this U communication possible so now go to this Lambda function then click on test so the function execution in progress and you can see connected to database successfully so from this Lambda by retriving secrets from Secrets manager with the help of VPC interface endpoint we are able to get connected to a private uh subnet running database and we are able to connect here all right so that is all about uh Secrets manager and how we can use Secrets manager uh to you know store our secrets and how we can get connected to our MySQL RDS MySQL from an easy to instance as well as Lambda function I hope you guys found this video helpful so see you again uh in next video thanks for watching please subscribe for more videos on a topics thank you

Channel: AWS with Avinash Reddy

Views: 1,724

Keywords: aws, aws with avinash, avinash reddy, aws avinash reddy vpc, naresh technologies, aws cloud computing, aws cloud practitioner, aws cloud practitioner full course, aws lambda

Id: 1wjrZpM7YDE

Length: 22min 57sec (1377 seconds)

Published: Thu Dec 21 2023