Store & manage secrets like API keys in Python - Tech Tip Tuesdays

Hello friends and welcome to another tech tip Tuesdays where we give you Tech tips on Tuesdays in this video we're going to be looking at how we can securely manage and store our secrets like API keys and other credentials in our python projects these are very sensitive and it's important that we get out of bad habits like hard coding them directly into our source code in this video we're going to look at the basics and how we can quickly do this securely and we're also going to look at some more advanced scenarios like at the end we'll look at how we can manage the secrets of different environments say production and development very easily and switch between the two using a simple package but before we get into all of that if you like this type of content then I hope you would please consider liking and subscribing to our Channel it really helps especially as we're just getting started we have content like this all the time with lots of great content in our library all right let's dive straight in on my screen you're gonna see an incredibly basic python project this python project has really got two parts to it now while this is really really basic these parts may transfer into something that you're working with in your own project so we're applying our secrets so that means we have a variable here we have two API key and API secret and then we're giving them some values and then we're using our secret now in my project I'm just printing them out in our console there's no real scenario where this is a good idea but we're going to do this just as a way of illustrating hey we've got our secrets we're doing something with our secrets so if I run this project here what happens well as I said it just prints out our secrets but this is really insecure why well we've committed some pretty big software engineering sins here we've hard-coded the secrets into our source code that means that this is saved in this file if we Version Control it it's going to be in our Version Control it's going to be cloned onto whoever else's computer it's going to maintain in our history and these are really really sensitive so this is a terrible idea what's the way that we can handle these secrets instead well we can use environment variables so an environment variable is in a variable that sits in your local memory now this might be the local memory of your machine or it could be the local memory of the cloud host that you're going to have your application running in when we deploy it doesn't matter it works in the same way Sentry we want to inject these secrets into our local memory then our application can use them without fear of them being exfiltrated from an attacker and we don't need to put them in our source code great right so how do we do this well it's actually really simple if I run the command e and V this prints out a list of variables that are running in our local environment so what I want to do is I want to put these variables here inside this environment so using the command export then the variable name API underscore key we're going to copy this value paste it in here we're going to do the exact same thing for the secret key right what happens next well if I type e and V again we'll see that we have two new variables down the bottom API key API secret that means that these keys are in our local environment all right so how do we actually use keys in our local environment well in our python project we're going to import our operating system so we're going to write import OS we're going to get rid of these values here and we're going to put OS get EnV and then we're going to put the name of the invite of the variable that we want in this case API key I'm going to do the same for the secret all right so what happens now when we run our project so here you see it prints out that values of our secret just as it did before but this time is using our local environment to be able to do this okay so that's all well and good but this is not a very efficient way of dealing with this problem if I destroy this terminal at the moment create a new one if I run the exact same command e and V well our secrets are gone if I try and run the project then our values come back it's nothing so that means that every time we create we run a project in a new window we have to inject the secrets again well that seems like a lot of work it's probably not something I'm going to do I'm going to go back to hard coding secrets but there is a way to be able to do this the project python.inf is a fantastic project let's use lots and lots around the industry and it's a way of automatically loading in our environment variables from at dot EnV file so we can install this on our machine really easily using Pi Pi so I'm just going to run the command pip3 install python.mv mine's already installed but yours will install in a couple of seconds so how do we actually use this project well it's incredibly simple the first step is we need to create a DOT EnV file so I'm going to use commandtouch.emv and you'll see that it's created this EnV file it's an environment variable file these files are used in multiple different languages to store environment variables now what we're going to do is we're going to create the same environment variables that we did before they were API key and API Secret so here we're going to put in our values of our secrets that we did before mine is just dummy data and we're going to save this file so now what happens well we're not quite done yet we need to bring in the dot Envy package into our code so we're going to go from dot in import mode and once I've added that package in we just need to run the function load underscore.if and save that all right that's actually it we have now created an environment variable file put our secrets in there loaded those environment variable files into our local memory and then we're using them so what happens when we run this project now well it spits out our secrets just as it did before and it's taken them directly from this file we can prove that I'll put a z in front of this let's save and run this project again and you'll see we have the Z in front of it now there is one problem here the dot EnV file that we've created is now incredibly sensitive and just like it's bad to have hard coded secrets in your code well it's especially bad to have this file end up somewhere like a git repository so we're going to add one more file in here that's really important we're going to create a file called dot git ignore there is a whole video I've made on exactly what this file is and how we can use it in various ways but essentially I'm going to put in here our DOT EnV file and save this all this means is that this is going to be excluded from any git command so if we do git egg all the EnV file won't be added into there so that's really important step that we take if we're using EnV files so there we have it that's how we handle our secrets as environment variables and pass them securely into our application to be able to use them but like anything in software engineering there's multiple ways to skin a cat I'm not sure if that's still an appropriate term to use but I'll stick with it there's lots of ways that we can do different things so let's come up with a scenario let's say that we have multiple environments we have our environment for development and testing and we have an environment for production and a number of other environments we want to be able to switch between the secrets that we're using for different environments and we can't effectively do this easily in this current method so we're going to change we're still going to use a EnV package we're not going to use our operating system environment so we can get rid of that I'm going to change this function to dot EnV values now the dot e and the values means that we can store Secrets as a dictionary they'll be converted into a dictionary and we can use them as such in our python package so here we're going to create a dictionary called secrets and these are going to equal Dot and underscore value and these are going to come from our EnV file all right easy now this one line actually gets rid of this whole section of applying secrets so we can get rid of all of that and we're also going to change a little bit of just how we print it so now we're going to print a section of our dictionary so we're going to go print secrets and then the name so API underscore key and we'll do the same for our secret API key all right so what happens now when we run this project well we get the exact same results so this is another way of handling Secrets now I said here that in this case we would have multiple environments so let's create another environment here I'm going to create a new environment variable file called dot env.dev so this is our development environment environment this is our development environment variable file I'm going to copy these across but we're going to just put Dash div at the end of both of them now back in our file if I want to load in a different environment I can change this run the code again and you'll see we get our development environment variable so this can be quite handy and just as a bonus exercise what I'll show you is how we can assign different environment variables based on the name of our file so we're going to create here we're going to wrap this in a function called Main now what we're going to do is we're going to write an if statement that say if our file name is main then I want you to run this function main so if equals then run the function Main so let's save this if we run this again it prints out our code if we were to rename our file div then it's not going to print out anything so this is just a silly example to show that hey if we have multiple different environments we can quickly use different EnV files to deal with them and using the dot EMV package we can handle these now even based on our username which is a little bit of a silly example but you get the idea and I'm sure as a software engineer you're going to be able to come up with much more creative ways of using this but that is the Crux of it so I hope you've enjoyed this video This is how we can use and manage secrets in Python using the dot EMV environment I'll be creating more tutorials on how to manage secrets in different environments including how to manage them from volts from kms's and from cloud providers so make sure you subscribe to the channel and ring that Bell so that you get notified we create content like this all the time and I do hope you'll consider joining the git Guardian family so thanks for watching and remember good code is secure code

Channel: GitGuardian

Views: 10,884

Keywords: python, secrets, API keys, secrets manager, passwords, dotenv, python dotenv, .env, storing secrets

Id: DVVYHlGYIHY

Length: 12min 46sec (766 seconds)

Published: Tue Mar 14 2023