Encrypting State With OpenTofu

are you wondering how to use open tofu to encrypt your state data well this is the video for [Music] [Music] you what's up everybody I'm Ned bellavance Ned inthe cloud.com and for today's video we're going to return once again to open tofu if you're not familiar with open tofu and how it differs from terraform then you may want to check out my previous video that should appear somewhere around now in this video I want to dig into the data encryption feature that was introduced with open tofu 1.7 we'll talk a little bit about the features history what problem it's attempting to solve and how to add or remove encryption to your state data and plan files if all you care about is the actual nuts and bolts jump to the timestamp that appears now or to the chapter called encrypting your state data for everyone else let's do a little background before open tofu launched one of the most popular requests on the terraform list of open issues was to support native encryption of State data and plan files at rest and the issue languished for years despite some possible solutions being put forth hashicorp as maintainers of the terraform repository argued that using terraform to encrypt State data directly was not a great idea in practice and plan files should be ephemeral and so they chose to leave the issue open rather than accept some of the PRS that had come in one of the big Promises of open tofu was that this feature would finally be added and the issue would be resolved and in open tofu 1.7 it was that's quite an accomplishment and I have to give the open tofu team their flowers for delivering on it I also feel that I should mention that this feature is not compatible with terraform and you would have to remove encryption if you wanted to migrate to terraform or have a third-party tool interact with your state data or plan files now why were people asking for this feature mostly because State data and plan files contain sensitive information that you don't want others to have access to that could be obvious stuff like API Keys passwords tokens but it could also be the less obvious fact that anyone who gets a copy of your state data or a plan file has a map of your deployed infrastructure they might be able to use that to develop a plan of attack or find weaknesses to be exploited having open tofu encrypt State data and plan files at rest means that even if someone gains access to wherever your state data is stored they still won't be able to read it without the necessary encryption keys in Hashi Corps defense there are lots of ways to encrypt State data at rest that do not rely on managing your own encryption keys and doing it natively with terraform the most popular state backends all support encryption of State at rest and in transit and they allow you to apply access permissions Azure storage for instance encrypts data by default and you can bring your own key for both data encryption and infrastructure encryption if you want to know how much you can truly lock down Azure storage check out my video on the topic so aside from adding another layer of complexity and more encryption keys to manage what does this additional encryption facility buy you well if you don't trust the backend you're using to properly store and protect your state data then encrypting state natively with open tofu removes the onus from the platform and place es it squarely on you especially if you're using some generic HTTP backend you may want to manage the encryption yourself it also provides a way for you to encrypt your state data if it's being stored locally now I know we should all be storing our state on a remote backend but that's not always the reality you could use another encryption solution to manage encryption at rest on your local file system but open tofu makes it so you can rely on the encryption provided natively by open tofu itself more importantly I think is the option to encrypt plan files when you generate an execution plan and save it to a file for later review it contains a ton of important information like plan changes new values and updated outputs this is information you might not want just anyone to have access to when you're setting up your terraform automation you could choose to stash that plan file in a location that supports encryption at rest and has permissions applied to it but that's not part of the state backend configuration that's something you need to bake yourself or get from an existing terraform automation platform I really think that encrypting plan files is the big win here and encrypting State data is kind of secondary with that being said let's see how it works first let's lay down some terminology and syntax the encryption settings for state and plan data can be configured using an encryption block inside of the terraform block or passed through the environment variable TF encryption or you can use a combination of both you can also use Json or HCL to describe the encryption settings though I think you already know my general feelings on Json encryption requires two components a key provider and a method the key provider defines where open tofu is getting the key material to perform the encryption and the method describes how the key is used to encrypt the data the method used to apply encryption to State data and plan files is managed separately so you can choose to use one form of encryption for state and another one for plan or you can choose to apply encryption to one and not the other so what kind of encryption key providers are are available right now as of recording there are four pbkdf2 I don't know what all that stands for but that is used to generate a key locally using a passphrase it doesn't rely on an external service of any kind the passphrase is stored with the configuration or can be injected using that TF encryption environment variable then there's the AWS KMS which unsurprisingly uses the AWS KMS service for the encryption key there's also gcp KMS which uses the Google Cloud KMS service for the encryption key or you can use open B which uses the transit engine in open B for management of the encryption key and if you're wondering what what the hell is open bow it's the open source Fork of Hashi Corp Vault the open bow key provider will also work with Vault versions that are under the MPL license which should be 1.14 or older Azure key vault is not yet on the list but I'm sure that's being worked on right now plus this whole thing is open source so if your preferred encryption service isn't on the list you can implement it yourself there are only two methods available for encryption right now and really there's only one that actually encrypts things AES GCM uses aes-gcm which is an industry standard encryption process that uses symmetric keys to encrypt and decrypt data the other method is called unencrypted and it's used for migration scenarios which we'll see in the later demo in fact let's put this all into practice with a simple example here's my terraform Tuesday repository and I have a folder for this demo called basic example let's take a look at the terraform ttf file in here I've got the encryption block nested in side of the terraform block to define a key provider I have the key provider block type which takes two labels the first is the key provider type which I have set to pbkdf2 and the second is the name label which I've used passphrase for inside of the block I have some arguments that set the passphrase the key length and some other options many of these arguments have default values which you can go with instead of adding them explicitly once I have my key provider I need a method block to Define which encryption method I'm using and which key to use for it the method block also has two labels the method type which in my case is aore GCM and a name label which I'm using passphrase GCM inside the block the only argument is the keys argument which I set to the key provider block identifier now to use this method in my configuration I can specify a state and or a plan block neither of these have any block labels inside of the block there is an argument method which is set to the method identifier I have both state and plan set here so any state data or plan files I create should be encrypted using this method let's give it a try I haven't defined a state backend in my configuration so we'll be using the local file backend in my main.tf I'm simply creating a local file and an output that's set to the file name we'll start by running tofu in it like you normally would once that finishes I'm going to run tofu plan- out plan. out which writes the execution plan to a file once that finishes let's take a look at my plan file now usually a plan file is an unreadable binary format but in this case we have a Json file that specifies some encryption information and has the encrypted data payload which would be that binary data can I still view the files contents yeah sure I can run tofu show. out and that will give me back the unencrypted contents of the plan so if I want to evaluate the plan information I'm going to have to do so through open tofu let's apply this plan by running tofu apply plan. out and once everything is created I now have a terraform dotf State file looking at its contents it's very similar to the plan file there's a serial number and a lineage entry and those are still in plain text so open tofu can do a comparison with a stored plan without decrypting the state data beyond that is the encryption information and then the state data payload I can still interact with state by using the tofu command but we cannot view the state data directly so that's a basic example but it relies on setting a passphrase and I don't feel great about that especially in automation setting let's take a look at an example that uses AWS KS and has existing state data that I want to encrypt all right this is more like it this is similar to a situation that you might already have in place I have an already deployed configuration and I want want to add encryption on top of it and I'm using S3 as my backend for State data storage how do I migrate to encrypted State the configuration here in use KMS S3 has the same resources as the basic example but if we look at the terraform dotf file it's using S3 as a state backend if I jump over to the AWS console I have a terraform ttf State object in an S3 bucket and if if I click to view its contents it's unencrypted Json of my state data okay let's get that encrypted I'm going to use the AWS KMS key provider for encryption so I've already provisioned an AWS KMS key and grab the key ID if you want to try this out for yourself the code for creating the S3 bucket for State data and the AWS KMS key for encryption are in the generate KMS S3 directory the KMS key is set for encrypt decrypt actions and the key rotation is also enabled the key type is symmetric default which is the default value if you don't specify it that's what's required for use with tofu encryption as of right now you cannot select an RSA or ECC key type because those are asymmetric meaning that the private key never leaves AWS KMS open tofu would have to send the encrypted data up to AWS KMS and have it perform a decrypt operation and then get the unencrypted data back and that's not how State data encryption works at the moment so I have this key and now I want to implement it but I might not want to hardcode the KMS key ID the region and all the other information directly into my configuration so I can use the TF encryption environment variable instead in the file setor env. PS1 I'm using her do syntax to set the TF encryption environment variable to HCL that contains my encryption settings let's take a look at what those settings are it includes the key provider of type AWS KMS with the key ID region and key speec the method is AES GCM and the state and plan blocks are both using that method for encryption but what's this other method unencrypted this method is specifically for when you want to set up or remove encryption unencrypted simply means that this method doesn't encrypt data looking in the state data block there is a fallback block that uses the unencrypted method when we run a tofu plan it wants to use the AES GCM method but the existing state data doesn't match that method it's still in plain text so open tofu falls back to the fallback method which is unencrypted when it writes the updated State data back out during an apply it will use the AES GCM method and our data will now be encrypted let's give it a try I'll run the set EnV PS1 script to set up my TF encryption environment variable then I'll make a change to main.tf changing the name of the file so there's actually something to apply then I'll run tofu apply and approve the plan that it produces once the apply is complete let's jump back to S3 and take a look at the contents of the S3 bucket once I open up the terraform dotf State object it is now encrypted using awms and this shows my migration to encrypted State data was successful if I want to go back to an unencrypted format for some reason I would swap the method and the fallback method so tofu can decrypt the state data using the fallback method and then write it back out in clear text using the unencrypted method it's no big deal State data is like really really important it's how terraform knows what resources it's managing if you lose access to your state data because you lost the encryption key you've got a long day or days ahead of you sure you can use import blocks and Discovery tools to make it a little easier but it's going to make for a bad time especially if you happen to use the same encryption key for multiple State data instances you lose that key you lose access to all of those State data instances as a protection against that AWS KMS is pretty cheap and so are all the other options so probably just use a different key for each configuration at least then if you lose the key you don't lose access to all of the state data instances unfortunately that makes even more for you to juggle and if someone gains access to your AWS KMS keys they could really wreck your day second although you're protecting your state data and plan files from an attacker who somehow gained access to wherever they're stored you aren't protecting your state data from people who have to work with that information on a regular basis that person still needs to have access to State data and they can easily make an unencrypted copy of State data or a plan file using the tofu State pull or tofu show commands I think you really need to ask yourself if adding this level of encryption is actually worthwhile if you've secured your state backend properly unauthorized people shouldn't be able to get to your state data anyway whether you're using an automation platform like gitlab or m0o or self-managing your state with S3 or Azure storage you have the ability right now to lock down access and encrypt that data at rest and in transit adding another layer of encryption might buy you a sliver of additional security but is that worth the additional cost of managing keys and potentially losing access to State data I can't answer that for you and I'm not trying to that's something you'll need to decide with your security and risk [Music] teams my personal feeling is that native encryption of state and plan files is not a huge Boon to security but it is nice to have the option and it's a feather in the cap of open tofu that they can offer this feature for those who have been waiting forever to see it implemented I'm curious to hear what you think will you be using this feature to encrypt your state or your plan why or why not leave me a comment down below and let me know that's going to do it for today thanks as always for watching if you like what I'm doing here and you want some more check out my terraform associate certification guide over on lean Pub the link is down in the description till next time keep calm and open tofu on bye everyone [Music] he [Music]

Channel: Ned in the Cloud

Views: 447

Keywords: opentofu encryption, opentofu demo

Id: e7Jh6RZB9W8

Length: 19min 36sec (1176 seconds)

Published: Wed Jun 26 2024