Terraform State

Terraform blocks and modules are the things you use to build your infrastructure, but state is how you manage it after deployment.

A Terraform state is a record of the managed infrastructure. Each time you make an execution plan and attempt to create or modify resource, Terraform will compare your configuration to the existing state.

Local backend

The location of the state file is specified in Terraform backend. By default the state is stored locally, in the working directory inside the terraform.tfstate file. Data is stored in the JSON format and is relatively human-readable. Please note that you should not modify this file. It is extremely rare for anyone to modify it and even in those instances you should know exactly what you are doing.

Backend settings are often specified in the terraform.tf file inside the terraform block. Not specifying a backend is equivalent to this code:

terraform {  backend "local" {    path = "terraform.tfstate"  }  # other settings...}

Viewing state

We can see what the current state contains by running terraform show command. Please note that if do not have any running resources the state file will be empty. You can run terraform apply to deploy resources and they will be visible in the show output.

$ terraform show

This command will output a lot of data and sometimes we are just interested in a more compressed view with just a list of resources and their names.
This can be view with the state list command:

$ terraform state list

Terraform state locking

We have touched a little bit on how critical the state is to Terraform and its operation. At some point you might want to have other people to work or manage your Terraform configuration. This is when you want to implement controls to lock the state to prevent accidental updates or changes to the state.

State locking does exactly that. When you have several people working on existing infrastructure you want control when and how changes to it are made. Locking will prevent corruption and it is something that everyone working with Terraform should know about.

To demonstrate Terraform state lock I will first apply my configuration with terraform apply but I will not enter yes to confirm the apply. The deployment will be paused. At this stage the state will be locked.

I will open another terminal window and run apply command again I get this output:

$ terraform apply Error: Error acquiring the state lock  Error message: resource temporarily unavailable Lock Info:   ID:        3853fd24-37f7-06be-f354-a6bb5bd4270b   Path:      terraform.tfstate   Operation: OperationTypeApply   Who:       [email protected]   Version:   1.1.9   Created:   2022-06-22 01:03:41.259138 +0000 UTC   Info:         Terraform acquires a state lock to protect the state from being written by multiple users at the same time. Please resolve the issue above and try again. For most commands, you can disable locking with the "-lock=false" flag, but this is not recommended.

As we can see Terraform encountered an error acquiring the state lock and realizes that there are 2 concurrent apply commands.

If there are several people working on a configuration and pushing changes, it helps to set the -lock-timeout=100s which sets the timeout of 100 seconds to the apply command and waits for the lock to be released before applying changes. Here is an example with 300 seconds (5min):

$ terraform apply -lock-timeout=300s

Not all Terraform backends support, but some do. Here is a list of the backends that support state locking:

• Local backend
• Remote backend (Terraform Enterprise, Terraform Cloud)
• AWS S3 backend (with DynamoDB)
• Google Cloud Storage backend
• Azure Storage backend

Terraform state backend authentication

Terraform state may contain a lot of sensitive data and protecting the state from unauthorized access is extremely important. Storing the state remotely allows team members to access it too, but it also exposes it to access from other people. Supported backends handle authentication differently.

Terraform S3 standard backend with environment variable authentication

This backend configuration allows us to store the state file inside an S3 bucket.

To setup this backend you would need to create an S3 bucket, make sure that Block ALL PUBLIC ACCESS checkmark is selected because we are trying to protect our state.

Then we need update the backend inside the terraform.tf file:

# terraform.tfterraform {  backend "s3" {    bucket = "my-demo-terraform-state-bucket"    key    = "dev/s3_terraform_state"    region = "ca-central-1"  }  # more settings below...}

key specifies where the state file will be stored inside the bucket.

We already imported AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY in installation and usage section this should work. If you need a refresher this is how it's done:

$ export AWS_ACCESS_KEY="my_key"$ export AWS_SECRET_ACCESS_KEY="my_secret_key"

Since the backend was changed we need to run terraform init command to initialize the backend.

In this exercise we covered two things: setting up an S3 bucket state location and reviewed using environment variables for authentication.

Terraform Remote Enhanced Backed with user token authentication

This is a service provided by Terraform. It allows users to store state files inside Terraform Cloud. In order to use it we need to create a user token and import it into our local environment.

To follow the example in this section you would need a Terraform account on Terraform Cloud. Once logged in create a new organization by providing an organization name and an email.

Once you are done creating an organization, let's go back to our terminal and punch in this command:

$ terraform login# outputTerraform will request an API token for app.terraform.io using your browser.If login is successful, Terraform will store the token in plain text inthe following file for use by subsequent commands:    /Users/andreyf/.terraform.d/credentials.tfrc.jsonDo you want to proceed?  Only 'yes' will be accepted to confirm.  Enter a value: 

Enter yes for the first prompt. It will open a browser window and print this output:

---------------------------------------------------------------------------------Terraform must now open a web browser to the tokens page for app.terraform.io.If a browser does not open this automatically, open the following URL to proceed:    https://app.terraform.io/app/settings/tokens?source=terraform-login---------------------------------------------------------------------------------Generate a token using your browser, and copy-paste it into this prompt.Terraform will store the token in plain text in the following filefor use by subsequent commands:    /Users/andreyf/.terraform.d/credentials.tfrc.jsonToken for app.terraform.io:  Enter a value: 

I named my token terraform login credentials:

Click Create API token and you will get the token.

Now we need to paste this token in our terminal to complete the setup process:

This is the output I got after I added the token:

----------------Generate a token using your browser, and copy-paste it into this prompt.Terraform will store the token in plain text in the following filefor use by subsequent commands:    /Users/andreyf/.terraform.d/credentials.tfrc.jsonToken for app.terraform.io:  Enter a value: Retrieved token for user andreyfrol---------------------------------------------------------------------------------                                          -                                                                          -----                           -                                          ---------                      --                                          ---------  -                -----                                           ---------  ------        -------                                             -------  ---------  ----------                                                ----  ---------- ----------                                                  --  ---------- ----------   Welcome to Terraform Cloud!                     -  ---------- -------                                                      ---  ----- ---   Documentation: terraform.io/docs/cloud             --------   -                                                      ----------                                                      ----------                                                       ---------                                                           -----                                                               -   New to TFC? Follow these steps to instantly apply an example configuration:   $ git clone https://github.com/hashicorp/tfc-getting-started.git   $ cd tfc-getting-started   $ scripts/setup.sh

I think this is a good point to wrap up this article. Terraform state is a big topic and it will likely require more than one article to cover the basics.

Thank you for reading and see you in the next article!