Sources Contact Advanced Search Tutorials

An Interest In:

Web News this Week

Search Archive

Some of Our Sources

View All Sources

Help Webnuz

Referal links:

January 23, 2022 03:14 pm GMT

Multi environment AZURE deployments with Terraform and GitHub

Overview

This tutorial uses examples from the following GitHub demo project template repository.

I have been wanting to do a tutorial to demonstrate how to perform large scale terraform deployments in Azure using a non-monolithic approach. I have seen so many large deployments fall into this same trap of using one big monolithic configuration when doing deployments at scale. Throwing everything into one unwieldy configuration can be troublesome for many reasons. To name a few:

Making a small change can potentially break something much larger somewhere else in the configuration unintentionally.
Build time aka terraform plan/apply is increased. A tiny change can take a long time to run as the entire state is checked.
It can become cumbersome and complex for a team or team members to understand the entire code base.
Module and provider versioning and dependencies can be fairly confusing to debug in this paradigm and may become restrictive.
It becomes unmanageable, risky and time consuming to plan and implement any changes.

There's also many blogs and tutorials out there on how to integrate Terraform with DevOps CI/CD processes using Azure DevOps. So I decided to share with you today how to use Terraform with GitHub instead.

In this tutorial we will use GitHub reusable workflows and GitHub environments to build enterprise scale multi environment infrastructure deployments in Azure using a non-monolithic approach, to construct and simplify complex terraform deployments into simpler manageable work streams, that can be updated independently, increase build time, and reduce duplicate workflow code by utilizing reusable GitHub workflows.

Things you will get out of this tutorial:

Learn about GitHub reusable workflows.
Learn how to integrate terraform deployments with CI/CD using GitHub.
Learn how to deploy resources in AZURE at scale.
Learn about multi-stage deployments and approvals using GitHub Environments.

Hopefully you can even utilize these concepts in your own organization to build AZURE Infrastructure at scale in your own awesome cloud projects.

Pre-Requisites

To start things off we will build a few pre-requisites that is needed to integrate our GitHub project and workflows with AZURE before we can start building resources.

We are going to perform the following steps:

Create Azure Resources (Terraform Backend): (Optional) We will first create a few resources that will host our terraform backend state configuration. We will need a Resource Group, Storage Account and KeyVault. We will also create an Azure Active Directory App & Service Principal that will have access to our Terraform backend and subscription in Azure. We will link this Service Principal with our GitHub project and workflows later in the tutorial.
Create a GitHub Repository: We will create a GitHub project and set up the relevant secrets and environments that we will be using. The project will host our workflows and terraform configurations.
Create Terraform Modules (Modular): We will set up a few terraform ROOT modules. Separated and modular from each other (non-monolithic).
Create GitHub Workflows: After we have our repository and terraform ROOT modules configured we will create our reusable workflows and configure multi-stage deployments to run and deploy resources in Azure based on our terraform ROOT Modules.

1. Create Azure resources (Terraform Backend)

To set up the resources that will act as our Terraform backend, I wrote a PowerShell script using AZ CLI that will build and configure everything and store the relevant details/secrets we need to link our GitHub project in a key vault. You can find the script on my github code page called AZ-GH-TF-Pre-Reqs.ps1.

First we will log into Azure by running:

az login

After logging into Azure and selecting the subscription, we can run the script that will create all the pre-requirements we'll need:

## code/AZ-GH-TF-Pre-Reqs.ps1#Log into Azure#az login# Setup Variables.$randomInt = Get-Random -Maximum 9999$subscriptionId = (get-azcontext).Subscription.Id$resourceGroupName = "Demo-Terraform-Core-Backend-RG"$storageName = "tfcorebackendsa$randomInt"$kvName = "tf-core-backend-kv$randomInt"$appName="tf-core-github-SPN$randomInt"$region = "uksouth"# Create a resource resourceGroupNameaz group create --name "$resourceGroupName" --location "$region"# Create a Key Vaultaz keyvault create `    --name "$kvName" `    --resource-group "$resourceGroupName" `    --location "$region" `    --enable-rbac-authorization# Authorize the operation to create a few secrets - Signed in User (Key Vault Secrets Officer)az ad signed-in-user show --query objectId -o tsv | foreach-object {    az role assignment create `        --role "Key Vault Secrets Officer" `        --assignee "$_" `        --scope "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.KeyVault/vaults/$kvName"    }# Create an azure storage account - Terraform Backend Storage Accountaz storage account create `    --name "$storageName" `    --location "$region" `    --resource-group "$resourceGroupName" `    --sku "Standard_LRS" `    --kind "StorageV2" `    --https-only true `    --min-tls-version "TLS1_2"# Authorize the operation to create the container - Signed in User (Storage Blob Data Contributor Role)az ad signed-in-user show --query objectId -o tsv | foreach-object {    az role assignment create `        --role "Storage Blob Data Contributor" `        --assignee "$_" `        --scope "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Storage/storageAccounts/$storageName"    }#Create Upload container in storage account to store terraform state filesStart-Sleep -s 40az storage container create `    --account-name "$storageName" `    --name "tfstate" `    --auth-mode login# Create Terraform Service Principal and assign RBAC Role on Key Vault$spnJSON = az ad sp create-for-rbac --name $appName `    --role "Key Vault Secrets Officer" `    --scopes /subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.KeyVault/vaults/$kvName# Save new Terraform Service Principal details to key vault$spnObj = $spnJSON | ConvertFrom-Jsonforeach($object_properties in $spnObj.psobject.properties) {    If ($object_properties.Name -eq "appId") {        $null = az keyvault secret set --vault-name $kvName --name "ARM-CLIENT-ID" --value $object_properties.Value    }    If ($object_properties.Name -eq "password") {        $null = az keyvault secret set --vault-name $kvName --name "ARM-CLIENT-SECRET" --value $object_properties.Value    }    If ($object_properties.Name -eq "tenant") {        $null = az keyvault secret set --vault-name $kvName --name "ARM-TENANT-ID" --value $object_properties.Value    }}$null = az keyvault secret set --vault-name $kvName --name "ARM-SUBSCRIPTION-ID" --value $subscriptionId# Assign additional RBAC role to Terraform Service Principal Subscription as Contributor and access to backend storageaz ad sp list --display-name $appName --query [].appId -o tsv | ForEach-Object {    az role assignment create --assignee "$_" `        --role "Contributor" `        --subscription $subscriptionId    az role assignment create --assignee "$_" `        --role "Storage Blob Data Contributor" `        --scope "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Storage/storageAccounts/$storageName" `    }

Lets take a closer look, step-by-step what the above script does as part of setting up the Terraform backend environment.

Create a resource group called Demo-Terraform-Core-Backend-RG, containing an Azure key vault and storage account.
Create an AAD App and Service Principal that has access to the key vault, backend storage account, container and the subscription.
The AAD App and Service Principal details are saved inside the key vault.

2. Create a GitHub Repository

For this step I actually created a template repository that contains everything to get started. Feel free to create your repository from my template by selecting Use this template. (Optional)

After creating the GitHub repository there are a few things we do need to set on the repository before we can start using it.

Add the secrets that was created in the Key Vault step above, into the newly created GitHub repository as Repository Secrets

Create the following GitHub Environments. Or environments that matches your own requirements. In my case these are: Development, UserAcceptanceTesting, Production. Note that GitHub environments are available on public repos, but for private repos you will need GitHub Enterprise.

Also note that on my Production environment I have set a Required Reviewer. This will basically allow me to set explicit reviewers that have to physically approve deployments to the Production environment. To learn more about approvals see Environment Protection Rules.

NOTE: You can also configure GitHub Secrets at the Environment scope if you have separate Service Principals or even separate Subscriptions in Azure for each Environment. (Example: Your Development resources are in subscription A and your Production resources are in Subscription B). See Creating encrypted secrets for an environment for details.

3. Create Terraform Modules (Modular)

Now that our repository is all configured and ready to go, we can start to create some modular terraform configurations, or in other words separate independent deployment configurations based on ROOT terraform modules. If you look at the Demo Repository you will see that on the root of the repository I have paths/folders that are numbered e.g. ./01_Foundation and ./02_Storage.

These paths each contain a terraform ROOT module, which consists of a collection of items that can independently be configured and deployed. You do not have to use the same naming/numbering as I have chosen, but the idea is to understand that these paths/folders each represent a unique independent modular terraform configuration that consists of a collection of resources that we want to deploy independently.

So in this example:

path: ./01_Foundation contains the terraform ROOT module/configuration of an Azure Resource Group and key vault.
path: ./02_Storage contains the terraform ROOT module/configuration for one General-V2 and one Data Lake V2 Storage storage account.

NOTE: You will also notice that each ROOT module contains 3x separate TFVARS files: config-dev.tfvars, config-uat.tfvars and config-prod.tfvars. Each representing an environment. This is because each of my environments will use the same configuration: foundation_resources.tf, but may have slightly different configuration values or naming.

Example: The Development resource group name will be called Demo-Infra-Dev-Rg, whereas the Production resource group will be called Demo-Infra-Prod-Rg.

4. Create GitHub Workflows

Next we will create a special folder/path structure in the root of our repository called .github/workflows. This folder/path will contain our GitHub Action Workflows.

You will notice that there are numbered workflows: ./.github/workflows/01_Foundation.yml and ./.github/workflows/02_Storage.yml, these are caller workflows. Each caller workflow represents a terraform module and is named the same as the path containing the ROOT terraform module as described in the section above. There are also 2x GitHub Reusable Workflows called ./.github/workflows/az_tf_plan.yml and ./.github/workflows/az_tf_apply.yml.

Let's take a closer look at the reusable workflows:

az_tf_plan.yml:

This workflow is a reusable workflow to plan a terraform deployment, create an artifact and upload that artifact to workspace artifacts for consumption.

## code/az_tf_plan.yml### Reusable workflow to plan terraform deployment, create artifact and upload to workspace artifacts for consumption ###name: 'Build_TF_Plan'on:  workflow_call:    inputs:      path:        description: 'Specifies the path of the root terraform module.'        required: true        type: string      tf_version:        description: 'Specifies version of Terraform to use. e.g: 1.1.0 Default=latest.'        required: false        type: string        default: latest      az_resource_group:        description: 'Specifies the Azure Resource Group where the backend storage account is hosted.'        required: true        type: string      az_storage_acc:        description: 'Specifies the Azure Storage Account where the backend state is hosted.'        required: true        type: string      az_container_name:        description: 'Specifies the Azure Storage account container where backend Terraform state is hosted.'        required: true        type: string      tf_key:        description: 'Specifies the Terraform state file name for this plan.'        required: true        type: string      gh_environment:        description: 'Specifies the GitHub deployment environment.'        required: false        type: string        default: null      tf_vars_file:        description: 'Specifies the Terraform TFVARS file.'        required: true        type: string    secrets:      arm_client_id:        description: 'Specifies the Azure ARM CLIENT ID.'        required: true      arm_client_secret:        description: 'Specifies the Azure ARM CLIENT SECRET.'        required: true      arm_subscription_id:        description: 'Specifies the Azure ARM SUBSCRIPTION ID.'        required: true      arm_tenant_id:        description: 'Specifies the Azure ARM TENANT ID.'        required: truejobs:  build-plan:    runs-on: ubuntu-latest    environment: ${{ inputs.gh_environment }}    defaults:      run:        shell: bash        working-directory: ${{ inputs.path }}    env:      STORAGE_ACCOUNT: ${{ inputs.az_storage_acc }}      CONTAINER_NAME: ${{ inputs.az_container_name }}      RESOURCE_GROUP: ${{ inputs.az_resource_group }}      TF_KEY: ${{ inputs.tf_key }}.tfstate      TF_VARS: ${{ inputs.tf_vars_file }}      ###AZURE Client details###      ARM_CLIENT_ID: ${{ secrets.arm_client_id }}      ARM_CLIENT_SECRET: ${{ secrets.arm_client_secret }}      ARM_SUBSCRIPTION_ID: ${{ secrets.arm_subscription_id }}      ARM_TENANT_ID: ${{ secrets.arm_tenant_id }}    steps:      - name: Checkout        uses: actions/checkout@v2      - name: Setup Terraform        uses: hashicorp/[email protected]        with:          terraform_version: ${{ inputs.tf_version }}      - name: Terraform Format        id: fmt        run: terraform fmt --check      - name: Terraform Init        id: init        run: terraform init --backend-config="storage_account_name=$STORAGE_ACCOUNT" --backend-config="container_name=$CONTAINER_NAME" --backend-config="resource_group_name=$RESOURCE_GROUP" --backend-config="key=$TF_KEY"      - name: Terraform Validate        id: validate        run: terraform validate      - name: Terraform Plan        id: plan        run: terraform plan --var-file=$TF_VARS --out=plan.tfplan        continue-on-error: true      - name: Terraform Plan Status        if: steps.plan.outcome == 'failure'        run: exit 1      - name: Compress TF Plan artifact        run: zip -r ${{ inputs.tf_key }}.zip ./*      - name: Upload Artifact        uses: actions/upload-artifact@v2        with:          name: '${{ inputs.tf_key }}'          path: '${{ inputs.path }}/${{ inputs.tf_key }}.zip'          retention-days: 5

NOTE: The reusable workflow can only be triggered by another workflow, aka the caller workflows. We can see this by the on: trigger called workflow_call:.

## code/az_tf_plan.yml#L3-L4on:  workflow_call:

As you can see the reusable workflow can be given specific inputs when called by the caller workflow. Notice that one of the inputs are called path: which we can use to specify the path of the ROOT terraform module that we want to plan and deploy.

Inputs	Required	Description	Default
path	True	Specifies the path of the root terraform module.	-
tf_version	False	Specifies version of Terraform to use. e.g: 1.1.0 Default=latest.	latest
az_resource_group	True	Specifies the Azure Resource Group where the backend storage account is hosted.	-
az_storage_acc	True	Specifies the Azure Storage Account where the backend state is hosted.	-
az_container_name	True	Specifies the Azure Storage account container where backend Terraform state is hosted.	-
tf_key	True	Specifies the Terraform state file name for this plan.	-
gh_environment	False	Specifies the GitHub deployment environment.	null
tf_vars_file	True	Specifies the Terraform TFVARS file.	-

We aso need to pass some secrets from the caller to the reusable workflow. This is the details of our Service Principal we created to have access in Azure and is linked with our GitHub Repository Secrets we configured earlier.

Secret	Required	Description
arm_client_id	True	Specifies the Azure ARM CLIENT ID.
arm_client_secret	True	Specifies the Azure ARM CLIENT SECRET.
arm_subscription_id	True	Specifies the Azure ARM SUBSCRIPTION ID.
arm_tenant_id	True	Specifies the Azure ARM TENANT ID.

This workflow when called will perform the following steps:

Check out the code repository and set the path context given as input to the path containing the terraform module.
Install and use the version of terraform as per the input.
Format check the terraform module code.
Initialize the terraform module in the given path.
Validate the terraform module in the given path.
Create a terraform plan based on the given TFVARS file specified at input.
Compress the plan artifacts.
Upload the compressed plan as a workflow artifact.

Let's take a look at our second reusable workflow.

az_tf_apply.yml:

This workflow is a reusable workflow to download a terraform artifact built by az_tf_plan.yml and apply the artifact/plan (Deploy the planned terraform configuration).

## code/az_tf_apply.yml### Reusable workflow to download terraform artifact built by `az_tf_plan` and apply the artifact/plan ###name: 'Apply_TF_Plan'on:  workflow_call:    inputs:      path:        description: 'Specifies the path of the root terraform module.'        required: true        type: string      tf_version:        description: 'Specifies version of Terraform to use. e.g: 1.1.0 Default=latest.'        required: false        type: string        default: latest      az_resource_group:        description: 'Specifies the Azure Resource Group where the backend storage account is hosted.'        required: true        type: string      az_storage_acc:        description: 'Specifies the Azure Storage Account where the backend state is hosted.'        required: true        type: string      az_container_name:        description: 'Specifies the Azure Storage account container where backend Terraform state is hosted.'        required: true        type: string      tf_key:        description: 'Specifies the Terraform state file name for this plan.'        required: true        type: string      gh_environment:        description: 'Specifies the GitHub deployment environment.'        required: false        type: string        default: null      tf_vars_file:        description: 'Specifies the Terraform TFVARS file.'        required: true        type: string    secrets:      arm_client_id:        description: 'Specifies the Azure ARM CLIENT ID.'        required: true      arm_client_secret:        description: 'Specifies the Azure ARM CLIENT SECRET.'        required: true      arm_subscription_id:        description: 'Specifies the Azure ARM SUBSCRIPTION ID.'        required: true      arm_tenant_id:        description: 'Specifies the Azure ARM TENANT ID.'        required: truejobs:  apply-plan:    runs-on: ubuntu-latest    environment: ${{ inputs.gh_environment }}    defaults:      run:        shell: bash        working-directory: ${{ inputs.path }}    env:      STORAGE_ACCOUNT: ${{ inputs.az_storage_acc }}      CONTAINER_NAME: ${{ inputs.az_container_name }}      RESOURCE_GROUP: ${{ inputs.az_resource_group }}      TF_KEY: ${{ inputs.tf_key }}.tfstate      TF_VARS: ${{ inputs.tf_vars_file }}      ###AZURE Client details###      ARM_CLIENT_ID: ${{ secrets.arm_client_id }}      ARM_CLIENT_SECRET: ${{ secrets.arm_client_secret }}      ARM_SUBSCRIPTION_ID: ${{ secrets.arm_subscription_id }}      ARM_TENANT_ID: ${{ secrets.arm_tenant_id }}    steps:      - name: Download Artifact        uses: actions/download-artifact@v2        with:          name: ${{ inputs.tf_key }}          path: ${{ inputs.path }}      - name: Decompress TF Plan artifact        run: unzip ${{ inputs.tf_key }}.zip      - name: Setup Terraform        uses: hashicorp/[email protected]        with:          terraform_version: ${{ inputs.tf_version }}      - name: Terraform Init        id: init        run: terraform init --backend-config="storage_account_name=$STORAGE_ACCOUNT" --backend-config="container_name=$CONTAINER_NAME" --backend-config="resource_group_name=$RESOURCE_GROUP" --backend-config="key=$TF_KEY"      - name: Terraform Apply        run: terraform apply --var-file=$TF_VARS --auto-approve

The inputs and secrets are the same as our previous reusable workflow which created the terraform plan.

Inputs	Required	Description	Default
path	True	Specifies the path of the root terraform module.	-
tf_version	False	Specifies version of Terraform to use. e.g: 1.1.0 Default=latest.	latest
az_resource_group	True	Specifies the Azure Resource Group where the backend storage account is hosted.	-
az_storage_acc	True	Specifies the Azure Storage Account where the backend state is hosted.	-
az_container_name	True	Specifies the Azure Storage account container where backend Terraform state is hosted.	-
tf_key	True	Specifies the Terraform state file name for this plan.	-
gh_environment	False	Specifies the GitHub deployment environment.	null
tf_vars_file	True	Specifies the Terraform TFVARS file.	-

Secret	Required	Description
arm_client_id	True	Specifies the Azure ARM CLIENT ID.
arm_client_secret	True	Specifies the Azure ARM CLIENT SECRET.
arm_subscription_id	True	Specifies the Azure ARM SUBSCRIPTION ID.
arm_tenant_id	True	Specifies the Azure ARM TENANT ID.

This workflow when called will perform the following steps:

Download the terraform plan (workflow artifact).
Decompress the terraform plan (workflow artifact).
Install and use the version of terraform as per the input.
Re-initialize the terraform module.
Apply the terraform configuration based on the terraform plan and values in the TFVARS file.

Let's take a look at one of the caller workflows next. These workflows will be used to call the reusable workflows.

01_Foundation.yml:

This workflow is a Caller workflow. It will call and trigger a reusable workflow az_tf_plan.yml and create a foundational terraform deployment PLAN based on the repository path: ./01_Foundation containing the terraform ROOT module/configuration of an Azure Resource Group and key vault. The plan artifacts are validated, compressed and uploaded into the workflow artifacts, the caller workflow 01_Foundation will then call and trigger the second reusable workflow az_tf_apply.yml that will download and decompress the PLAN artifact and trigger the deployment based on the plan. (Also demonstrated is how to use GitHub Environments to do multi staged environment based deployments with approvals - Optional)

## code/01_Foundation.ymlname: '01_Foundation'on:  workflow_dispatch:  pull_request:    branches:      - masterjobs:  Plan_Dev:    #if: github.ref == 'refs/heads/master' && github.event_name == 'pull_request'    uses: Pwd9000-ML/Azure-Terraform-Deployments/.github/workflows/az_tf_plan.yml@master    with:      path: 01_Foundation ## Path to terraform root module (Required)      tf_version: latest ## Terraform version e.g: 1.1.0 Default=latest (Optional)      az_resource_group: your-resource-group-name ## AZ backend - AZURE Resource Group hosting terraform backend storage acc (Required)      az_storage_acc: your-storage-account-name ## AZ backend - AZURE terraform backend storage acc (Required)      az_container_name: your-sa-container-name ## AZ backend - AZURE storage container hosting state files (Required)      tf_key: foundation-dev ## AZ backend - Specifies name that will be given to terraform state file (Required)      tf_vars_file: config-dev.tfvars ## Terraform TFVARS (Required)    secrets:      arm_client_id: ${{ secrets.ARM_CLIENT_ID }} ## ARM Client ID      arm_client_secret: ${{ secrets.ARM_CLIENT_SECRET }} ## ARM Client Secret      arm_subscription_id: ${{ secrets.ARM_SUBSCRIPTION_ID }} ## ARM Subscription ID      arm_tenant_id: ${{ secrets.ARM_TENANT_ID }} ## ARM Tenant ID  Deploy_Dev:    needs: Plan_Dev    uses: Pwd9000-ML/Azure-Terraform-Deployments/.github/workflows/az_tf_apply.yml@master    with:      path: 01_Foundation ## Path to terraform root module (Required)      tf_version: latest ## Terraform version e.g: 1.1.0 Default=latest (Optional)      az_resource_group: your-resource-group-name ## AZ backend - AZURE Resource Group hosting terraform backend storage acc (Required)      az_storage_acc: your-storage-account-name ## AZ backend - AZURE terraform backend storage acc (Required)      az_container_name: your-sa-container-name ## AZ backend - AZURE storage container hosting state files (Required)      tf_key: foundation-dev ## AZ backend - Specifies name that will be given to terraform state file (Required)      gh_environment: Development ## GH Environment. Default=null - (Optional)      tf_vars_file: config-dev.tfvars ## Terraform TFVARS (Required)    secrets:      arm_client_id: ${{ secrets.ARM_CLIENT_ID }} ## ARM Client ID      arm_client_secret: ${{ secrets.ARM_CLIENT_SECRET }} ## ARM Client Secret      arm_subscription_id: ${{ secrets.ARM_SUBSCRIPTION_ID }} ## ARM Subscription ID      arm_tenant_id: ${{ secrets.ARM_TENANT_ID }} ## ARM Tenant ID  Plan_Uat:    #if: github.ref == 'refs/heads/master' && github.event_name == 'pull_request'    uses: Pwd9000-ML/Azure-Terraform-Deployments/.github/workflows/az_tf_plan.yml@master    with:      path: 01_Foundation      az_resource_group: your-resource-group-name      az_storage_acc: your-storage-account-name      az_container_name: your-sa-container-name      tf_key: foundation-uat      tf_vars_file: config-uat.tfvars    secrets:      arm_client_id: ${{ secrets.ARM_CLIENT_ID }}      arm_client_secret: ${{ secrets.ARM_CLIENT_SECRET }}      arm_subscription_id: ${{ secrets.ARM_SUBSCRIPTION_ID }}      arm_tenant_id: ${{ secrets.ARM_TENANT_ID }}  Deploy_Uat:    needs: [Plan_Uat, Deploy_Dev]    uses: Pwd9000-ML/Azure-Terraform-Deployments/.github/workflows/az_tf_apply.yml@master    with:      path: 01_Foundation      az_resource_group: your-resource-group-name      az_storage_acc: your-storage-account-name      az_container_name: your-sa-container-name      tf_key: foundation-uat      gh_environment: UserAcceptanceTesting      tf_vars_file: config-uat.tfvars    secrets:      arm_client_id: ${{ secrets.ARM_CLIENT_ID }}      arm_client_secret: ${{ secrets.ARM_CLIENT_SECRET }}      arm_subscription_id: ${{ secrets.ARM_SUBSCRIPTION_ID }}      arm_tenant_id: ${{ secrets.ARM_TENANT_ID }}  Plan_Prod:    #if: github.ref == 'refs/heads/master' && github.event_name == 'pull_request'    uses: Pwd9000-ML/Azure-Terraform-Deployments/.github/workflows/az_tf_plan.yml@master    with:      path: 01_Foundation      tf_version: latest      az_resource_group: your-resource-group-name      az_storage_acc: your-storage-account-name      az_container_name: your-sa-container-name      tf_key: foundation-prod      tf_vars_file: config-prod.tfvars    secrets:      arm_client_id: ${{ secrets.ARM_CLIENT_ID }}      arm_client_secret: ${{ secrets.ARM_CLIENT_SECRET }}      arm_subscription_id: ${{ secrets.ARM_SUBSCRIPTION_ID }}      arm_tenant_id: ${{ secrets.ARM_TENANT_ID }}  Deploy_Prod:    needs: [Plan_Prod, Deploy_Uat]    uses: Pwd9000-ML/Azure-Terraform-Deployments/.github/workflows/az_tf_apply.yml@master    with:      path: 01_Foundation      az_resource_group: your-resource-group-name      az_storage_acc: your-storage-account-name      az_container_name: your-sa-container-name      tf_key: foundation-prod      gh_environment: Production      tf_vars_file: config-prod.tfvars    secrets:      arm_client_id: ${{ secrets.ARM_CLIENT_ID }}      arm_client_secret: ${{ secrets.ARM_CLIENT_SECRET }}      arm_subscription_id: ${{ secrets.ARM_SUBSCRIPTION_ID }}      arm_tenant_id: ${{ secrets.ARM_TENANT_ID }}

Notice that we have multiple jobs: in the caller workflow, one job to generate a terraform plan and one job to deploy the plan, per environment.

You will see that each plan job uses the different TFVARS files: config-dev.tfvars, config-uat.tfvars and config-prod.tfvars respectively of each environment, but using the same ROOT module configuration in the path: ./01_Foundation/foundation_resources.tf.

Each reusable workflows inputs are specified on the caller workflows jobs: using with:, and Secrets using secret:.

You will also note that only the Deploy jobs: Deploy_Dev:, Deploy_Uat:, Deploy_Prod:, are linked with an input gh_environment which specifies which GitHub environment the job is linked to. Each Plan jobs: Plan_Dev:, Plan_Uat:, Plan_Prod:, are not linked to any GitHub Environment.

Each Deploy jobs: Deploy_Dev:, Deploy_Uat:, Deploy_Prod: are also linked with the relevant needs: setting of it's corresponding plan. This means that the plan job must be successful before the deploy job can initialize and run. Deploy jobs are also linked with earlier deploy jobs using needs: so that Dev gets built first and if successful be followed by Uat, and if successful followed by Prod. However if you remember, we configured a GitHub Protection Rule on our Production environment which needs to be approved before it can run.

NOTE: if you have been following this tutorial step by step, and used a cloned copy of the Demo Repository you will need to update the caller workflows: ./.github/workflows/01_Foundation.yml and ./.github/workflows/02_Storage.yml with the inputs specified under with: using the values of your environment.

Testing

Let's run the workflow: 01_Foundation and see what happens.

After the run you will see that each plan was created and DEV as well as UAT terraform configurations have been deployed to Azure as per the terraform configuration under path: ./01_Foundation:

After approving Production we can see that approval has triggered the production deployment and now we also have a production resource group.

You will notice that each resource group contains a key vault as per our foundation terraform configuration under path: ./01_Foundation.

Let's run the workflow: 02_Storage and after deploying DEV and UAT, also approve PRODUCTION to run.

Now you will notice that each of our environments resource groups also contains storage accounts as per the terraform configuration under path: ./02_Storage.

Lastly, if we navigate to the terraform backend storage account, you will see that based on the tf_key inputs we gave each of our caller workflow jobs:, each terraform deployment has its own state file per ROOT module/collection, per environment, which nicely segregates the terraform configuration state files independently from each other.

I hope you have enjoyed this post and have learned something new. You can find the code samples used in this blog post on my Github page. You can also look at the demo project or even create your own projects and workflows from the demo project template repository.

Author

Like, share, follow me on: GitHub | Twitter | LinkedIn

.ltag__user__id__620034 .follow-action-button { background-color: #0cbb58 !important; color: #000000 !important; border-color: #0cbb58 !important; }

Marcel.LFollow

Cloud Solutions & DevOps Architect.

Original Link: https://dev.to/pwd9000/multi-environment-azure-deployments-with-terraform-and-github-2450

Share this article:

View Full Article

Dev To

An online community for sharing and discovering great ideas, having debates, and making friends

More About this Source Visit Dev To