Introduction

As many of you will already know, I recently started an exciting new role as Head of Technology at FirstPort, having previously been at a great consultancy called DevOpsGroup.

A key part of my role is delivering FirstPorts vision of People First technology. To do this, it is imperative that I select the right technology to underpin the delivery of services that help make customers lives easier.

Today I want to talk about my selection of GitHub Enterprise

GitHub is a best-in-breed, secure, collaborative, code hosting platform with built-in CI/CD capabilities with over 56 million users.

Many organisations use it, and all of them have to manage users, repositories, branch rules, and much more. Standardising all of this is quite challenging and involves creating things like CLI tools, scripts, and other ways to try to automate how people deal with GitHub.

Here at FirstPort, I decided to use Terraform to manage and standardise all of the repository and user management, and today I want to share with you how we are doing this!

Team and user management

If you are using GitHub enterprise, you will probably use Single Sign-on (SSO). This does the job but isnt great, especially when having to manage collaborators outside of your Identity Provider (However, you can use Terraform to manage sync groups) - This still leaves you to manage repositories, labels, pull requests all manually. In our case, I decided to use Terraform's GitHub provider to automate most of this work.

Firstly, I created the teams:

# Creates a parent teamresource "github_team" "engineering" {  name        = "engineering"  description = "This is a parent team"  privacy     = "closed"}# Creates a sub team of front-end developersresource "github_team" "frontend" {  name           = "frontend"  description    = "This is a sub team "  parent_team_id = github_team.engineering.id  privacy        = "closed"}

Every time we have someone new join, anyone at FirstPort can open a PR and invite them to the team.

# Invites a user to the organizationresource "github_membership" "anewstarter" {  username = "firstportuser"  role     = "member"}# Adds the user to a teamresource "github_team_membership" "frontend-firstportuser1" {  username = github_membership.firstportuser.username  team_id  = github_team.frontend.id  role     = "maintainer"}

This will run in a GitHub Actions workflow, and the user will receive an invite to join our organisation.

Repository creation

This is probably the main reason why I decided to use Terraform for managing GitHub. If repositories aren't standardised this can create a chaotic situation if the way engineers open issues, raise PRs etc is different in every repository. To avoid this I decided to create most of our repositories in the same way.

A few things were very important to us that every repository had to have. Let's go through the mains ones:

Release notes

I wanted to provide a way to be able to easily create release notes, to enhance transparency on what we are delivering. I chose to use a tool called Release Drafter in our workflow. I wanted to configure how labels would be used in every repository, and for that, we standardised all labels across all repositories. I use: docs, dependencies, bug, feature, and maintenance.

Every time someone opens a PR, they need to select at least one of these labels to define how this PR is categorised (This is enforced using the Enforce Label Action). When a PR is merged, I use the GitHub Action release drafter, which creates a draft release with the commits from the PR into one of those categories. This helps us to create release notes for all of our services!

This is how the release notes would look like:

Every time someone creates a new repo, the release drafter comes pre-configured and you can start enjoying good release notes without any effort.

Branch protection

Setting up rules for when you can merge a branch is very important for code quality, and in our case for legal reasons (at FirstPort, we deal with personal user data, so all changes must go through strict code quality checks). GitHub gives the possibility for you to configure branch protection rules to solve that.

I wanted to define the same set of basic branch protection for all repositories. This was easily solved with our Terraform module.

Terraform module

To achieve all of this, I created a Terraform module that manages users, teams, and repositories. Here is an example on how we use the module:

module "my-firstport-repo" {  source = "./modules/repo"  version = "~> 1.0"  name                   = "my-firstport-repo"  description            = "A firstport repository description"  visibility             = "private"  delete_branch_on_merge = true  vulnerability_alerts   = true  teams                  = { (data.github_team.engineering.id) = "push" }  collaborators          = { (data.github_user.external.username) = "push" }  labels = {    "type: bug"          = "d73a4a"    "type: docs"         = "0f727f"    "type: feature"      = "a2eeef"    "type: maintenance"  = "a5f7da"    "type: dependencies" = "0366d6"  }  branch_protection = {    master = {      enforce_admins    = true      push_restrictions = []      required_status_checks = {        strict   = true        contexts = []      }      required_pull_request_reviews = {        dismiss_stale_reviews      = true        require_code_owner_reviews = true        dismissal_restrictions     = []      }    }  }}

GitHub Actions

GitHub Actions is our CI tool of choice. GitHub Actions is a hosted runner service provided by GitHub. Any user can write individual tasks, called actions, and put them together into a workflow. These workflows can trigger off numerous events, such as pull requests, comments, labels, releases, and so forth. I think of it as having a box of LEGO bricks that can be put together as needed; I can build a space ship or a pirate ship as my heart desires.

Users are free to write their own actions or consume them from the GitHub Marketplace. For example, the action that performs code checkout is written by GitHub and is on the Marketplace. For a more in-depth introduction to GitHub Actions, I suggest reading the Getting started with GitHub Actions documentation.

For the purpose of this article, I am using GitHub Actions to construct a workflow to provide CI functionality.

There is one workflow that runs on every PR:

name: Terraform Planon:  - pull_requestjobs:  terraform:    name: Terraform    runs-on: ubuntu-latest    env:      ARM_CLIENT_ID: ${{ secrets.CLIENT_ID }}      ARM_CLIENT_SECRET: ${{ secrets.GH_TERRAFORM_PRD }}      ARM_SUBSCRIPTION_ID: ${{ secrets.SUB_ID }}      ARM_TENANT_ID: ${{ secrets.TENANT_ID }}      ARM_ACCESS_KEY: ${{secrets.GH_TERRAFORM_PRD_STATE_BLOB_KEY}}    steps:      - name: Checkout        uses: actions/checkout@master      - name: "Security Scan"        uses: triat/terraform-security-scan@master        env:          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}      - name: Lint Code Base        uses: github/super-linter@master        env:          GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}          VALIDATE_ALL_CODEBASE: true          VALIDATE_MD: true          VALIDATE_TERRAFORM: true      - uses: hashicorp/setup-terraform@master        with:          terraform_version: latest      - name: Terraform Format        id: fmt        run: terraform fmt -check      - name: Terraform Init        id: init        run: terraform init      - name: Terraform Validate        id: validate        run: terraform validate -no-color        env:          TF_VAR_github_token: ${{ secrets.GH_TOKEN }}      - name: Terraform Plan        id: plan        run: terraform plan -no-color        continue-on-error: true        env:          TF_VAR_github_token: ${{ secrets.GH_TOKEN }}      - uses: actions/github-script@master        env:          PLAN: "terraform
${{ steps.plan.outputs.stdout }}"        with:          github-token: ${{ secrets.GH_TOKEN }}          script: |            const output = `#### Terraform Security Scan  \`Success! The configuration is secure.\`            #### Terraform Format and Style  \`${{ steps.fmt.outcome }}\`            #### Terraform Initialization  \`${{ steps.init.outcome }}\`            #### Terraform Validation  ${{ steps.validate.outputs.stdout }}            #### Terraform Plan  \`${{ steps.plan.outcome }}\`            <details><summary>Show Plan</summary>            \`\`\`${process.env.PLAN}\`\`\`            </details>            *Pusher: @${{ github.actor }}, Action: \`${{ github.event_name }}\`, Working Directory: \`${{ env.tf_actions_working_dir }}\`, Workflow: \`${{ github.workflow }}\`*`;            github.issues.createComment({              issue_number: context.issue.number,              owner: context.repo.owner,              repo: context.repo.repo,              body: output            })      - name: Terraform Plan Status        if: steps.plan.outcome == 'failure'        run: exit 1

On a merge, we run the following workflow:

name: Terraform Applyon:  push:    branches:      - mainjobs:  terraform:    name: Apply    runs-on: ubuntu-latest    env:      ARM_CLIENT_ID: ${{ secrets.CLIENT_ID }}      ARM_CLIENT_SECRET: ${{ secrets.GH_TERRAFORM_PRD }}      ARM_SUBSCRIPTION_ID: ${{ secrets.SUB_ID }}      ARM_TENANT_ID: ${{ secrets.TENANT_ID }}      ARM_ACCESS_KEY: ${{ secrets.GH_TERRAFORM_PRD_STATE_BLOB_KEY }}    steps:      - name: Checkout        uses: actions/checkout@master      - uses: hashicorp/setup-terraform@master        with:          terraform_version: 0.14.3       - name: Terraform Init        id: init        run: terraform init      - name: Terraform Apply        id: apply        run: terraform apply -auto-approve -input=false        env:          TF_VAR_github_token: ${{ secrets.GH_TOKEN }}

Release Drafter is set up as follows:

name: Release Drafteron:  push:    branches:      - mainjobs:  update_release_draft:    runs-on: ubuntu-latest    steps:      - uses: release-drafter/release-drafter@master        env:          GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}

with the following YAML for config:

categories:  - title: ' Features'    labels:      - 'feature'  - title: ' Bug Fixes'    labels:      - 'bug'  - title: ' Maintenance'    label: 'maintenance'  - title: ' Docs'    label: 'docs'  - title: ' Dependencies'    label: 'dependencies'change-template: '- $TITLE @$AUTHOR (#$NUMBER)'change-title-escapes: '\<*_&' # You can add # and @ to disable mentions, and add ` to disable code blocks.template: |  ## Changes  $CHANGES

To ensure everything is captured in the release notes, we also use the Enforce Label Action:

name: Enforce PR labelson:  pull_request:    types: [labeled, unlabeled, opened, edited, synchronize]jobs:  enforce-label:    runs-on: ubuntu-latest    steps:    - uses: yogevbd/enforce-label-action@master      with:        REQUIRED_LABELS_ANY: "bug,docs,feature,maintenance,dependencies"        REQUIRED_LABELS_ANY_DESCRIPTION: "Select at least one label ['bug','docs','feature','maintenance','dependencies']"

This is what you see in the Pull Request Tab, each time the PR workflow runs. Allowing you to quickly see if a PR can be merged:

We also push this information into Microsoft Teams using the GitHub Integration App

DevSecOps

An additional benefit of using a CI workflow is adding automated tests. In this scenario, Ive added a step leveraging tfsec to scan for static code vulnerabilities. In the example below, tfsec warns against creating an Azure network security rule which is fully open. This will halt and fail the workflow unless I provide an ignore comment to accept the warning.

Summary

In this post, I explored using GitHub Actions as a CI workflow that could build and maintain a GitHub organization including users, teams, permissions, security etc. I started by generating a new GitHub repository, then wrote the GitHub Workflow files, and finally started testing the CI workflow and introducing small, incremental changes.

Using Terraform to manage GitHub organisations can really help your developer teams to onboard quickly and securely.

I hope I could help you learn something new today, and share how we do things here at FirstPort.

Any questions, get in touch on Twitter

Andrew [MVP]

@ghostinthewire5

Azure Readiness Checklist

Are you ready to go to prod on Azure? Use this checklist to find out: azurechecklist.com

Call for #contributors! - Help make this THE go to list for #Azure

#AzureDevOps #OpenSource - PLS RT

09:24 AM - 01 Nov 2019