tl;dr: Shisho is an open-source static code analyzer that lets you build your own lint rules for Terraform codes. You can find and refactor specific code patterns easily with a handy configuration language.

Building Linter / Static Analyzer is Too Hard

Every developer wants to avoid embedding issues in their software, while finding issues tends to be boring. So here's where a linter / a static analyzer come: they will be a great supporter of you and your team by detecting common bugs with pre-defined rules before the bugs are shipped to the world.

Sometimes, you will want to enforce custom rules for your code to standardize best practices specific to your team. When you want to prevent your team members from using uniform_bucket_level_access = true in google_storage_bucket resources like the following snippet, for example, you hope there's a flexible linter that lets you add custom rules quickly:

resource "google_storage_bucket" "test" {  project  = var.project  name     = "${var.project}-test"  location = var.location  uniform_bucket_level_access = true  force_destroy               = true}

However, adding and maintaining custom rules is quite hard! You need to learn how to write custom rules for each programming language your team use, although different programming languages have different linters or analyzers, with different DSLs and APIs. This difficulty is one of the severe problems of standard linters / static analyzers.

Shisho: A Customizable Static Code Analyzer

Shisho, a lightweight static code analyzer, will help you build custom lint rules for your codebase. I'll explain what and how it is.

echo "func test(v []string) int { return len(v) + 1; }" | docker run -i ghcr.io/flatt-security/shisho-cli:latest find "len(:[...])" --lang=go

echo "func test(v []string) int { return len(v) + 1; }" > file.godocker run -i -v $(PWD):/workspace ghcr.io/flatt-security/shisho-cli:latest find "len(:[...])" --lang=go /workspace/file.go

# Linuxwget https://github.com/flatt-security/shisho/releases/latest/download/build-x86_64-unknown-linux-gnu.zip -O shisho.zipunzip shisho.zipchmod +x ./shishomv ./shisho /usr/local/bin/shisho# macOSwget https://github.com/flatt-security/shisho/releases/latest/download/build-x86_64-apple-darwin.zip -O shisho.zipunzip shisho.zipchmod +x ./shishomv ./shisho /usr/local/bin/shisho

Find Codes

First of all, Shisho enables us to run AST-aware code search over your code. Here's an example command which finds the occurence of uniform_bucket_level_access = true inside google_storage_bucket resource:

docker run -i -v $(pwd):/workspace ghcr.io/flatt-security/shisho-cli:latest find "resource \"google_storage_bucket\" :[_] {  :[...]  uniform_bucket_level_access = true  :[...]}" --lang hcl ./code.tf

The command will make the following outputs in your console:

Here :[_] is an anonymous metavariable, which matches an arbitrary single node in AST (like a function call, identifier, and so on). Similarly, :[...] is an anonymous ellipsis metavariable, which matches zero or more nodes in AST. These operators are something like capture groups in regular expressions. They let you search over your code in a structured but flexible manner.

You can also define a rule, which includes a pattern and the explaination for it. The following YAML snippet is an example of rules describing the use of uniform_bucket_level_access is prohibited:

version: "1"rules:  - id: sample-policy    language: hcl    pattern: |      resource "google_storage_bucket" :[_] {        :[...X]        uniform_bucket_level_access = true        :[...Y]      }    message: |      Our team policy prohibits the use of uniform bucket-level access.

You can find patterns by executing shisho find path/to/rule.yaml path/to/search command, resulting in the following outputs:

This is how Shisho makes it possible to build your own lint rules for Terraform codes. You can use Shisho in the CI pipeline with your own rules, let alone your local machine. Please see Learn Shisho for further details.

Refactor Codes

Additionally, Shisho rules can include how detected code patterns should be fixed. The following YAML snippet describes a custom lint rule that suggests the use of uniform_bucket_level_access = true should be deleted:

version: "1"rules:  - id: sample-policy    language: hcl    pattern: |      resource "google_storage_bucket" :[NAME] {        :[...X]        uniform_bucket_level_access = true        :[...Y]      }    message: |      Our team policy prohibits use of uniform bucket-level access.    rewrite: |      resource "google_storage_bucket" :[NAME] {        :[X]        :[Y]      }

Once this rule is run over your codes and the use of uniform_bucket_level_access = true is detected, Shisho suggests changes following the rule's rewrite section like:

Usecases

You can use Shisho for standardizing your codebase. In addition, it could be a means of conducting "security-as-code" or "policy-as-code"!

For instance, when you want to keep your team's EBS volumes encrypted, you can define a rule as follows:

version: '1'rules:  - id: 'unencrypted-ebs-volume'    language: hcl    message: |      There was unencrypted EBS module.    pattern: |      resource  "aws_ebs_volume" :[NAME] {        :[...X]      }    constraints:      - target: X        should: not-match        pattern: |          encrypted = true    rewrite: |      resource "aws_ebs_volume" :[NAME] {        :[X]        encrypted = true      }

When you want your colleagues to follow the naming convention for resources, the following rule will work well:

version: "1"rules:  - id: "invalid-resource-name"    language: hcl    message: |      A resource was named badly.    pattern: |      resource  :[_] :[NAME] {        :[...]      }    constraints:      - target: NAME        should: not-match-regex        pattern: '"team1-.*"'

The rule will report the result like:

Why Shisho?

"Modern Static Analysis: how the best tools empower creativity" explains that good code analyzers or linters are often interoperable, moldable, efficient, and community-driven and that Semgrep works well from these viewpoints. Semgrep is also grep-like (or sed-like) software that lets us find bugs with useful DSLs.

As for Shisho, it is at least interoperable (since it's open-sourced), moldable (though some efforts are needed; see issue #7). Moreover, Shisho is surprisingly efficient! Here's the result of a micro-benchmark of Semgrep, Comby (a similar tool), and Shisho¹:

In fact, Shisho aims to refine the existing tools and to make it more feasible to run for large projects. You can use Shisho for your monorepo without hesitation. For your information, this speed is supported by Rust ².

On the other hand, it's true that Shisho lacks some features of Semgrep and Comby. For instance, Semgrep has a feature to match patterns with type information while Shisho doesn't. Semgrep also has Semgrep Registry, in which you can share your own lint rules for the worldwide community. Now I'm making efforts to design and implement these features. Stay tuned!

Now What?

This article explained the usage of Shisho for Terraform codes, but Shisho is extending other language supports! Especially Dockerfile support will be shipped soon. You can follow @y0n3uchy to see the news on Shisho and star our GitHub project to encourage us :-)

Additionally, I'll release a SaaS which supports your Terraform development workflows with this engine. See https://shisho.dev/ for further details.

1. Time is the average of 20 consecutive command executions. The measurement was run on Ubuntu 20.04.2 LTS with AMD Ryzen 5 3600 / 64GB RAM. The scan target was parser.go.

2. Both Semgrep and Comby is written in OCaml.