Store Secrets in Vault with Symfony

TL;DR: Integrating Vault and Symfony does not require any PHP code. Using vault-agent secrets variables can be dumped into .env file. Dynamic secrets can even be used as feature flags.

Prisma Media's websites and applications are mostly developed using Symfony framework. Many of them require secrets values: API keys, database credentials, private certificats they needs to be treated carefully.

Why we choose Vault to store secrets

If you are there, you may know that storing secrets in your Git repository is a terrible practice that could lead to severe security issues.

Symfony introduced a "Vault" mecanism to allow storing encrypted secrets in repositories and artifacts. This is a simple solution for basic needs. That has the advantage of being independent from any external system (simplicity, scalability).

We preferred HashiCorp Vault, a server solution that we deployed on our infrastructure. It is centralized, auditable and can handle dynamic secrets. This open-source product is not tied to a cloud provider. Easy to run locally for dev.

Vault works as a key-value store were secrets variables are pushed and read using a REST API.

The need for frequent reload of secrets

Even if we generate the most secure password for your database, it will leak somewhere: logs, APM, error pages... To be safe, credentials needs to change, change all the time.

To get short-living secrets, Vault has a concept of Dynamic Secrets. Unlike key/value secrets where you had to put data into the store yourself, dynamic secrets are generated when they are accessed.

Not only we have to load the secrets from Vault, but also they have to be reloaded every time they change.

Dynamic config of Symfony with .env

Fabien Potencier

@fabpot

My yearly reminder that you should NOT use env vars to configure your #Symfony application. Use env vars ONLY if you want to be able to change values WITHOUT redeploying your application (aka flushing the cache). Parameters ARE the way to configure applications.

08:54 AM - 29 Dec 2020

Thank you Fabien, this is exactly what I need: changing the values without redeploying the application.

Unlike the YAML/PHP files in config/ directory, that are read only when the container cache is built, the .env file are read on every HTTP request. Updating this file while the application is running is a good way to update its configuration without impacting performance.

Vault Agent can write secrets into a file

vault-agent is a small utility that can be used as a cache proxy for the Vault server; or as a schedule to write secrets into a file using a template (every 5 minutes, configurable).

Example of vault-agent configuration for an app running on AWS EC2 instance.

# vault.confvault {  address = "https://vault.example.com"  retry {    num_retries = 5  }}auto_auth {  method "aws" {    config = {      type = "iam"      role = "<iam role>"      region = "eu-west-1"      header_value = "vault.example.com"    }  }}template {  source = "./.env.local.ctmpl"  destination = "./.env.local"}

The template maps Vault secrets to environment variables. The templating syntax allows some flexibility, but it looks very primitive for a developer with Twig practice.

# .env.local.ctmpl# This will generate a regular .env fileAPP_ENV=prod{{ with secret "secret/example/app" }}APP_SECRET={{ .Data.data.APP_SECRET }}{{ end }}# Real environment variables can be read{{ $env := (env "ENVIRONMENT") }}{{ with secret (printf "secret/example/%s/database" $env) }}DB_HOST={{ .Data.data.DB_HOST }}DB_NAME={{ .Data.data.DB_NAME }}DB_USER={{ .Data.data.DB_USER }}DB_PASSWORD={{ .Data.data.DB_PASSWORD }}{{ end }}

Finally, vault agent can be launched with any process manager (supervisord or systemd).

# Daemon for prod servervault agent -config=vault.conf# Single run for testingvault agent -config=vault.conf -exit-after-aut

The last security recommendation is to not write secrets on disks. We can create a tmpfs volume and symlink from the project root to that volume.

For kubernetes, vault-agent runs in a sidecar container that renders Vault secrets to a shared memory volume.

Use dynamic configuration feature flag

Feature flags are a benefit of using Vault and supporting dynamic configuration. Even if they are not secrets, flags can be stored in Vault. With fine tuned policies, product managers could manage feature flags and being rejected from other secrets.

Example of feature flag to render a block in Twig

In this example, we create a feature flag in Vault, with is a boolean to show or hide a "sales" block on a page.

Create a variable in Vault KV:

# secret/example/app/{  "FEATURE_FLAG_SALES": "true"}

Read this secret in the vault agent template

# .env.local.ctmpl{{ with secret "secret/example/app" }}FEATURE_FLAG_SALES={{ .Data.data.FEATURE_FLAG_SALES }}{{ end }}

Share the value of the variable with Twig context.

# app/config/packages/twig.yamltwig:  variables:    feature_flag_sales: "%env(bool:FEATURE_FLAG_SALES)%"

Use the variable to render the block conditionally:

{% if feature_flag_sales %}    <div>My conditional sales block</div>{% endif %}

Run the vault-agent.
When the variable updated, the block is shown or hidden after few minutes.