End to end SOPS dotenv tutorial needed

[evangineer]

Novops looks like a really good project. I discovered it yesterday and it should be ideal for my use case which is to encrypt an entire .env using sops and then decrypt and retrieve all of the env vars.

However, being new to sops, age and novops, I was not able to pull all the pieces together to make it work.

Would it be possible for you to create an end to end tutorial that covers this very common use case, starting with key creation and moving onto .env encryption.

This is so needed, here's proof: https://github.com/djenko-it/mattermostbehindtraefik/issues/1

[PierreBeucher]

Indeed it's a bit rough on the edges without knowing this. I'll try and add a bit of doc !

[evangineer]

I've got the encryption piece down as follows and I can manually decrypt with sops decrypt enc.env after setting up $HOME/.config/sops/age/keys.txt:

Steps

1. Generate age keypair if you don't have existing age recipient(s) using age-keygen -o key.txt.
2. Create .sops.yaml in the root of the repo and specify age recipient(s).
3. sops encrypt .env > enc.env

Resources

• SOPS: Encrypting using age
• Using .sops.yaml conf to select KMS, PGP and age for new files

Haven't got novops load working yet despite having a very simple .novops.yml based on the SOPs example in the novops docs.

[evangineer]

.novops.yml is in the root of my repo and reads as follows:

environments:
  dev:
    # This is a direct sub-key of environment name
    # Not a sub-key of files or variables
    sops_dotenv:

      # Use plain file content as dotenv values
      - file: enc.env

I'm getting the following error:

novops load --skip-tty-check
thread 'tokio-runtime-worker' panicked at src/modules/sops.rs:121:52:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: An error occured. Set environment variable RUST_LOG=[trace|debug|info|warn] or RUST_BACKTRACE=1 for more verbosity.

Caused by:
    0: Failed to load environment.
    1: Failed to resolve one or more Inputs:

       ---
       task 17 panicked with message "called `Option::unwrap()` on a `None` value"

[evangineer]

Manually running sops --decrypt --output-type dotenv ./enc.env works fine, so I don't understand why novops load is failing.

[evangineer]

Dry run is fine...

novops load --dry-run --skip-tty-check
export RESULT='./enc.env:'
export NOVOPS_ENVIRONMENT='dev'

... problem happens at the SOPS decrypt stage.

RUST_LOG=novops=debug novops load --skip-tty-check
[2025-03-06T15:51:17Z DEBUG novops] env_logger::try_init() error: SetLoggerError(()) - logger was probably already initialized, no worries.
[2025-03-06T15:51:17Z DEBUG novops] Loading context for NovopsLoadArgs { config: None, env: None, working_directory: None, skip_working_directory_check: Some(false), dry_run: Some(false) }
[2025-03-06T15:51:17Z DEBUG novops] Loading config file path '"/home/evangineer/mattermost-docker/.novops.yml"'
[2025-03-06T15:51:17Z DEBUG novops] Only one environment configured, using it by default
[2025-03-06T15:51:17Z INFO  novops] Using workdir: "/run/user/1000/novops/mattermost-docker/dev"
[2025-03-06T15:51:17Z DEBUG novops] Prepared context: NovopsContext { env_name: "dev", app_name: "mattermost-docker", workdir: "/run/user/1000/novops/mattermost-docker/dev", config_file_data: NovopsConfigFile { name: None, environments: {"dev": NovopsEnvironmentInput { variables: None, files: None, aws: None, hashivault: None, sops_dotenv: Some([SopsDotenvInput { file: "enc.env", additional_flags: None, extract: None }]) }}, config: None }, env_var_filepath: "/run/user/1000/novops/mattermost-docker/dev/vars", dry_run: false }
[2025-03-06T15:51:17Z INFO  novops::resolve] Resolving SOPS Dotenv inputs
[2025-03-06T15:51:17Z DEBUG novops::modules::sops] Running sops command with args: ["--decrypt", "--output-type", "dotenv", "enc.env"]
[2025-03-06T15:51:17Z DEBUG novops::modules::sops] sops stderr: ''
thread 'tokio-runtime-worker' panicked at src/modules/sops.rs:121:52:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: An error occured. Set environment variable RUST_LOG=[trace|debug|info|warn] or RUST_BACKTRACE=1 for more verbosity.

Caused by:
    0: Failed to load environment.
    1: Failed to resolve one or more Inputs:

       ---
       task 17 panicked with message "called `Option::unwrap()` on a `None` value"

[PierreBeucher]

Can you share the content of your .novops.yml and a redacted (remove secrets and replace them with dummy value) SOPS content to reproduce ?

[evangineer]

If I can find time next week, I will.

I'm deep in the project that I was looking at this for and managed to get agebox working for it.

However, I was disturbed by the fact that unlike with novops, there was still a risk of committing unencrypted secrets so also implemented a pre-commit hook using gitleaks that has already saved me from one mishap.

[evangineer]

Deleted .novops.yml did contain the following:

environments:
  dev:
    # This is a direct sub-key of environment name
    # Not a sub-key of files or variables
    sops_dotenv:

      # Use plain file content as dotenv values
      - file: enc.env

The following is example SOPS content:

# Domain of service
DOMAIN=mm.example.com

# Container settings
## Timezone inside the containers. The value needs to be in the form 'Europe/Berlin'.
## A list of these tz database names can be looked up at Wikipedia
## https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
TZ=UTC
RESTART_POLICY=unless-stopped

# Postgres settings
## Documentation for this image and available settings can be found on hub.docker.com
## https://hub.docker.com/_/postgres
## Please keep in mind this will create a superuser and it's recommended to use a less privileged
## user to connect to the database.
## A guide on how to change the database user to a nonsuperuser can be found in docs/creation-of-nonsuperuser.md
SOLIDBLOCKS-RDS-POSTGRESQL_IMAGE_TAG=14-v0.3.2
POSTGRES_IMAGE_TAG=13-alpine
POSTGRES_DATA_PATH=./volumes/db/var/lib/postgresql/data

POSTGRES_USER=mmuser
POSTGRES_PASSWORD=mmuser_password
POSTGRES_DB=mattermost

# Nginx
## The nginx container will use a configuration found at the NGINX_MATTERMOST_CONFIG. The config aims
## to be secure and uses a catch-all server vhost which will work out-of-the-box. For additional settings
## or changes ones can edit it or provide another config. Important note: inside the container, nginx sources
## every config file inside */etc/nginx/conf.d* ending with a *.conf* file extension.

## Inside the container the uid and gid is 101. The folder owner can be set with
## `sudo chown -R 101:101 ./nginx` if needed.
## Note that this repository requires nginx version 1.25.1 or later
NGINX_IMAGE_TAG=alpine

## The folder containing server blocks and any additional config to nginx.conf
NGINX_CONFIG_PATH=./nginx/conf.d
NGINX_DHPARAMS_FILE=./nginx/dhparams4096.pem

CERT_PATH=./volumes/web/cert/cert.pem
KEY_PATH=./volumes/web/cert/key-no-password.pem
#GITLAB_PKI_CHAIN_PATH=<path_to_your_gitlab_pki>/pki_chain.pem
#CERT_PATH=./certs/etc/letsencrypt/live/${DOMAIN}/fullchain.pem
#KEY_PATH=./certs/etc/letsencrypt/live/${DOMAIN}/privkey.pem

## Exposed ports to the host. Inside the container 80, 443 and 8443 will be used
HTTPS_PORT=443
HTTP_PORT=80
CALLS_PORT=8443

# Mattermost settings
## Inside the container the uid and gid is 2000. The folder owner can be set with
## `sudo chown -R 2000:2000 ./volumes/app/mattermost`.
MATTERMOST_CONFIG_PATH=./volumes/app/mattermost/config
MATTERMOST_DATA_PATH=./volumes/app/mattermost/data
MATTERMOST_LOGS_PATH=./volumes/app/mattermost/logs
MATTERMOST_PLUGINS_PATH=./volumes/app/mattermost/plugins
MATTERMOST_CLIENT_PLUGINS_PATH=./volumes/app/mattermost/client/plugins
MATTERMOST_BLEVE_INDEXES_PATH=./volumes/app/mattermost/bleve-indexes

## Bleve index (inside the container)
MM_BLEVESETTINGS_INDEXDIR=/mattermost/bleve-indexes

## This will be 'mattermost-enterprise-edition' or 'mattermost-team-edition' based on the version of Mattermost you're installing.
MATTERMOST_IMAGE=mattermost-enterprise-edition
## Update the image tag if you want to upgrade your Mattermost version. You may also upgrade to the latest one. The example is based on the latest Mattermost ESR version.
MATTERMOST_IMAGE_TAG=9.11.6

## Make Mattermost container readonly. This interferes with the regeneration of root.html inside the container. Only use
## it if you know what you're doing.
## See https://github.com/mattermost/docker/issues/18
MATTERMOST_CONTAINER_READONLY=false

## The app port is only relevant for using Mattermost without the nginx container as reverse proxy. This is not meant
## to be used with the internal HTTP server exposed but rather in case one wants to host several services on one host
## or for using it behind another existing reverse proxy.
APP_PORT=8065

## Configuration settings for Mattermost. Documentation on the variables and the settings itself can be found at
## https://docs.mattermost.com/administration/config-settings.html
## Keep in mind that variables set here will take precedence over the same setting in config.json. This includes
## the system console as well and settings set with env variables will be greyed out.

## Below one can find necessary settings to spin up the Mattermost container
MM_SQLSETTINGS_DRIVERNAME=postgres
MM_SQLSETTINGS_DATASOURCE=postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}?sslmode=disable&connect_timeout=10

## Example settings (any additional setting added here also needs to be introduced in the docker-compose.yml)
MM_SERVICESETTINGS_SITEURL=https://${DOMAIN}

[PierreBeucher]

Thanks, I'll take a look asap