Improving Docker Image Creation

[sgillespie]

Currently the Docker image for DB Sync is large, complex, and discourages customization. Given the majority of our users are using the docker image, I would like to revisit this strategy.

Introduction

Docker images are built from the NixOS module. While this approach does have benefits in reproduceability, it also presents several challenges:

• Image Size: Recent releases are 1GB+, even though the binary is less that 100MB
• Nix specific: Store paths confuse non-Nix users
• Bundled Configs: Included network configuration makes customization difficult

The NixOS module has significant complexity, mostly bash wrappers that inject network-specific configuration from the Cardano Operations Book.

A while back, we added configuration options to tune DB Sync to each user's specific needs. While convenient, the batteries-included approach make customization more difficult.

Proposed Solution

Instead of building the Docker images with Nix, I propose we take a hybrid approach:

1. Build static binaries with Nix
2. Use a simple Dockerfile to copy artifacts to the Docker image

Example Workflow

# Create a static executable
nix build .#cardano-db-sync-linux

# Copy the executable to docker context
cp -L result/bin/cardano-db-sync docker/

# Build the docker image
docker build docker

Configuration Files

The configuration files would not be available in the image. Instead, users have two options:

• Option 1: Mount the configs in a data volume
• Option 2: Create a derived image with the configs baked in

The latest configuration files are available in the Cardano Operations Book, which matches the workflow for non-containerized installations.

Benefits

• 10x smaller images: 100MB instead of 1GB
• Standard Docker Experience: Easier for non-nix users to debug
• Explicit Configuration: Straightforward customization for all users

[rdlrt]

That's a welcome change, something also worth considering especially for docker use-case is snapshot validation before extraction begins (potentially include config of what's included in snapshot), could be optimisation for postgresql-setup.sh script instead of just docker

[aleeusgr]

Cool idea @sgillespie thanks

Contact me: [email protected] | https://cal.com/alexeusgr

testing: https://github.com/aleeusgr/cardano-db-sync/pull/1