Results are not persisted even with a volume and PROMPTFOO_SHARE_STORE_TYPE=filesystem

[Valkea]

Hello, I deployed promptfoo on a VM using a Docker-compose and the pre-built image available at ghcr.io/promptfoo/promptfoo:main

However, if I run several evaluations and then explore the volume, there is nothing to be found :

And indeed each time in run my CI/CD on GitLab all data are lost.

Expected behavior The data should be visible in the volume, otherwise it can't be persisted.

System information (please complete the following information):

• Promptfoo version: 0.81.5

[mldangelo]

Thank you for reporting this issue, @Valkea. I apologize for the data persistence problems you're experiencing.

To help me diagnose the issue, could you please provide the following information:

1. Your complete docker-compose file
2. The exact command you're using to start the container
3. Any relevant logs from the container

It's unclear from our current setup exactly where the data is being stored. Your input will help us understand the situation better and provide a more accurate solution.

In the meantime, could you try running the container with a general volume mount like this:

volumes:
  - ./promptfoo_data:/app

This will map the entire /app directory in the container to a local promptfoo_data directory. After running some evaluations, check if any data appears in this directory.

If this doesn't help, we may need to investigate further into how the application is handling data storage and file permissions within the container.

Please let us know the results of this test and provide the additional information requested. I am happy to jump on a call if it's easier to debug.

[Valkea]

Hello @mldangelo ,

Here my docker-compose-local.yml file

services:
  database:
    image: postgres:14.1-alpine
    restart: always
    environment:
      - POSTGRES_USER=xxxx
      - POSTGRES_PASSWORD=xxxx
      - POSTGRES_DB=xxxx
    expose:
      - 5432
    ports:
      - "5432:5432"

 promptfoo:
    container_name: "xxxx_promptfoo"
    image: ghcr.io/promptfoo/promptfoo:main
    restart: always
    environment:
    - PROMPTFOO_SHARE_STORE_TYPE="filesystem"
    - PROMPTFOO_SHARE_TTL=2419200
    expose:
    - 3000
    ports:
    - "3000:3000"
    volumes:
    - promptfoo_data:/root/.promptfoo

 mlflow:
    container_name: "xxxx_mlflow"
    image: ghcr.io/mlflow/mlflow:v2.15.1
    command: "mlflow ui --host 0.0.0.0 -p 5001"
    restart: always
    expose:
    - 5001
    ports:
    - "5001:5001"
    volumes:
    - mlflow_runs:/mlruns
    - mlflow_artifacts:/mlartifacts

volumes:
  promptfoo_data:
    name: "promptfoo_data"
  mlflow_runs:
    name: "mlflow_runs"
  mlflow_artifacts:
    name: "mlflow_artifacts"

I'm starting it with the following command :

sudo docker compose -f docker-compose-local.yml up

Here are the promptfoo starting logs

(.venv_dev_uv) [docker]$ sudo docker compose -f docker-compose-local.yml up
[+] Running 3/3
 - Container xxxx_mlflow                                 Created          0.0s 
 - Container xxxx_promptfoo                              Created          0.1s 
 - Container docker-database-1                           Created          0.0s 
Attaching to xxxx_mlflow, xxxx_promptfoo, database-1

xxxx_promptfoo | Writing environment variables to .env file...
xxxx_promptfoo | Loaded environment variables:
xxxx_promptfoo | NODE_VERSION=20.17.0
xxxx_promptfoo | HOSTNAME=0.0.0.0
xxxx_promptfoo | YARN_VERSION=1.22.22
xxxx_promptfoo | SHLVL=2
xxxx_promptfoo | PORT=3000
xxxx_promptfoo | HOME=/home/nextjs
xxxx_promptfoo | PROMPTFOO_SHARE_STORE_TYPE="filesystem"
xxxx_promptfoo | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
xxxx_promptfoo | NEXT_TELEMETRY_DISABLED=1
xxxx_promptfoo | PROMPTFOO_SHARE_TTL=2419200
xxxx_promptfoo | PWD=/app
xxxx_promptfoo | Starting server...
xxxx_promptfoo | - ready started server on 0.0.0.0:3000, url: http://localhost:3000
xxxx_promptfoo | - info Loaded env from /app/src/web/nextui/.env
xxxx_promptfoo | - info Loaded env from /app/src/web/nextui/.env
xxxx_promptfoo | Listening on port 3000 url: http://0.0.0.0:3000

Then when running 2 experiments in a row:

xxxx_promptfoo  | Using Filesystem store
xxxx_promptfoo  | Storing eval result with id f:ec4da7a1-cbb7-4011-b50e-31674b9fbc61
xxxx_promptfoo  | Storing eval result with id f:993c1395-20f2-44ac-ac27-6e2c901e1ad6

[Valkea]

@mldangelo

I tried several volumes configurations :

    volumes:
    - promptfoo_data:/root/.promptfoo

Result: no data in the promptfoo_data volume

---

    volumes:
    - ./promptfoo_data:/app

Result : the docker returns the following logs

Attaching to xxx_mlflow, xxx_promptfoo, database-1
xxx_promptfoo  | sh: can't open 'entrypoint.sh': No such file or directory
xxx_promptfoo exited with code 0
xxx_promptfoo  | sh: can't open 'entrypoint.sh': No such file or directory
xxx_promptfoo  | sh: can't open 'entrypoint.sh': No such file or directory
xxx_promptfoo  | sh: can't open 'entrypoint.sh': No such file or directory
xxx_promptfoo  exited with code 2
xxx_promptfoo  | sh: can't open 'entrypoint.sh': No such file or directory
xxx_promptfoo  | sh: can't open 'entrypoint.sh': No such file or directory
...

---

    volumes:
    - ./promptfoo_data:/root/.promptfoo

Result: promptfoo is acting normally, but no data in the promptfoo_data folder

---

    volumes:
    - ./promptfoo_data/app:/app
    - ./promptfoo_data/root:/root

Result : the docker returns the following logs

Attaching to xxx_mlflow, xxx_promptfoo, database-1
xxx_promptfoo  | sh: can't open 'entrypoint.sh': No such file or directory
xxx_promptfoo exited with code 0
xxx_promptfoo  | sh: can't open 'entrypoint.sh': No such file or directory
xxx_promptfoo  | sh: can't open 'entrypoint.sh': No such file or directory
xxx_promptfoo  | sh: can't open 'entrypoint.sh': No such file or directory
xxx_promptfoo  exited with code 2
xxx_promptfoo  | sh: can't open 'entrypoint.sh': No such file or directory
xxx_promptfoo  | sh: can't open 'entrypoint.sh': No such file or directory
...

[cmrfrd]

@Valkea From the logs you're showing

sh: can't open 'entrypoint.sh': No such file or directory

This error shouldn't happen in future versions as an entrypoint script isn't being used. Try upgrading and running your compose setup again.

[mldangelo]

Hi @Valkea, thanks for the detailed report with screenshots.

This issue should be resolved in current versions. As @cmrfrd noted in November 2024, the entrypoint.sh error you encountered (sh: can't open 'entrypoint.sh': No such file or directory) was fixed - we no longer use an entrypoint script that could conflict with volume mounts.

For anyone encountering persistence issues today

The recommended setup is:

services:
  promptfoo:
    image: ghcr.io/promptfoo/promptfoo:latest
    volumes:
      - promptfoo_data:/root/.promptfoo
    environment:
      - PROMPTFOO_CONFIG_DIR=/root/.promptfoo

volumes:
  promptfoo_data:

Key points:

• Mount to /root/.promptfoo (the default config directory), not /app
• Use a named volume rather than a bind mount to avoid permission issues
• The PROMPTFOO_SHARE_STORE_TYPE=filesystem setting is no longer required for basic persistence

If anyone is still experiencing this issue with recent versions (0.100+), please open a new issue with your docker-compose configuration and we'll investigate.

Closing as stale - the underlying entrypoint issue has been fixed. @Valkea please reopen if you're still experiencing this with the latest image!