dagu-org
Nix-Shell not found but dry run succeeds
Running Dagu via Docker Compose. I don't know if it's the image or what, but the invocation for nix-shell doesn't work when called in a DAG. Strange thing is, this failure isn't picked up when executing dagu dry <DAG>. Info from within the container's below:
DAG Snippet
- name: list-all-databases
shell: nix-shell
shellPackages: [mariadb]
command: |
mariadb -h mariadb -u root -p"${MARIADB_ROOT_PASSWORD}" -Nse "SHOW DATABASES;" | egrep -v '(information|performance)_schema|mysql|sys'
output: MADB_LIST
- name: db-folder-check-creation-backup
depends: list-all-databases
shell: nix-shell
shellPackages: [mariadb]
workingDir: /backups/dbs/mariadb
script: |
for mdatabase in $(echo ${MADB_LIST}) ; do
mkdir -p ${mdatabase}
mariadb_dump --u root --p"${MARIADB_ROOT_PASSWORD}" --databases ${madb} > ${mdatabase}/dump-$(date +%Y%m%d)-$(cat /proc/sys/kernel/random/uuid).sql
done
Container Info
┌─ DAG: mariadb-backup ────────────────────────────────────────────────────────────────────────────┐
│ Status: Failed ✗ | Started: 13:39:27 | Elapsed: 401ms │
│ Run ID: 0199494f-7186-7c91-9dec-37a1e39815f1 │
└──────────────────────────────────────────────────────────────────────────────────────────────────┘
Progress: ████████████████████████████████████████ 100% (3/3 steps)
Completed Steps:
✗ list-all-databases [0s] Error: failed to execute step "list-all-databases": exec: "nix-shell": executable file not found in $PATH
⚠ db-backup-cleanup
⚠ db-folder-check-creation-ba...
d3a6be505b43:/dagu/dags# dagu dry mariadb-backups.yaml
time=2025-09-14T13:40:04.924-04:00 level=INFO msg="Step started" dag=mariadb-backup dagRunId=01994950-04ba-728c-8737-ccbae3ee5175 step=list-all-databases
time=2025-09-14T13:40:04.931-04:00 level=INFO msg="Step finished" dag=mariadb-backup dagRunId=01994950-04ba-728c-8737-ccbae3ee5175 step=list-all-databases status=finished
time=2025-09-14T13:40:04.934-04:00 level=INFO msg="Step finished" dag=mariadb-backup dagRunId=01994950-04ba-728c-8737-ccbae3ee5175 step=list-all-databases status=finished
time=2025-09-14T13:40:05.034-04:00 level=INFO msg="Step started" dag=mariadb-backup dagRunId=01994950-04ba-728c-8737-ccbae3ee5175 step=db-folder-check-creation-backup
time=2025-09-14T13:40:05.040-04:00 level=INFO msg="Step finished" dag=mariadb-backup dagRunId=01994950-04ba-728c-8737-ccbae3ee5175 step=db-folder-check-creation-backup status=finished
time=2025-09-14T13:40:05.047-04:00 level=INFO msg="Step finished" dag=mariadb-backup dagRunId=01994950-04ba-728c-8737-ccbae3ee5175 step=db-folder-check-creation-backup status=finished
time=2025-09-14T13:40:05.140-04:00 level=INFO msg="Step started" dag=mariadb-backup dagRunId=01994950-04ba-728c-8737-ccbae3ee5175 step=db-backup-cleanup
time=2025-09-14T13:40:05.143-04:00 level=INFO msg="Step finished" dag=mariadb-backup dagRunId=01994950-04ba-728c-8737-ccbae3ee5175 step=db-backup-cleanup status=running
time=2025-09-14T13:40:05.149-04:00 level=INFO msg="Step finished" dag=mariadb-backup dagRunId=01994950-04ba-728c-8737-ccbae3ee5175 step=db-backup-cleanup status=finished
time=2025-09-14T13:40:05.244-04:00 level=INFO msg="Dry-run completed" dag=mariadb-backup dagRunId=01994950-04ba-728c-8737-ccbae3ee5175 params=[]
Summary ->
+--------------------------------------+----------------+---------------------------+---------------------------+----------+--------+-------+
| RUN ID | NAME | STARTED AT | FINISHED AT | STATUS | PARAMS | ERROR |
+--------------------------------------+----------------+---------------------------+---------------------------+----------+--------+-------+
| 01994950-04ba-728c-8737-ccbae3ee5175 | mariadb-backup | 2025-09-14T13:40:04-04:00 | 2025-09-14T13:40:05-04:00 | finished | | |
+--------------------------------------+----------------+---------------------------+---------------------------+----------+--------+-------+
Details ->
+---+---------------------------------+------------+---------------------------+----------+----------------------------------------------------------------------------------------------------------------------------+-------+
| # | STEP | STARTED AT | FINISHED AT | STATUS | COMMAND | ERROR |
+---+---------------------------------+------------+---------------------------+----------+----------------------------------------------------------------------------------------------------------------------------+-------+
| 1 | list-all-databases | | 2025-09-14T13:40:04-04:00 | finished | -h mariadb -u root -p"${MARIADB_ROOT_PASSWORD}" -Nse SHOW DATABASES; | egrep -v (information|performance)_schema|mysql|sys | |
| 2 | db-folder-check-creation-backup | | 2025-09-14T13:40:05-04:00 | finished | | |
| 3 | db-backup-cleanup | | 2025-09-14T13:40:05-04:00 | finished | $(pwd) -type f -name *.sql -ctime +7 | xargs rm -fv | |
+---+---------------------------------+------------+---------------------------+----------+----------------------------------------------------------------------------------------------------------------------------+-------+
d3a6be505b43:/dagu/dags# dagu version
1.22.4-be57f3d
d3a6be505b43:/dagu/dags# which nix-shell
d3a6be505b43:/dagu/dags#
@TrezOne There is a couple of issues here. First of all, dagu dry does not actually run the commands (https://docs.dagu.cloud/reference/cli#dry), which is why it looks like its succesful.
Second of all, nix-shell is not installed by default in the upstream dagu container. I am using the following Dockerfile to install it (only been using this for a few days, long term I will probably change this to install a specific version of nix)
FROM ghcr.io/dagu-org/dagu:latest
# install curl
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y \
curl \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# install nix
RUN curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install linux \
--extra-conf "sandbox = false" \
--init none \
--no-confirm
ENV PATH="${PATH}:/nix/var/nix/profiles/default/bin"
# need this to fix path when running commands from web interface
ENTRYPOINT dagu start-all
At the risk of sounding rude, if nix-shell isn't included in the upstream Dagu container, why is it part of the documentation then? It's a bit misleading 🫤 Anyway, I'll probably take the same route a la the custom image/Dockerfile in the interim.
I don't think the only way to use dagu is in a container, and packaging nix in a container can be tricky because of the nix store. But probably would be nice to add a note in the docs that nix is not installed by default.
I think its the same thing with docker though - by default, the docker executor is also not going to work in the container because docker is not installed (you have to mount the docker volume).
Hopefully the dockerfile works for you in the meantime! Might not be a bad idea to revise it to use the official upstream installer instead of the determinate systems installer, that was just my "quick and dirty" dockerfile so I could get some stuff going.
All that being said, I am not affiliated with the project, I just ran across your issue while trying to figure out my own issues with nix-shell and scripts. ¯_(ツ)_/¯
Good luck!