gogs icon indicating copy to clipboard operation
gogs copied to clipboard
verified publisher icon gogs

Docker: `Failed to import 'data': rename /data/backup-restore-tmp/gogs-backup/data/avatars /app/gogs/data/avatars: no such file or directory`

Open shtrom opened this issue 1 year ago • 2 comments

Gogs version

0.14.0+dev

Git version

  • Server: 2.39.5
  • Client: n/a

Operating system

Linux / Docker

Database

SQLITE3

Describe the bug

I have created a backup from my Docker container manually, with

gosu ${USER} ./gogs backup --target "${BACKUP_DIR}" --archive-name "${BACKUP}"

To reproduce

I am now trying to restore the backup in a freshly started container with an empty /data volume.

Essentially a combination of:

  1. (in a container with my backup dir mounted in /host)
  2. creating a temp directory with the correct permissions in /data: mkdir /data/tmp; chown git /data/tmp
  3. running the import with gosu git ./gogs restore --tempdir /data/tmp --from /host/gogs-backup.zip

Neither of the following work:

  • Running without starting the server
$ docker run --rm -it -v ./docker/gogs/:/host gogs/gogs bash
usermod: no changes
16d13d6cb74f:/app/gogs# ./docker/
runtime/  s6/       start.sh  
16d13d6cb74f:/app/gogs# ./docker/s
s6/       start.sh  
16d13d6cb74f:/app/gogs# ./docker/s6/gogs/setup   # to setup symlinks for logs
16d13d6cb74f:/app/gogs# mkdir /data/tmp; chown git /data/tmp
16d13d6cb74f:/app/gogs# gosu git ./gogs restore --tempdir /data/tmp --from /host/gogs-backup.zip
2024/11/13 02:34:40 [ INFO] Restoring backup from: /host/gogs-backup.zip
2024/11/13 02:34:40 [TRACE] Auto migrated "Access"
2024/11/13 02:34:40 [TRACE] Auto migrated "AccessToken"
2024/11/13 02:34:40 [TRACE] Auto migrated "Action"
2024/11/13 02:34:40 [TRACE] Auto migrated "EmailAddress"
2024/11/13 02:34:40 [TRACE] Auto migrated "Follow"
2024/11/13 02:34:40 [TRACE] Auto migrated "LFSObject"
2024/11/13 02:34:40 [TRACE] Auto migrated "LoginSource"
2024/11/13 02:34:40 [TRACE] Auto migrated "Notice"
2024/11/13 02:34:41 [FATAL] [....io/gogs/internal/cmd/restore.go:149 runRestore()] Failed to import 'data': rename /data/tmp/gogs-backup/data/avatars /app/gogs/data/avatars: no such file or directory
16d13d6cb74f:/app/gogs# find /data/ -type d
/data/
/data/git
/data/git/.ssh
/data/ssh
/data/tmp
/data/tmp/gogs-backup
/data/tmp/gogs-backup/db
/data/tmp/gogs-backup/data
/data/tmp/gogs-backup/data/avatars
/data/gogs.bak
/data/gogs.bak/data
/data/gogs.bak/conf
/data/gogs.bak/log
/data/gogs
/data/gogs/conf
/data/gogs/log
/data/gogs/log/hooks
  • Running alongside the server
$ GOGS=$(docker run --rm -d -v ./docker/gogs/:/host gogs/gogs)
$ docker exec -it ${GOGS} bash
d23e9d26bb45:/app/gogs# mkdir /data/tmp; chown git /data/tmp
d23e9d26bb45:/app/gogs# gosu git ./gogs restore --tempdir /data/tmp --from /host/gogs-backup.zip 
2024/11/13 02:31:09 [ INFO] Restoring backup from: /host/gogs-backup.zip
2024/11/13 02:31:09 [TRACE] Auto migrated "Access"
2024/11/13 02:31:09 [TRACE] Auto migrated "AccessToken"
2024/11/13 02:31:09 [TRACE] Auto migrated "Action"
2024/11/13 02:31:09 [TRACE] Auto migrated "EmailAddress"
2024/11/13 02:31:09 [TRACE] Auto migrated "Follow"
2024/11/13 02:31:09 [TRACE] Auto migrated "LFSObject"
2024/11/13 02:31:09 [TRACE] Auto migrated "LoginSource"
2024/11/13 02:31:09 [TRACE] Auto migrated "Notice"
2024/11/13 02:31:09 [FATAL] [....io/gogs/internal/cmd/restore.go:149 runRestore()] Failed to import 'data': rename /data/tmp/gogs-backup/data/avatars /app/gogs/data/avatars: no such file or directory
d23e9d26bb45:/app/gogs# find /data/ -type d
/data/
/data/git
/data/git/.ssh
/data/ssh
/data/tmp
/data/tmp/gogs-backup
/data/tmp/gogs-backup/db
/data/tmp/gogs-backup/data
/data/tmp/gogs-backup/data/avatars
/data/gogs.bak
/data/gogs.bak/data
/data/gogs.bak/conf
/data/gogs.bak/log
/data/gogs
/data/gogs/conf
/data/gogs/log
/data/gogs/log/hooks

Expected behavior

Restore succeed.

Additional context

In both cases it looks like /app/gogs/data/avatars cannot be created, because the parent /app/gogs/data no longer exists. I suspect this https://github.com/gogs/gogs/blob/main/internal/cmd/restore.go#L144-L146 is not moving the right directory out of the way.

Code of Conduct

  • [X] I agree to follow this project's Code of Conduct

shtrom avatar Nov 13 '24 02:11 shtrom

This may be an issue with GOGS_CUSTOM. Unsetting it, and creating the base structure next to the gogs tool works.

9b76de082879:/app/gogs# export -n GOGS_CUSTOM=
9b76de082879:/app/gogs# mkdir data log custom tmp; chown git data log custom tmp  .
9b76de082879:/app/gogs# gosu git ./gogs restore --tempdir ./tmp --from /host/gogs-backup.zip 
2024/11/13 03:35:17 [ INFO] Restoring backup from: /host/gogs-backup.zip
2024/11/13 03:35:17 [TRACE] Auto migrated "Access"
2024/11/13 03:35:17 [TRACE] Auto migrated "AccessToken"
2024/11/13 03:35:17 [TRACE] Auto migrated "Action"
2024/11/13 03:35:17 [TRACE] Auto migrated "EmailAddress"
2024/11/13 03:35:17 [TRACE] Auto migrated "Follow"
2024/11/13 03:35:17 [TRACE] Auto migrated "LFSObject"
2024/11/13 03:35:17 [TRACE] Auto migrated "LoginSource"
2024/11/13 03:35:17 [TRACE] Auto migrated "Notice"
2024/11/13 03:35:17 [ INFO] Restore succeed!

shtrom avatar Nov 13 '24 03:11 shtrom

I ran into this issue as well. The docker image flattens the directory structure a bit and defines the data directory within the custom directory. However, here the custom directory is renamed (backup), and thus the data as well as the log directory no longer exist. As the folder structure is different in the backup archive, moving the custom folder form the archive data does not recreate the data and log folder again, leaving us with the missing folders.

A workaround: One can modify the backup.zip and add the data and log folders in the custom folder.

matamegger avatar Nov 13 '24 22:11 matamegger