Tar archive does not restore file (or directory) modification time stamps
Describe the bug
Restoring the cache doesn't restore file modification times.
This can effect application which check if a file has expired based on the time stamp. In my case, the dependency-check-maven, which updates CVE database daily and seems to check this based on the cache file timestamp.
To Reproduce
I used the following bash script to develop a fix for this, thus it works as a way to reproduce.
tag=$(git describe --dirty)
image="meltwater/drone-cache:$tag"
mkdir -p my-app/{m2,a/b/c} my-cache
touch -t 2201010101 my-app/m2/my-test-file1.txt
touch -t 2202020202 my-app/m2/my-test-file2.txt
touch -t 2212120000 my-app/a/b/c/foo.txt
touch -t 2212120000 my-app/a/b/c/bar.txt
touch -t 2212120000 my-app/a/b/c
touch -t 2212120000 my-app/a/b
touch -t 2212120000 my-app/a
ls -lR my-app
stat my-app/a
echo " ## Rebuild"
docker run --rm \
-v "$PWD/my-app":/app \
-v "$PWD/my-cache":/cache \
-w /app \
-e DRONE_REPO=octocat/hello-world \
-e DRONE_REPO_BRANCH=master \
-e DRONE_COMMIT_BRANCH=master \
-e PLUGIN_BACKEND=filesystem \
-e PLUGIN_CACHE_KEY=volume \
-e PLUGIN_FILESYSTEM_CACHE_ROOT=/cache \
-e PLUGIN_ARCHIVE_FORMAT=gzip \
-e PLUGIN_MOUNT=m2,a \
-e PLUGIN_RESTORE=false \
-e PLUGIN_REBUILD=true \
"$image"
rm -rf my-app/*
echo " ## Restore"
docker run --rm \
-v "$PWD/my-app":/app \
-v "$PWD/my-cache":/cache \
-w /app \
-e DRONE_REPO=octocat/hello-world \
-e DRONE_REPO_BRANCH=master \
-e DRONE_COMMIT_BRANCH=master \
-e PLUGIN_BACKEND=filesystem \
-e PLUGIN_CACHE_KEY=volume \
-e PLUGIN_FILESYSTEM_CACHE_ROOT=/cache \
-e PLUGIN_ARCHIVE_FORMAT=gzip \
-e PLUGIN_MOUNT=m2,a \
-e PLUGIN_RESTORE=true \
-e PLUGIN_REBUILD=false \
"$image"
ls -lR my-app
stat my-app/a
Expected behavior
After the restore at least files should have the same modification times as before rebuild. Possibly also the directories, but I'm not sure if that is mandatory or even simple.
Desktop (please complete the following information):
- Linux VM running inside Docker Desktop for Mac; Docker version is 20.10.11
- Public container run inside a Drone step
Additional context
I looked into a fix. It currently works for files, but not directories (duh, will be updated by the first extracted file). Also, I don't have tests yet.
- https://github.com/raphendyr/drone-cache/commit/56e257917569053e4c8c21ceddbd3d976818d1dc
Confirmed issue present in v1.3.0 (latest) since git describe --dirty returned v1.2.2-25-g44c16f4:
$ ./script
total 0
drwxr-xr-x 3 brianburnett staff 96 Dec 12 2022 a
drwxr-xr-x 4 brianburnett staff 128 Aug 9 15:46 m2
my-app/a:
total 0
drwxr-xr-x 3 brianburnett staff 96 Dec 12 2022 b
my-app/a/b:
total 0
drwxr-xr-x 4 brianburnett staff 128 Dec 12 2022 c
my-app/a/b/c:
total 0
-rw-r--r-- 1 brianburnett staff 0 Dec 12 2022 bar.txt
-rw-r--r-- 1 brianburnett staff 0 Dec 12 2022 foo.txt
my-app/m2:
total 0
-rw-r--r-- 1 brianburnett staff 0 Jan 1 2022 my-test-file1.txt
-rw-r--r-- 1 brianburnett staff 0 Feb 2 2022 my-test-file2.txt
16777221 95374982 drwxr-xr-x 3 brianburnett staff 0 96 "Dec 12 00:00:00 2022" "Dec 12 00:00:00 2022" "Aug 9 15:46:44 2022" "Aug 9 15:46:44 2022" 4096 0 0 my-app/a
## Rebuild
level=info name=drone-cache ts=2022-08-09T19:46:44.422583Z caller=main.go:492 version=dev commit=none date=unknown
level=warn name=drone-cache ts=2022-08-09T19:46:44.422731Z caller=backend.go:78 component=plugin msg="using filesystem as backend"
level=info name=drone-cache ts=2022-08-09T19:46:44.422918Z caller=rebuilder.go:40 component=plugin component=rebuilder msg="rebuilding cache"
level=info name=drone-cache ts=2022-08-09T19:46:44.422965Z caller=metadata.go:48 component=plugin msg="using provided cache key template"
level=info name=drone-cache ts=2022-08-09T19:46:44.425187Z caller=rebuilder.go:74 component=plugin component=rebuilder msg="rebuilding cache for directory" local=m2 remote=volume/m2
level=info name=drone-cache ts=2022-08-09T19:46:44.426027Z caller=rebuilder.go:125 component=plugin component=rebuilder msg="uploading archived directory" local=/app/m2 remote=volume/m2
level=info name=drone-cache ts=2022-08-09T19:46:44.428528Z caller=rebuilder.go:74 component=plugin component=rebuilder msg="rebuilding cache for directory" local=a remote=volume/a
level=info name=drone-cache ts=2022-08-09T19:46:44.428593Z caller=rebuilder.go:125 component=plugin component=rebuilder msg="uploading archived directory" local=/app/a remote=volume/a
level=info name=drone-cache ts=2022-08-09T19:46:44.428731Z caller=rebuilder.go:113 component=plugin component=rebuilder msg="archiving directory" src=/app/m2
level=info name=drone-cache ts=2022-08-09T19:46:44.430172Z caller=rebuilder.go:113 component=plugin component=rebuilder msg="archiving directory" src=/app/a
level=info name=drone-cache ts=2022-08-09T19:46:44.459193Z caller=rebuilder.go:93 component=plugin component=rebuilder msg="cache built" took=36.229ms
## Restore
level=info name=drone-cache ts=2022-08-09T19:46:45.192766Z caller=main.go:492 version=dev commit=none date=unknown
level=warn name=drone-cache ts=2022-08-09T19:46:45.193018Z caller=backend.go:78 component=plugin msg="using filesystem as backend"
level=info name=drone-cache ts=2022-08-09T19:46:45.193135Z caller=restorer.go:37 component=plugin component=restorer msg="restoring cache"
level=info name=drone-cache ts=2022-08-09T19:46:45.193235Z caller=metadata.go:48 component=plugin msg="using provided cache key template"
level=info name=drone-cache ts=2022-08-09T19:46:45.193401Z caller=restorer.go:55 component=plugin component=restorer msg="restoring directory" local=m2 remote=volume/m2
level=info name=drone-cache ts=2022-08-09T19:46:45.193442Z caller=restorer.go:55 component=plugin component=restorer msg="restoring directory" local=a remote=volume/a
level=info name=drone-cache ts=2022-08-09T19:46:45.193542Z caller=restorer.go:96 component=plugin component=restorer msg="extracting archived directory" remote=volume/a local=a
level=info name=drone-cache ts=2022-08-09T19:46:45.193605Z caller=restorer.go:87 component=plugin component=restorer msg="downloading archived directory" remote=volume/a local=a
level=info name=drone-cache ts=2022-08-09T19:46:45.1938Z caller=restorer.go:96 component=plugin component=restorer msg="extracting archived directory" remote=volume/m2 local=m2
level=info name=drone-cache ts=2022-08-09T19:46:45.193885Z caller=restorer.go:87 component=plugin component=restorer msg="downloading archived directory" remote=volume/m2 local=m2
level=error name=drone-cache ts=2022-08-09T19:46:45.199941Z caller=main.go:608 err="[IMPORTANT] restore cache, restore failed, 2 errors: download from <volume/a> to <a>, extract files from downloaded archive, pipe reader failed, extract regular file, open extracted file for writing <a/b/c/bar.txt>, open a/b/c/bar.txt: no such file or directory;\ndownload from <volume/m2> to <m2>, extract files from downloaded archive, pipe reader failed, extract regular file, open extracted file for writing <m2/my-test-file1.txt>, open m2/my-test-file1.txt: no such file or directory\n"
total 0
stat: my-app/a: stat: No such file or directory