storage icon indicating copy to clipboard operation
storage copied to clipboard

Request: Support atomic writes for local storage backend

Open mac-chaffee opened this issue 3 years ago • 1 comments

It looks like there could be a risk of partially-written files with the local backend: https://github.com/chartmuseum/storage/blob/c3a87c79f56958041cb945ecf9706ed3da02e7f0/local.go#L102

That code appears to be writing directly to the real location of the file. So if the write is large and gets interrupted partway through (like chartmuseum is SIGKILL'd or if the file is copied/snapshotted by an external backup program), that could leave a partially-written file on disk.

Minio solves this problem by writing all files to a temporary directory, then atomically moving them into their real location after the write is fully complete: https://github.com/minio/minio/blob/d0862ddf866e6ac358155e3ca660f36610d8834e/cmd/fs-v1.go#L1105

This feature would allow users to backup their chartmuseum data the naive way (cping or taring all the files without having to stop/quiesce the application ahead of time).

Thoughts?

mac-chaffee avatar Apr 14 '22 19:04 mac-chaffee

thanks for reporting and fixing this @mac-chaffee! We’ll definitely want to support this, I’ll take a look at the PR shortly.

cbuto avatar Apr 21 '22 22:04 cbuto