s3gw
s3gw copied to clipboard
RFC: sfs file tree: store metadata in MP / version files
What needs to be done
Store metadata (e.g object id, mp id, commit time, owner) with the MP part / object version file in the SFS dir tree.
Why it needs to be done
It would help a fsck tool and a human debugging.
Design Space
Header / footer
Let file data start at a fixed offset. Append data after file data.
A fixed offset header needs to take alignment into account as SFS will never read it.
Can be implemented so that head -1
works.
XAttrs
Store everything in a single / separate xattrs.
xattrs have performance / overhead on the filesystem that needs to be taken into account. Backing up / copying the datastore may loose xattrs.
Is this intended to duplicate, supplement, or replace the metadata that's already present in the sqlite database?
Duplicate. Put all information necessary with the file to make it self describing
Seems reasonable, and definitely helpful to humans debugging.
For fsck it adds another thing to check: that file metadata matches db metadata :-P
This should be defined in an ADR