pyxattr icon indicating copy to clipboard operation
pyxattr copied to clipboard

Consider adding incremental read and seek interface for large attribute values

Open joachimmetz opened this issue 4 years ago • 5 comments

Some file systems, such as XFS, allow to store large amounts of data in extended attributes. In pyxattr I only see getxattr to retrieve attribute value data, which returns all data. For large attribute values it would be useful to have means to do incremental reads and seeks.

joachimmetz avatar Jul 28 '21 08:07 joachimmetz

Oh, good idea - thanks!

iustin avatar Jul 28 '21 08:07 iustin

While looking into another xattr related thing, while reviewing man 2 getxattr my request might be easier said/asked than implemented since getxattr() does not appear to have an equivalent that supports incremental read.

Instead it states:

E2BIG  The size of the attribute value is larger than the maximum size allowed; the attribute cannot be retrieved.  This can happen on filesystems that support very large attribute values such as NFSv4, for example

So feel free to close if this turns out to be infeasible

joachimmetz avatar Jul 28 '21 15:07 joachimmetz

I always thought that it's the main feature of xattrs: they're always read/written in one piece (maybe even atomically).

BTW, XFS xattrs are limited to 64k, I don't think this value is that large.

Nable80 avatar Jul 28 '21 15:07 Nable80

@Nable80 indeed the XFS documentation states 64k but the format technically allows for larger data. So this might not be that useful feature request at this time.

joachimmetz avatar Jul 28 '21 15:07 joachimmetz

I'm not the only one who mentions these design considerations: https://www.spinics.net/lists/linux-fsdevel/msg85757.html

Nable80 avatar Jul 28 '21 20:07 Nable80

Given the state of xattrs upstream, this is infeasible right now, so I’ll mark it as such. Feel free to reopen if things change.

iustin avatar Oct 12 '22 18:10 iustin