composefs
composefs copied to clipboard
containers
integration with IPE tracker
As of the current Linux kernel merge window, the IPE LSM was merged. Per the docs an example policy is e.g.
op=EXECUTE dmverity_signature=TRUE action=ALLOW
This relies on the idea that the dm-verity signature was verified in-kernel I believe. In contrast, the fsverity bits in IPE match by digest (annoying at scale) or rely on the in-kernel fsverity signatures which the fsverity maintainer doesn't like.
As folks here know with composefs we can combine the advantages of these. I think there is a bit of plumbing work that needs to be done, Eric Biggers (fsverity maintainer) noted this at one point.
I think in the end state what we want is something like:
op=EXECUTE composefs_signature=TRUE action=ALLOW
Now, this gets into a messy topic of whether signatures are verified in kernel or in user space, and if in user space how we set a property that then later is trusted by the kernel.
And since composefs doesn't exist at the kernel level maybe this would need to be e.g. op=EXECUTE overlayfs_fsverity_signature=TRUE action=ALLOW or so (and covering both the loopback-erofs fsverity vs backing file fsverity.
And in general I think we need propagate metadata across our chain of 3 components (overlays, erofs, and the backing filesystem) so the higher level overlayfs inode that the LSM sees can reliably see that e.g. fsverity was required and enabled on the backing file, etc.
File-backed mount support for EROFS has been merged for Linux v6.12. As that avoids the loopback device, it should make it easier to associate the fsverity signature to the erofs mount.
File-backed mount support for EROFS has been merged for Linux v6.12. As that avoids the loopback device, it should make it easier to associate the fsverity signature to the erofs mount.
Geert has requested for marking it as default n https://lore.kernel.org/r/CAMuHMdVqa2Mjqtqv0q=uuhBY1EfTaa+X6WkG7E2tEnKXJbTkNg@mail.gmail.com
That was why I tended to avoid this stuff in the beginning not because it cannot (just because the kernel file-backed mount controversy) although I know there are many use cases.
Yeah, what would be very good is if EROFS, when using the file-backed mounts could take an optional fs-verity digest mount option, and do the validation of it in the kernel. Then it could somehow mark the mount with a "on fs-verity" flag. And then further, overlayfs could, if all layers except data only layers are marked "on fs-verity" and veryity=enabled is used, then it could also set the "on fs-verity" flag.
Then one could use this flag with something like IPE.
However, this is just the "fs-verity" part, we also would need to propagate the fact that the initial mountpoint was signed, and that seems much more tricky.
, if all layers except data only layers are marked "on fs-verity" and veryity=enabled is used, then it could also set the "on fs-verity" flag.
would the order matter? If two overlay layers are switched, the resulting tree could be different
Yeah, that is true, maybe it would be easiest to just require there to be a single signed layer.
Hi folks, I don't have deep understanding of IPE but since it's related to EROFS I look into related code for a while.
I guess the IPE policy of Overlayfs should follow the metadata layer instead of data-only layers, because data-only file blobs are just deduplicated blobs and could be even from several sources or vendors: I think we shouldn't apply IPE or some signed-sig on those blob files (since they only provide identified binary data), otherwise one blob file may be shipped by many vendors (so one data blob would got many different sigs in principle, I don't think it's useful.)
Specifically, see what ipe_build_eval_ctx does, it calls ino = d_real_inode(file->f_path.dentry); -> d_real(dentry, D_REAL_DATA);,
I'm not sure why it's D_REAL_DATA here.
In principle, IPE will follow the metadata-only layer/fs IPE policy if it uses D_REAL_METADATA in d_real() instead, then we could apply a valid dm-verity signed sig to the meta-only layer and then a IPE policy like:
op=EXECUTE dmverity_signature=TRUE action=ALLOW
will just allow execute files in the signed metadata-layer by dm-verity. BTW, I followed the related logic, I think we cannot use fsverity for the meta-only file because the fsverity IPE policy is a per-inode policy in the filesystem (rather than applied to the filesystem itself) rather than per-device/per-fs policy which can apply to all files in the filesystem.
Then if an inode can be executed decided by the IPE policy above, overlayfs will check fsverity digest according to the metadata layer xattr and use the related data-only file blob.
Does it make sense? Please correct me if I'm wrong.
fs-verity IPE rules are not very interesting in general, because any file can be made fs-verity, it doesn't imply any particular trust in them. What is interesting for IPE is external signatures where someone declared that "this particular set of files are trusted". For dm-verity that means all the files from any filesystem on that partition. For composefs, it means "the data in the composefs, and any external referenced files that matches the digests encoded in the erofs".
So, yes, the signatures are not on the "backing files" in the data-only layer themselves. Only when you access those via a signed erofs, then they are considered "signed".
I have not really looked at the implementation side of IPE in the kernel, but basically this is how I believe this must work:
- Somehow we must be able to sign an erofs image. The signature could be embedded in the fs, or supplied separately at mount time.
- The signature should not be of the full erofs content, but rather of its fs-verity digest. That way it is efficient to verify in the "mount from file" code. (I.e. we don't have to read the entire erofs image file.)
- When you mount such an EROFS it tries to verify the signature against a key in the keyring. If it matches, we mark the erofs superblock as "signed".
- Optionally we could have a mount option where the mount fails if its not signed, but that is not always what you want.
- If you mount an overlayfs, we check if the lowest non-data-only layer is a signed erofs superblock, and the "verity=require" overlayfs mount option is used. If this is true, then all files from the lowest layers are considered "signed", including those read from any data-only layer (because they are required to have a verity digest which will be verified on access)
- Files from higher layers (if any) are not considered signed.
Open question:
- If there are multiple signed erofs lower layers, do we still trust them? One could remount and reorder such layers which would let you access files that otherwise would not be accessible, but you can't get files with arbitrary data this way,
- How does overlayfs work with dm-verity signatures today? If I mount a signed dm-verity filesystem and mount a lower layer from it, are those files then considered signed?
...
- When you mount such an EROFS it tries to verify the signature against a key in the keyring. If it matches, we mark the erofs superblock as "signed".
- Optionally we could have a mount option where the mount fails if its not signed, but that is not always what you want.
- If you mount an overlayfs, we check if the lowest non-data-only layer is a signed erofs superblock, and the "verity=require" overlayfs mount option is used. If this is true, then all files from the lowest layers are considered "signed", including those read from any data-only layer (because they are required to have a verity digest which will be verified on access)
- Files from higher layers (if any) are not considered signed.
Personally I don't think those steps are needed if IPE uses d_real(inode, D_REAL_METADATA), because IPE policies (whether or not to execute) will be applied to the "real" metadata inode, in other words, EROFS itself inode itself.
Overlayfs just check fsverity digest according to the EROFS xattrs as previous. Yes, "verity=require" is a must to keep the trust of chain working. (IOWs, each EXT4/XFS data-only inode can just be considered as "a virtual extent" of EROFS, so the content must be guaranteed by "verity=require")
But I might be wrong on this, otherwise I don't think overlayfs needs to get a "signed" mark or even some additional feature. Just make IPE use the real EROFS metadata inode (D_REAL_METADATA) for IPE execute policies (and all real EROFS metadata inodes apply per-filesystem policies like what the current dm-verity does) instead of EXT4/XFS data-only inodes (D_REAL_DATA).
Open question:
- If there are multiple signed erofs lower layers, do we still trust them? One could remount and reorder such layers which would let you access files that otherwise would not be accessible, but you can't get files with arbitrary data this way,
Does it really matter if "verity=require" (I think it's a must), if the digest mismatches, it will fail the I/O as the current status even IPE policies pass.
- How does overlayfs work with dm-verity signatures today? If I mount a signed dm-verity filesystem and mount a lower layer from it, are those files then considered signed?
According to the IPE codebase, I think dm-verity is a per-filesystem signature, also see ipe_build_eval_ctx() and evaluate_dmv_sig_true because it uses the blockdev context ctx->ipe_bdev = ipe_bdev(INO_BLOCK_DEV(inode)) (basically derived from inode->s_sb->s_bdev) rather than fsverity the inode itself.
I never looked at the actual IPE implementation, but if you're right it seems that this would not be so much work.
If there are multiple signed erofs lower layers, do we still trust them? One could remount and reorder such layers which would let you access files that otherwise would not be accessible, but you can't get files with arbitrary data this way,
Does it really matter if "verity=require" (I think it's a must), if the digest mismatches, it will fail the I/O as the current status even IPE policies pass.
My worry isn't really one of matching verity digest, but of access in the first place. For example, suppose your base for these things is layered like an OCI image, and you have a CVE in /bin/app. To fix this you push out a fixed version of your image, which has a new layer on top that replaces /bin/app. If the layers are mounted as expected you can't see the old /bin/app, but if you reorder them you can. So, signing the individual layers is less expressive than signing the combined order.
Otoh, the same would be true for a dm-verity update if the old dm-verity image was still around and they were signed with the same key. So, maybe this isn't a huge deal.
..
My worry isn't really one of matching verity digest, but of access in the first place. For example, suppose your base for these things is layered like an OCI image, and you have a CVE in /bin/app. To fix this you push out a fixed version of your image, which has a new layer on top that replaces /bin/app. If the layers are mounted as expected you can't see the old /bin/app, but if you reorder them you can. So, signing the individual layers is less expressive than signing the combined order.
Otoh, the same would be true for a dm-verity update if the old dm-verity image was still around and they were signed with the same key. So, maybe this isn't a huge deal.
I think IPE won't deny mount, so dm-verity can also mount on the top, which is similiar to the overlayfs case you mentioned. So there is nothing specific to deal with.