steam-runtime
steam-runtime copied to clipboard
ValveSoftware
pressure-vessel: Fall back to copying instead of hardlinks
pressure-vessel creates hardlinked copies of the runtime from steamrt v2 (Soldier) onwards for temporary operations.
These copies can be quite heavy in cases where the filesystem is networked, causing useless wait times for small operations (cephfs on HDDs takes up to 40 minutes for setting up the hardlinks). Choosing to have ./var as a tmpfs fails because hardlinks aren't allowed across different filesystems.
The container runtime should have a option to fall back to normal copies or use overlayfs instead to reduce the amount of operations required for these operations.
pressure-vessel already does fall back to copying if hard-linking fails (although not if it succeeds but is merely slow).
We can't really win here: none of the options for what to do with a non-local filesystem are great. I'd recommend using an ordinary Unix filesystem like ext4, btrfs or zfs for the Steam library that contains SteamLinuxRuntime_* if at all possible; that doesn't necessarily have to be the same filesystem that contains your actual games, although I suspect games and Proton are also not really tested or supported on networked filesystems.
Falling back to copying should work (with a warning Unable to create hard link...), but it's really slow on some filesystems, which is why a var symlink is automatically removed if present (to avoid people unintentionally setting up a very slow situation and reporting bugs about it). A var bind-mount or setting the PRESSURE_VESSEL_VARIABLE_DIR environment variable should work via the copy fallback, with a warning, at your own risk (in particular don't use a world-writeable directory like /tmp if you want to be secure against symlink attacks by a malicious local user).
overlayfs isn't yet supported by bubblewrap and can have some very weird behaviours as a result of it combining two dissimilar filesystems into one, leading to being unable to create hard-links within what appears to be the same directory, and safety mechanisms like rm -r --one-file-system not working as expected because the files in question no longer count as being on the same filesystem.
pressure-vessel already does fall back to copying if hard-linking fails
Oh, i've seemed to see multiple var/tmp.XXXXXX directories being created in the strace when i debugged this but it seems like my assumption is wrong.
I'd recommend using an ordinary Unix filesystem like ext4, btrfs or zfs for...
That's what i am currently doing right now to avoid this problem.
although I suspect games and Proton are also not really tested or supported on networked filesystems.
Most of them deal surprisingly well when there is posix feature compatibility but some seem to have a filesystem specific init like some source games with filesystem_*.so
Minding all this, wouldn't it be better to make APPID specific runtime directories like Proton does for wine prefixes? One could instead have the "expensive" copy operation just once and then work on top of that.
wouldn't it be better to make APPID specific runtime directories like Proton does for wine prefixes? One could instead have the "expensive" copy operation just once and then work on top of that
That would use a lot of disk space for one extra copy of soldier per installed game that you have ever run, instead of up to one temporary copy per game running right now; and we often wouldn't be able to optimize away the extra disk space and copying time by using hard-links or reflinks, because there's no guarantee that soldier is in the same Steam library as the game's compatdata directory.
That would also get us into dealing with cache invalidation, which is bad for reliability (it's often said to be one of the hardest things in computer science). The copy of soldier in var/tmp-* is not just a simple copy, it's a copy that has been edited to remove libraries that would cause incompatibilities with the ones we import from your host system - but the edits we need to make can change whenever you make changes to the installed libraries on the host system (most commonly during upgrades), so we have to detect that situation and invalidate the cached copy, with any failure to do that potentially breaking your games.
I've been thinking about some ways to avoid the need to actually copy anything by building up a very long bubblewrap command-line that separately bind-mounts every library except the ones we need to remove, but that's not going to be a quick change to make, and I'm not sure yet whether it's feasible (doing 1000+ separate bind-mounts could have its own performance problems).
I am encountering some issues related to this as well. I am trying to do something rather unique and looking for any advice.
My goal is to preserve DRM-free games I own onto read-only SD cards. One game per card, with some nice printed artwork. I am trying to make it such that the SD card contains everything the game needs to run, including the Steam runtime and Proton (if it is a Windows game).
I am trying to make this portable, so that you can run the games on almost any Linux system.
Furthermore, and this is where the trouble comes in, I am running the games with an overlayfs intended to capture the game save data. For native Linux games that work without the Steam runtime, this setup is working great.
For games requiring the Steam runtime though, there is a lot of superfluous data being captured by the overlay. Including what appears to be pressure vessel writing temporary files. Furthermore, startup time is severely compromised, often taking 30 seconds or even more.
I had thought it would be as simple as forcing pressure vessel to use the runtime as is since each copy of the runtime is for a single game only. But from the above description it sounds like the host system also plays a part in the ultimate composition of the runtime.
Using PRESSURE_VESSEL_VARIABLE_DIR works to avoid writes to the overlayfs, but as stated, causes the runtime to be copied which is not ideal either.
My understanding is overlayfs is supposed to deal with hard links efficiently, but it seems like the entire runtime as well as the runtime var directory is being almost completely rewritten and captured in the overlayfs. Maybe there is some overlayfs option I am missing.
@alkazar: The Steam Linux Runtime (SLR) and Proton are designed to be used in conjunction with Steam, with one copy of each major version of SLR and one copy of each major version of Proton, kept up-to-date by the Steampipe distribution network and shared between lots of games. Other use-cases are unsupported. The way you're hoping to use them is technically possible, but not what they've been designed for, and therefore not going to be efficient; also not necessarily sustainable in the long term. We keep old, unmodified games mostly-working on newer OSs by updating SLR and Proton, so that the effort required for those updates is centralized into SLR, rather than being distributed across all of your games.
My goal is to preserve DRM-free games I own onto read-only SD cards. One game per card, with some nice printed artwork. I am trying to make it such that the SD card contains everything the game needs to run, including the Steam runtime and Proton (if it is a Windows game).
If your objective is to be able to run these games 5 or 10 years in the future, on a Linux OS release that hasn't even started development at the time you're archiving them, you'll need to bear in mind that sometimes SLR or Proton needs to be updated to compensate for behaviour changes in the operating system - so you will likely need a way to swap in a newer SLR and/or a newer Proton that is compatible with the OS you'll be using in 10 years' time, even if there is also a fallback copy saved on your SD card to be used if no newer version is available.
For games requiring the Steam runtime though, there is a lot of superfluous data being captured by the overlay. Including what appears to be pressure vessel writing temporary files.
Yes, that'll happen. SLR is not designed to be read-only; it's designed to operate from your Steam library, which is required to be read/write anyway (because if it wasn't, Steam would be unable to keep it up to date).
If you use a separate overlayfs to use a tmpfs as an upper layer over the Steam Linux Runtime, that might help?
Furthermore, startup time is severely compromised, often taking 30 seconds or even more.
In configurations where it isn't possible to create hard links (like your overlayfs), having to read every file is probably unavoidable. A tmpfs would mitigate this by at least making the destination very fast.
There is no valuable user data stored below SteamLinuxRuntime{,_soldier,_sniper} - it's all either public files released by Valve, or transient temporary data that can be thrown away any time (except for while a game is running!) - so you can use a tmpfs for it without data loss.
I had thought it would be as simple as forcing pressure vessel to use the runtime as is since each copy of the runtime is for a single game only. But from the above description it sounds like the host system also plays a part in the ultimate composition of the runtime.
Yes, there are two things that look a bit like a container filesystem:
- a read-only, "immutable" runtime library stack (named
sniper_platform_3.0.20250519.130773or similar); - a temporary, writeable, mutable container filesystem (named
var/tmp-*) which is constructed on-demand by combining references to your host system's graphics driver stack with the "immutable" runtime
(Technically the runtime library stack is not 100% immutable: it gets its original permissions and timestamps applied to it the first time you run it, because Steampipe flattens the permissions of all files to 0755 and doesn't preserve timestamps.)
This has to be done because the user-space graphics drivers (Mesa or Nvidia) can't be bundled in SLR: a runtime library stack from 2019 or 2021 most likely won't support your 2025 GPU, and it certainly won't support your 2035 GPU when you try to play these games again in 10 years' time. The need to write to the filesystem is the price we pay for this modern GPU support.
If your games are old or simple enough to be playable with software rendering (no GPU acceleration) then you could use the OCI (Docker/Podman) images from https://gitlab.steamos.cloud/steamrt/scout/platform, https://gitlab.steamos.cloud/steamrt/soldier/platform and/or https://gitlab.steamos.cloud/steamrt/sniper/platform instead? But those don't include user-space graphics drivers that will work with a modern GPU (they're intended for developer and dedicated-server use-cases), and unlike SLR they also can't borrow user-space graphics drivers from your host system.
My understanding is overlayfs is supposed to deal with hard links efficiently, but it seems like the entire runtime as well as the runtime var directory is being almost completely rewritten and captured in the overlayfs.
Maybe the hard link is resulting in the actual file content being copied from the overlayfs' read-only lower filesystem to its upper filesystem - the jargon term is that this is a "copy-up" operation.
Or, when the container runtime framework sets the correct permissions on each file, maybe that is causing a copy-up.