Always store compatdata in the internal Steam library

[Damglador]

Your system information

• Steam client version: 1745876290
• Distribution: Arch Linux
• Opted into Steam client beta?: No
• Have you checked for system updates?: Yes
• Steam Logs: [generate by running this command in a terminal tar -zcvf ~/Desktop/steam-logs.tar.gz ~/.steam/steam/logs]
• GPU: Nvidia/AMD

Please describe your issue in as much detail as possible:

When initializing Proton, Steam client passes to it variable STEAM_COMPAT_DATA_PATH which defines where compatdata should be stored. For games installed on an external library, this variable points to that external library.

compatdata stores Wine prefix, which can't be shared between two users. If Wine prefix is owned by user1, user2 will not be able to use it, even if they have permissions to. And even if involving solutions like bindfs to bind the library to other place with changed permissions, it'll cause issues with game data, including saves and may even wipe data in Steam cloud.

That makes just installing all games on external drives and letting other users on the same system use them without installing them twice, wasting a lot of space, require overengineered solutions like using bindfs to duplicate the library in other place and mount a partition or btrfs subvolume over compatdata in that place and then add all that in fstab for persistence.

On Windows game data is stored per user, in their home directory, so Proton compatdata should just follow that and always use ~/.steam. Solutions like https://github.com/ValveSoftware/Proton/pull/4861 will make compatdata of both users accessible to one another, which can be a privacy and/or security issue, and will also cause data loss in case the drive with game installations dies. I think most users expect their big drive with just installation of games to contain just installation of games, and not their valuable saves from games.

Upsides of moving compatdata to ~:

• compatdata of each user is private
• compatdata is stored with other user data
• External Steam libraries can be easily shared between two local users on the same system

Downsides:

• It takes considerable amount of space

In my opinion, upsides outweigh downside, especially considering that the space penalty would also apply on Windows, because the game data is always stored on the main drive.

TLDR: compatdata for games is stored in .../SteamLibrary/steamapps/compatdata/ (the library where the game is installed), but instead should always be stored in ~/.steam/steam/steamapps/compatdata to avoid Wine conflicts, privacy issues and potential data loss.

Steps for reproducing this issue:

1. Create a library on external drive as user1
2. Install a game that uses Proton on it as user1
3. Launch the game as user1, perhaps change settings or create a save, then quit it
4. Log out of the session
5. Log in a session as user2
6. Start Steam as user2
7. Add the same drive as external library
8. Try to launch the game from the external library

[helix26j]

I'm not sure that is the best solution as that would be the entire prefix for every game. Not being able to choose to store this data on the external disk could cause issues for games that download their own additional content.

There are some games such as Microsoft Flight Simulator which download 200+ GB to the user's AppData/Roaming directory. The Cities Skylines 2 DLC downloads to the user's AppData/LocalLow directory (currently over 34 GB). This doesn't include the additional space taken up by save games.

[Damglador]

@helix26j that would also be an issue on Windows. Besides, users should still be able to manually set STEAM_COMPAT_DATA_PATH to wherever they want. Considering these are more of niche cases, it'll be much easier to set that for a couple of games if you really want to save space in home rather than having to do the filesystem black magic or patching every version of Proton for every user, because otherwise your game will get bricked, or Steam cloud will corrupt or delete your game data. It would be much easier if one could just start Steam with STEAM_COMPAT_DATA_PATH set to somewhere else, but STEAM_COMPAT_DATA_PATH points to /home/Games/SteamLibrary/steamapps/compatdata/<appid>, and not /home/Games/SteamLibrary/steamapps/compatdata (at least from what I remember), so it has to be set for each game separately

So between having 20 (my current compatdata size) additional GiB in my home and having to mount a btrfs subvolume with the library userN amount of times to overlay the compatdata in each one with another user-specific subvolume just to make the games work, I pick having 20 additional GiB in my home. Not even including patching Proton, because it requires too much maintenance. That's not counting the fact that files created by Steam don't inherit permissions from the parent folder, which requires acl permission tweakery to fix.

As for https://github.com/ValveSoftware/Proton/pull/4861, it's either a security nightmare or even more complex permission configuration to not let the other user to just wipe your save files. And it doesn't count for moving the drive to another system, because it uses uids to name folders, it'll mess up saves if user1 is 1000 on the original system, but 1001 on another.

[Lyssers]

No, please do not muck up my home partition or OS drive with Compatdata.

That's the only reason steam is usable for me. It's bad enough there's all sorts of crap Windows puts on the C:\ drive other than just the OS and program executables and configs, which is all that actually belongs there, if software on Linux did the same idk I'd probably just stop gaming.

For instance Oculus software on Windows is unusable because it downloads entire 3D environments into my OS drive, made me fork out the $20 for virtual desktop.

I have 6TB of SSD space on my system some of which is used for games, but only 120GB of which is my OS drive and root partition, 40GB of which is my home partition, as it is meant to be, storing compatdata there would eat that up in minutes and render my computer unusable for no reason.

The drive is almost always full, because all that belongs there is the OS and post-install of the OS, very few things should be written there apart from some small executables and configs.

Actually storing save data alongside games is how it should be, instead of wherever if fancies a dev, it's actually quite a nice side effect of compatdata stored alongside the game in the library, rather than anywhere between C:\Users%random% or ~/.dotfile or ~/.config/%randomfile%.

If we must have per-user compatdata, then please just add a per-user directory to the /mnt/shareddrive/steamapps/compatdata/%userid% to the directory.

The cybersec implications are no concern of steam. If anything sensitive is stored in save data like account passwords in plaintext, that's on the game developer and shouldn't be stored there in that manner.

It's not steam's concern or job to clean up sloppy data storage by arbitrarily enforcing home partition slaughter on unsuspecting end-users. Save file encryption is a common practice just for such a reason.

[Damglador]

@Lyssers just symlink the compatdata folder to your bigger drive and the problem is solved.

This is multiple times easier than solving the issue without moving the compatdata to the home folder.

Move the compatdata folder wherever you want and then

ls -s /path/to/compatdata ~/.steam/steam/steamapps/compatdata or do it using your file manager

This will work even better than what we have now, because ALL compatdata will be on the remote location, instead of being distributed between the home folder and the library based on where a particular game is installed.

[etyarews]

+1 for this.

It is incredibly difficult to share a computer with multiple users due to this weird behavior. I cannot simply share a library between Linux Users, I need to do a whole song and dance with permissions and symlinks to get it to work so that the compatdata isn't shared between users.

This also makes it a pain to set up removable storage as a way to share the same library between multiple computers because there is no way.

I would also like to add that Steam should have their own user/group in the background. All Steam Libraries should be set up to be owned by Steam, not the user that downloaded it. Again, this makes sharing the same library a pain because the file permissions are wrong and a different user cannot update a game because they do not have the write permission for that.

[mathew2214]

I would also like to add that Steam should have their own user/group in the background. All Steam Libraries should be set up to be owned by Steam, not the user that downloaded it. Again, this makes sharing the same library a pain because the file permissions are wrong and a different user cannot update a game because they do not have the write permission for that.

what a great idea! wouldnt it be fantastic if we could Daemonize the steam backend? and then the userspace applications simply interact with it with some sort of API that the community could make any number of frontends for.

[etyarews]

I would also like to add that Steam should have their own user/group in the background. All Steam Libraries should be set up to be owned by Steam, not the user that downloaded it. Again, this makes sharing the same library a pain because the file permissions are wrong and a different user cannot update a game because they do not have the write permission for that.

what a great idea! wouldnt it be fantastic if we could Daemonize the steam backend? and then the userspace applications simply interact with it with some sort of API that the community could make any number of frontends for.

My ultimate dream is for Steam Library to be standardized among different users, computers and distros, in a way that you could install a game on a removable device, plop into another computer and Steam would be able to use it without hitch. Right now you can do that, but there are three caveats:

1. File permissions issues that I mentioned
2. Steam scans for newly added devices, but only if that device was previously set-up
3. Steam can figure out when a device is plugged in, but not when it's plugged off. If you add a library to an external drive and reboot the computer, Steam will correctly show that the game isn't installed, and it will automatically show up as installed the moment you add the drive back, but it will be shown as installed until you reboot the computer.

I totally understand there are security concerns, but frankly I think there would be easier ways to attack a computer than praying a user picks an external drive and play a game through Steam. Although that does sounds like good creepy pasta material

[dannymate]

Just lending my voice to the need for the ability for multiple users on the same PC to utilise the same game installs. It's semi-niche I'll admit but it makes logical sense that it should work. I have spoken to people where this is a deal breaker for them using Linux.

[eilimrib]

This should be implemented already. There's literally no reason for Steam to keep overwriting other user's prefixes or being unable to run because you don't own them.

[Damglador]

With current behaviour to actually share a library (unless you want to play only Linux games), I had to

1. Create a (btrfs) subvolume for the game library
2. Mount the subvolume for each user (because it's impossible to share one mount)
3. Create a compatdata subvolume for each user
4. Mount compatdata subvolumes over compatdata in respective library subvolumes
5. Add the mounts as external library to respective Steam clients (one mount should only be accessible by one user)

So in the end I have:

This is the only reliable workaround I've found. This excludes regular group and setfacl shenanigans for sharing the main game files. It shares all the library files except compatdata (as it should be). I've wasted way too much time to come up with this workaround.

The group and setfacl shenanigans guide

Just in case, gonna leave a guide here.

Before doing the steps, move compatdata out of the library, because it's user data and should keep its permissions.

1. To distribute access to the library, I created a group steamlibrary,
2. recursively chmoded the library so users in group can r+w (chmod -R 775 /home/Games/SteamLibrary),
3. recursively set group owner of files to steamlibrary (chown -R $(whoami):steamlibrary /home/Games/SteamLibrary),
4. recursively for all folders setgid bit so new files and folders inherit group owner (find /home/Games/SteamLibrary -type d -exec chmod g+s {} +), (gid bit on files does a different thing, that's why it's not just chmod -R)
5. recursively set ACL so new files and folders inherit r+w permission for group owner (setfacl -R -d -m u::rwX,g::rwX,o::rX /home/Games/SteamLibrary)

Everything is recursive because I did this on my existing library. Do this from the user account of who created the library and replace /home/Games/SteamLibrary with path to your external library. Then add users who should have access to the library to the steamlibrary group

So, I don't think this should be labelled as a «Feature Request», because current behaviour clearly breaks something that should just work.

[etyarews]

@Damglador I will copy a comment I made here. I figured out my setup using symlinks, the only issue is that new games aren't automatically installed, each user needs to "re-install" it, which just boils down to Steam verifying it:

Each user has three steam libraries:

/home/USER1/.local/share/steam - Default Steam Install, Proton and Steam Runtimes are installed here /games/USER1/Personal - Sorta of a personal stash of games, it's usually used for games where one user wants certain mods, but another doesn't. Each one have their own copy with different mods. /games/USER1/Shared - This is the one with heavy symlinks

Alright, so let's look at /games/USER1/Shared: It has the usual steamapps folder, let's see how it works we see how it works:

/games/USER1/Shared/steamapps/common -> /games/SHARED LIBRARY/Steam/steamapps/common /games/USER1/Shared/steamapps/downloading -> /games/SHARED LIBRARY/Steam/steamapps/downloading /games/USER1/Shared/steamapps/shadercache -> /games/SHARED LIBRARY/Steam/steamapps/shadercache /games/USER1/Shared/steamapps/temp -> /games/SHARED LIBRARY/Steam/steamapps/temp /games/USER1/Shared/steamapps/compatdata -> /games/USER1/Personal/steamapps/compatdata

Every folder is being symlinked to the respective SHARED LIBRARY folder. Except compatdata, that one is symlinked to their Personal library /games/USER1/Personal

Now, on the /games/USER1/Personal/steampps folder, we only have two symlinks:

/games/USER1/Personal/steamapps/downloading -> /games/SHARED LIBRARY/Steam/steamapps/downloading /games/USER1/Personal/steamapps/shadercache -> /games/SHARED LIBRARY/Steam/steamapps/shadercache

downloading and shadercache are symlinked to the shared library, so all users have the same shadercache and downloading status, which should at least keep Steam from downloading the same game twice, and should help the madness that is shadercache from going out of control (it still goes out of control).

Now, here's the problem: The entire /games folder and subfolders need to have permissions for all users, including new files. I did that by adding all users to a group called gamers and setting it up like so:

sudo chown -R root:gamers /games sudo setfacl -R -m g:gamers:rwx /games sudo setfacl -R -m mask::rwx /games sudo setfacl -R -d -m g:gamers:rwx /games sudo setfacl -R -d -m mask::rwx /games

However, wine really wants the prefixes to be owned by the person who is executing it, so you need to make it so that each compatdata is owned by a specific user, basically you need to undo what you've done on the previous step on only the compatdata folder.

sudo chown -R user1:user1 /games/USER1/Personal/steamapps/compatdata sudo setfacl -b -R /games/USER1/Personal/steamapps/compatdata sudo setfacl -m d:group::--- /games/USER1/Personal/steamapps/compatdata

sudo chown -R user2:user2 /games/USER2/Personal/steamapps/compatdata sudo setfacl -b -R /games/USER2/Personal/steamapps/compatdata sudo setfacl -m d:group::--- /games/USER2/Personal/steamapps/compatdata

sudo chown -R user3:user3 /games/USER3/Personal/steamapps/compatdata sudo setfacl -b -R /games/USER3/Personal/steamapps/compatdata sudo setfacl -m d:group::--- /games/USER3/Personal/steamapps/compatdata

I barely understand permissions and I pretty much only have a vague understanding of what each command is doing, but hey, it works. I do recommend you writing this list of commands down because if you ever needed to copy a compatdata folder to another user, you need to re-run all of this

[Damglador]

the only issue is that new games aren't automatically installed, each user needs to "re-install" it

This is why the btrfs shenanigans are the only normal solution. It just works like it should've worked.

I've tried doing it with symlinks at the beginning, but it requires too much manual labour. (too much for me is anything past the initial install).

The only downside of the btrfs solution is the requirment of btrfs. Thought it might not be. If other partitions allow two mounts (they do, I can mount root and boot inside my home), compatdata can be overlayed with mount --bind (for example mount --bind $HOME/.local/share/Steam/steamapps/compatdata/ /home/Games/SteamLibrary/steamapps/compatdata, which imo is even better than a dedicated compatdata subvolume). But this won't work if library is duplicated with a symlink or a mount --bind, it'll overlay the compatdata on source library as well.

Something like that:

[etyarews]

Hmm, your solution is creative, and does fix the need to redownload the game. (Although your second illustration is way easier to understand than the btrfs one, I recommend creating a separate @subvol for user1 compatdata to make it easier to understand what is going on)

I do notice that .vcf files have a "last played" tag, does this mean user2 list of recent games has the last game that user1 played, if user1 played right before user2 started the session?

[Damglador]

Yeah, the first diagram was made for my personal setup, that's why it the way it is (I, the user1, use the original compatdata without overlaying it). The second one applies to btrfs as well, so I'll probably just use it instead.

User2's list of recently played isn't affected, neither is "Last played" for individual games.

[meysq]

Has any progress been considered on this issue? This is the last thing on my list keeping me from switching my gaming PC over to Linux, I would love for this to be implemented!

[Saiv46]

Related: #6958 #7158

No progress sadly