steam-runtime icon indicating copy to clipboard operation
steam-runtime copied to clipboard

Game is unable to see workshop contents

Open grappas opened this issue 3 years ago • 10 comments

Compatibility Report

  • Name of the game with compatibility issues: Arma 3
  • Steam AppID of the game: 107410

System Information

  • GPU: RX6700XT
  • Driver/LLVM version: mesa 22.1.4-1 xf86-video-amdgpu 22.0.0-1
  • Kernel version: 5.18.16-arch1-1
  • Full system information: https://gist.github.com/grappas/586f85acaef66226f267aea0d0b38ae3
  • Proton version: Proton Experimental

I confirm:

  • [x] that I haven't found an existing compatibility report for this game.
  • [x] that I have checked whether there are updates for my system available.

steam-107410.log

Symptoms

Launcher is unable to find mods on the disk drive. It sees only contents of a pfx's drive_c, home directory and nothing else.

Reproduction

STEAM_COMPAT_CLIENT_INSTALL_PATH=/home/grappas/.local/share/Steam STEAM_COMPAT_DATA_PATH=/home/grappas/.local/share/Steam/steamapps/compatdata/107410 /home/grappas/.local/share/Steam/steamapps/common/Proton\ -\ Experimental/proton run "C:"

Before launching the game:

obraz

After launching the game:

obraz

Hiding system files from game makes sense, but at least give it chance to find it's own mods ;)

Dayz has the very same issue.

grappas avatar Aug 05 '22 15:08 grappas

STEAM_COMPAT_CLIENT_INSTALL_PATH=/home/grappas/.local/share/Steam STEAM_COMPAT_DATA_PATH=/home/grappas/.local/share/Steam/steamapps/compatdata/107410 /home/grappas/.local/share/Steam/steamapps/common/Proton\ -\ Experimental/proton run "C:"

This is unsupported. The only supported way to launch Proton 5.13+ is to let Steam run it, using the "Steam Linux Runtime - soldier" container runtime. It is not designed to run directly on the host system.

Before launching the game

Do you mean that this is the result of running Proton without using the container runtime, using the unsupported command quoted above?

After launching the game

Do you mean that this is the result of opening a file browser from within the game, having launched the game via Steam? Or do you mean something else?

Launcher is unable to find mods on the disk drive. It sees only contents of a pfx's drive_c, home directory and nothing else.

Please give specific examples of the paths that you believe the game should be able to access, which of those paths it can see, and which ones it can't (if some mods work and some don't, one example of a mod that works and one example of a mod that doesn't work is enough). Also, if any of these paths involve symbolic links to another location, as either the path itself or one of its parent directories, please explain (the namei command is a useful way to show this information).

For instance, you might say something like this:

I expected the game to be able to access /home/me/SteamLibrary/steamapps/common/Arma 3/Workshop/@Foo/files/... and /home/me/SteamLibrary/steamapps/common/Arma 3/Workshop/@Bar/files/..., but only @Foo appears in the file browser.

$ namei /home/me/SteamLibrary/steamapps/common/Arma 3/Workshop/@Foo/files
f: /home/me/SteamLibrary/steamapps/common/Arma 3/Workshop/@Foo/files
 d /
 d home
 d me
 l SteamLibrary -> /media/large-disk/SteamLibrary
  d /media
  d large-disk
  d SteamLibrary
 d steamapps
 d common
 d Arma 3
 d Workshop
 d @Foo
 d files
$ namei /home/me/SteamLibrary/steamapps/common/Arma 3/Workshop/@Foo/files
f: /home/me/SteamLibrary/steamapps/common/Arma 3/Workshop/@Bar/files
 d /
 d home
 d me
 l SteamLibrary -> /media/large-disk/SteamLibrary
  d /
  d /media
  d large-disk
  d SteamLibrary
 d steamapps
 d common
 d Arma 3
 d Workshop
 l @Bar -> /media/large-disk/Arma mods/Bar
  d /
  d /media
  d large-disk
  d Arma mods
  d Bar
 d files

smcv avatar Aug 05 '22 17:08 smcv

The reason I ask about symbolic links is that I suspect you are seeing something similar to https://github.com/ValveSoftware/steam-runtime/issues/470 and https://github.com/ValveSoftware/steam-runtime/issues/334. Because we're using a container, directories are only shared with the container that runs the game if the container runtime specifically asks for them to be shared. The container runtime knows the Steam library is needed, because Steam gives it that information, so it shares that directory automatically. However, it won't search the shared directory for symbolic links and also share their targets, because that would be really slow for large directory trees (particularly on older hard drives). This means the symbolic link ends up pointing to nothing, and Wine handles that by behaving as though it didn't exist at all.

The beta version of the container runtime shares more directories by default, so if you are using symbolic links, please try that beta and see whether it improves the situation.

If you have symbolic links pointing into custom locations like /storage or /games, those will not be shared by default, even with the beta linked there. However, you can share extra locations by setting the game's Launch Options to something like

STEAM_COMPAT_MOUNTS=/storage:/games:/other-stuff %command%

or by setting this environment variable globally for Steam, for instance by running it as STEAM_COMPAT_MOUNTS=/storage:/games:/other-stuff steam.

smcv avatar Aug 05 '22 17:08 smcv

namei

$ namei /run/media/windows/D/SteamLibrary/steamapps/common/Arma\ 3/\!Workshop/@vurtual\'s\ Car\ Seat\ \&\ Stretcher
f: /run/media/windows/D/SteamLibrary/steamapps/common/Arma 3/!Workshop/@vurtual's Car Seat & Stretcher
 d /
 d run
 d media
 d windows
 d D
 d SteamLibrary
 d steamapps
 d common
 d Arma 3
 d !Workshop
 l @vurtual's Car Seat & Stretcher -> ../../../../../../D/SteamLibrary/steamapps/workshop/content/107410/1381327410
   d ..
   d ..
   d ..
   d ..
   d ..
   d ..
   d D
   d SteamLibrary
   d steamapps
   d workshop
   d content
   d 107410
   d 1381327410

For instance, you might say something like this:

I expect container to have access to it game files as well as to it's workshop contents i.e.:

/run/media/windows/D/SteamLibrary/steamapps/common/Arma\ 3
/run/media/windows/D/SteamLibrary/steamapps/workshop/content/107410

It's reasonable to assume that container should have access to it.

grappas avatar Aug 05 '22 17:08 grappas

Thanks, that level of detail helps a lot.

/run/media/windows/D/SteamLibrary/steamapps/common/Arma\ 3
/run/media/windows/D/SteamLibrary/steamapps/workshop/content/107410

It's reasonable to assume that container should have access to it.

I'm surprised it doesn't: Steam is meant to tell us about all the libraries involved in launching this game via the STEAM_COMPAT_LIBRARY_PATHS environment variable, which should include /run/media/windows/D/SteamLibrary, resulting in that directory being shared.

Please could you get a detailed debug log from the container runtime framework? To do that, set your game's launch options to:

STEAM_LINUX_RUNTIME_LOG=1 STEAM_LINUX_RUNTIME_VERBOSE=1 %command%

then launch the game, and exit from the game again. You should get a log recorded in SteamLinuxRuntime_soldier/var/slr-latest.log which will give more details of the information that's coming from Steam. Please provide that log file as an attachment or a gist.

SteamLinuxRuntime_soldier will be in one of your Steam library folders, but I can't tell which one from the information that's here so far.

(@kisak-valve, if you happen to see this, this issue can be moved to ValveSoftware/steam-runtime: it's more steam-runtime than Proton.)

smcv avatar Aug 05 '22 18:08 smcv

Because the path you're interested in is in /run/media, using the beta version of "Steam Linux Runtime - soldier" would probably avoid the problem for you (more details) - but please provide the verbose log I asked for in the previous comment, either with the default version or the beta version, so we can understand why this didn't already work.

smcv avatar Aug 05 '22 18:08 smcv

https://gist.github.com/grappas/444b9dd3df4e5a8a5e52625f172b157a

I've tried wtth STEAM_COMPAT_MOUNTS=/run/media/windows/D/SteamLibrary/steamapps/workshop/content/107410:/home/grappas/.local/share/Steam/steamapps/workshop/content/107410 %command%

Now folders are seen by explorer, but launcher still has the issues with mods (don't recognize if are already downloaded, or corrupted).

  • [x] EDIT: sorry that's me using proton tricks. After getting rid of excess drives in winecfg mods are properly recognized.

grappas avatar Aug 05 '22 20:08 grappas

Steam thinks the game is installed in ~/.local/share/Steam/steamapps:

22:22:53.829736: pressure-vessel-wrap[281313]: D: 	'STEAM_COMPAT_INSTALL_PATH=/home/grappas/.local/share/Steam/steamapps/common/Arma 3

and as far as I know, workshop content always lives in the same library as the game itself; but Steam also thinks the game is installed in /run/media/windows/D/SteamLibrary:

22:22:53.829601: pressure-vessel-wrap[281313]: D: 	'PWD=/run/media/windows/D/SteamLibrary/steamapps/common/Arma 3'

So I'm confused, or Steam is confused, or both.

I think this is because it exists in both libraries, but one copy is a symlink to the other:

22:22:54.322735: pressure-vessel-wrap[281313]: D: /home/grappas/.local/share/Steam/steamapps/common/Arma 3 is a symlink, resolved to /run/media/windows/D/SteamLibrary/steamapps/common/Arma 3/
22:22:54.322742: pressure-vessel-wrap[281313]: D: Trying to export the target instead: /run/media/windows/D/SteamLibrary/steamapps/common/Arma 3/

STEAM_COMPAT_MOUNTS=/run/media/windows/D/SteamLibrary/steamapps/workshop/content/107410:/home/grappas/.local/share/Steam/steamapps/workshop/content/107410

Are those physically different locations? Do you perhaps have half your mods in one place and half your mods in the other?

I think perhaps what has happened here might be that you moved the game from one library to the other "behind Steam's back", which only moved the actual game installation, and not the other files that should follow it around (workshop content, Wine prefix, shader cache)? The recommended way to move games between drives is to have a Steam library on each drive, and use Steam's built-in library management, which should move all the local files associated with the game as well as the game itself.

There isn't much that the container runtime can do to solve this, beyond what was done in #470. If the Steam client told us about all your Steam libraries (not just the one that it thinks the game is installed in), then that would ensure they were all visible, but at the moment we don't get that information.

smcv avatar Aug 05 '22 23:08 smcv

the game from one library to the other "behind Steam's back".

Answer is yes, but don't tell kissak, gonna be mad ;). Since I use both Linux and Windows from time to time this is my disk space precaution, to have one installation for both systems.

Since Soldier runtime has no problem following symlink to original game folder, why it not following symlink to workshop content folder by it's own (which I also made)?

Although Arma 3 workshop content is properly recognized by launcher and now works without a problem, unfortunately for DayZ problem remains and I don't have any clue how to resolve that.

https://gist.github.com/grappas/6b3846c5a31efa5d99efdf4a64bd5cab

It clearly has access to workshop path: obraz

grappas avatar Aug 06 '22 07:08 grappas

Since Soldier runtime has no problem following symlink to original game folder, why it not following symlink to workshop content folder by it's own (which I also made)?

Because the container runtime has no way to know that there might be symlinks in the workshop content that point to the other library, and no other way to know that the other library might be relevant. The container runtime doesn't know where all your games are installed, and it doesn't know a complete list of your Steam libraries; it only knows about the games and libraries that the Steam client tells it about.

We're not going to add code to search each shared directory for symbolic links and automatically share all of their targets, because that would slow down launching every game for every user, particularly on magnetic hard disks. On one of my test machines with a not-particularly-large Steam library (150G), searching my Steam library for symbolic links with find ~/SteamLibrary -type l took nearly 2 minutes. We don't want to slow down every game launch by 2 minutes!

If the Steam client told the container runtime about every Steam library (not just the ones that it thinks are involved in launching this particular game), then it would share them all. The Steam client doesn't currently do that, because if you move games around in the way that is supported, it wouldn't be necessary.

The beta version of the soldier runtime automatically shares all of /run/media with the game, which would solve the problem for your particular installation locations. Please try it.

unfortunately for DayZ problem remains

I can't do anything with "problem remains". If you want this to be solved, please describe the situation in more detail: where your DayZ installation, DayZ mods and anything those mods depend on are physically installed, and what symbolic links are pointing to them.

smcv avatar Aug 08 '22 11:08 smcv

Hi! That mention was me being directed to this issue for the game in question, so I suppose this is more relevant for my case.

I only recently found this to be an issue with Black Ops 3; though BO3 does have a local mods/usermaps workaround (that does require mangling workshop contents somewhat to conform to a different standard, and also makes mods' saves read/write to a different directory), the problem was still quite bothersome to overcome.

Unfortunately, in my case (on Arch), I have my games installed on a secondary internal drive and partition mounted as a subdirectory in /mnt, meaning my games' and workshop mods' installations are in roughly /mnt/Name/SteamGames/steamapps/..., so even the beta Soldier runtime mentioned wouldn't account for this.

I really can't pretend to know the ramifications of adding more whitelisted directories to search for, and it's not like I'd be aware of a better solution for this. I'm just adding my own input here.

SeongGino avatar Aug 16 '22 05:08 SeongGino

Unfortunately, in my case (on Arch), I have my games installed on a secondary internal drive and partition mounted as a subdirectory in /mnt, meaning my games' and workshop mods' installations are in roughly /mnt/Name/SteamGames/steamapps/..., so even the beta Soldier runtime mentioned wouldn't account for this.

The beta soldier runtime linked above should be OK for this case: it shares /mnt and everything "below" /mnt by default. Please try it?

Or, if that is not sufficient, please describe your situation in detail, and attach logs. I can't do anything with "a different directory" or "a local mods/usermaps workaround", I need to know specifics before I can usefully change anything.

The only situations that the beta runtime does not address are those where required files are physically located in a custom top-level directory that is not shared by default (/storage, /games, etc.), and the the game is accessing them via symbolic links in a directory that is shared by default, and the required files are not in any of the locations that Steam tells the runtime about (in particular, Steam tells the runtime a list of some of your Steam library directories, and the runtime shares those automatically).

smcv avatar Aug 16 '22 10:08 smcv

The beta soldier runtime linked above should be OK for this case: it shares /mnt and everything "below" /mnt by default.

Ah, I did not know this when reading your earlier post prior to my own. The reason being because of you saying:

The beta version of the soldier runtime automatically shares all of /run/media with the game...

...and thus I assumed that it was exclusively /run/media since that was all that was mentioned, and wasn't aware of any other exceptions being made. I know that might be a stretch to say, but forgive me for my slow learning, considering I'd only just learned that the Soldier Runtime had a beta branch yesterday.

Regardless, I just now had the time to check this with the beta branch, and it does in fact seem to work for my setup now. I appreciate the clarification.

SeongGino avatar Aug 17 '22 05:08 SeongGino

Because the path you're interested in is in /run/media, using the beta version of "Steam Linux Runtime - soldier" would probably avoid the problem for you

The beta change described in #470 was promoted to the default branch today.

smcv avatar Aug 25 '22 15:08 smcv

I have retested and can confirm, that the default branch of Soldier Runtime works with at least Call of Duty: Black Ops III Workshop mods: image

On my machine, both the game and the workshop are installed in a Steam directory under /media/[user]/[UUID]/SteamLibrary/[standard Steam Directory structure], using the Steam Client beta, but I did revert to the stable branch of the Soldier Runtime itself.

SteveHeist avatar Sep 02 '22 13:09 SteveHeist

Closing as fixed.

kisak-valve avatar Oct 08 '22 17:10 kisak-valve