Some games added as non-steam games not setting compat paths correctly with Proton.

[Okxa]

Your system information

• Steam Runtime Version: Soldier
• Distribution: Arch Linux
• Link to your full system information: https://gist.github.com/Okxa/0b9124ac2adc7950df1d61e740f2220b
• What compatibility tool are you using?: Proton 6.3-6.
• If you are using Steam Linux Runtime, or Proton 5.13 or newer: What versions are listed in SteamLinuxRuntime_soldier/VERSIONS.txt?

#Name	Version		Runtime	Runtime_Version	Comment
depot	0.20210907.26			# Overall version number
pressure-vessel	0.20210906.2			
scripts	v0.20210818.0-0-gc6be089			# Entry point scripts, etc.
soldier	0.20210906.1	soldier	0.20210906.1	# soldier_platform_0.20210906.1/

Please describe your issue in as much detail as possible:

First, the issue came up when I was trying to run Half Life alyx tools with proton, installed with steamcmd to a separate folder to keep the native linux version installed. For that I am using a custom proton (https://github.com/stevenlafl/Proton/releases/tag/proton-5.0-7-alyx-2), but that is only to get the tools to work.

But that is a version before 5.13, so I also tried proton-6.3-4, this issue is about non-steam games in general.

Running the exe game/bin/win64/hlvrcfg.exe with that proton version (older than 5.13), log reports the error: Proton: No compat data path?. It can be fixed by launching steam with env STEAM_COMPAT_DATA_PATH=/path/to/folder/ steam It did work previously though, so I am not sure that it is intended to be broken.

But anyways, I also tried to run the exe with proton-6.3-6, I got the error:
pressure-vessel-adverb[2868954]: E: Failed to execute child process "/mnt/HDD2/Pelit/Steam/steamapps/common/Proton 6.3/proton" (No such file or directory)
regardless if I have the STEAM_COMPAT_DATA_PATH defined or not.
I had to define STEAM_EXTRA_COMPAT_TOOLS_PATHS and then I got the error KeyError: 'STEAM_COMPAT_CLIENT_INSTALL_PATH'.

To fix both errors, I had to set the following env vars and launch steam with
env STEAM_COMPAT_CLIENT_INSTALL_PATH=~/.local/share/Steam STEAM_EXTRA_COMPAT_TOOLS_PATHS=/path/to/the/proton/install/location STEAM_COMPAT_DATA_PATH=/path/to/desired/compatdata/folder steam
to allow the non-steam game to run with proton.

Possible related issues: ValveSoftware/steam-runtime#287 #6475

Steps for reproducing this issue:

1. Add a non-steam windows game to steam
2. Set compability tool to some proton version after 5.13
3. Launch the game in steam.
4. It wont launch unless the aforementioned enviroment variables are set.

[RyuzakiKK]

Can you please just set the environment variable STEAM_LINUX_RUNTIME_LOG=1 (explained here https://github.com/ValveSoftware/steam-runtime/blob/master/doc/reporting-steamlinuxruntime-bugs.md#essential-information) and post the entire log?

If you follow these steps, are you able to reproduce this issue even with Minetest and Proton 6.3-6? I'm asking because it will be easier for us to reproduce instead of using Alyx tools.

[Okxa]

So the minetest works (even with the old proton-5.0-7-alyx-2). however, the alyx tools do not.

Here are both logs, run with proton 6.3-6

minetest log

alyx tools log

The difference is that alyx tools log does not report bind mounting the folders, unlike in the minetest log:

22:05:29.719863: pressure-vessel-wrap[2817143]: I: Bind-mounting STEAM_COMPAT_CLIENT_INSTALL_PATH="/home/asko/.local/share/Steam" from the current env as STEAM_COMPAT_CLIENT_INSTALL_PATH="/home/asko/.local/share/Steam" in the host
22:05:29.719910: pressure-vessel-wrap[2817143]: I: Bind-mounting STEAM_COMPAT_DATA_PATH="/home/asko/.local/share/Steam/steamapps/compatdata/4109900275" from the current env as STEAM_COMPAT_DATA_PATH="/home/asko/.local/share/Steam/steamapps/compatdata/4109900275" in the host
22:05:29.719947: pressure-vessel-wrap[2817143]: I: Bind-mounting STEAM_COMPAT_LIBRARY_PATHS="...:/mnt/HDD2/Pelit/Steam/steamapps:..." from the current env as STEAM_COMPAT_LIBRARY_PATHS="...:/mnt/HDD2/Pelit/Steam/steamapps:..." in the host
22:05:29.719977: pressure-vessel-wrap[2817143]: I: Bind-mounting STEAM_COMPAT_LIBRARY_PATHS="...:/mnt/SSD/Pelit/Steam/steamapps:..." from the current env as STEAM_COMPAT_LIBRARY_PATHS="...:/mnt/SSD/Pelit/Steam/steamapps:..." in the host
22:05:29.720009: pressure-vessel-wrap[2817143]: I: Bind-mounting STEAM_COMPAT_TOOL_PATHS="...:/mnt/HDD2/Pelit/Steam/steamapps/common/Proton 6.3:..." from the current env as STEAM_COMPAT_TOOL_PATHS="...:/mnt/HDD2/Pelit/Steam/steamapps/common/Proton 6.3:..." in the host
22:05:29.720040: pressure-vessel-wrap[2817143]: I: Bind-mounting STEAM_COMPAT_TOOL_PATHS="...:/mnt/SSD/Pelit/Steam/steamapps/common/SteamLinuxRuntime_soldier:..." from the current env as STEAM_COMPAT_TOOL_PATHS="...:/mnt/SSD/Pelit/Steam/steamapps/common/SteamLinuxRuntime_soldier:..." in the host

But what causes this? I'll reiterate that using proton for this setup worked fine some time ago (Around September 2020 what I recall, a bit later too.), so something has changed with steam.

Now the alyx tools are downloaded with SteamCMD (because I wanted an separate install of the windows version, and keep the native version installed in Steam), so it has it's own steamapps folder in the place where it has been downloaded, maybe that affects things? (However, removing this steamapps folder from there does not affect the result.)

I added the "/mnt/HDD2/Pelit/hl_alyx_tools_proton/game/bin/win64/hlvrcfg.exe", with launch options -steam -retail. Removing the -steam param does not change anything, so maybe it has something to do with the way I installed the tools. Readding the exe does not help.

One thing comes to mind though, from the log names. Alyx log has the Alyx AppID in it's name. Maybe steam gets confused somehow that the game which has native version currently installed is trying to use proton, and does not set the correct paths? Yet this worked before, so I think not intended?

[smcv]

The difference is that alyx tools log does not report bind mounting the folders

The container runtime tool gets these from environment variables. I think this must mean that for whatever reason, the Steam client (the proprietary part of Steam) is not setting the environment variables STEAM_COMPAT_CLIENT_INSTALL_PATH, STEAM_COMPAT_DATA_PATH, STEAM_COMPAT_LIBRARY_PATHS, STEAM_COMPAT_TOOL_PATHS before launching the compatibility tool to run this particular game/tool/program. If Steam does not do this, the compatibility tool cannot guess what their values should be (the purpose of these environment variables is to tell us what we need to know), so we are unable to correct for this.

Similarly, Proton needs to receive these paths as environment variables, and will not work if they are missing from the environment. We have a semi-official specification for the compatibility tool interface which we're trying to keep up to date with whatever Steam does.

If the Steam client is doing something wrong here, then this is out of scope for the Steam Runtime, and we'll need to move the issue report to https://github.com/ValveSoftware/steam-for-linux and get someone with Steam code access involved.

However, before we reassign anything, please could you describe steps that a developer could go through to reproduce the situation you have created (or any simpler situation that fails in the same way), making it as simple and as specific as possible? (For example, please use an official, supported version of Proton 6.3 or 5.13, rather than discussing unofficial versions that will make the report less clear.)

The steps you provided in the original issue report are sufficiently general that our test-case with Minetest, a genuinely non-Steam game (which worked for you) would also fit that description, which means those steps are not specific enough to guarantee that a non-working situation is reached - so they are probably not enough for a Steam developer to reproduce what you did and experience the issue themselves.

Now the alyx tools are downloaded with SteamCMD (because I wanted an separate install of the windows version, and keep the native version installed in Steam)

I think that might be part of the problem - Steam might "know" somehow that Alyx tools downloaded in this way are actually a Steam game that has been moved to a different place, and not genuinely a non-Steam game.

Alyx log has the Alyx AppID in it's name

That means the Steam client (the proprietary part) must have been able to figure out, somehow, that the program you are trying to run is part of app ID 546560, i.e. Half-Life: Alyx. If the Steam client thought it was genuinely a non-Steam game, it would have told us to use app ID 0 in the log filename, like we did for Minetest.

The container runtime infrastructure gets the app ID from the STEAM_COMPAT_APP_ID environment variable set by Steam, and uses that to decide how to name the log file.

[Okxa]

I tried running steam with env STEAM_COMPAT_APP_ID=0 steam, but the tools fail to launch with the same way as before.

But actually I found out why this happens. There is a file steam_appid.txt with the HL:Alyx AppID 546560 in it, which tells steam the appID of the running software.

If I set ID in the file to 0, the tools launch, however the appID in the file gets reset back to 546560 on launching the game. If the file is deleted it gets recreated. Making it read only does not allow the game to start. If I try any other appID, such as a native linux steam game or a steam game using the same proton version, proton fails the same way again, and the appID stays the same.

Furhermore, if I add steamappid.txt with anything other than the appID 0 to the minetest executable folder, it will fail to start too with the same errors.

So the issue seems to be that when launching a non-steam game, if Steam detects the appID of the started application is anything other than 0, it wont load the required proton variables.

please could you describe steps that a developer could go through to reproduce the situation you have created

Instructions

This can be tested with any official proton version, and minetest should be fine to test this with.

Setup:

1. Install manually any official proton version, such as 6.3-6 or Experimental (because non-steam games can't IIRC download proton versions.)
2. Add a non-steam game to your library, such as minetest (minetest-5.4.1-win64/bin/minetest.exe)
3. Set the shortcut to use the Proton you downloaded.
4. Launch the game using the shortcut.
5. Game launches.

To replicate this issue:

1. Go to the folder minetest-5.4.1-win64/bin/ and create a text file there called steam_appid.txt, with the contents of 12345
2. Try launching the game again using the same shortcut.
3. The game will fail to launch.
4. Change the number 12345 to 0 in the steam_appid.txt -file.
5. Try launching the game again using the same shortcut.
6. The game launches again.

It will either fail with pressure-vessel-adverb[1852509]: E: Failed to execute child process "/mnt/HDD2/Pelit/Steam/steamapps/common/Proton - Experimental/proton" (No such file or directory)
or if it's an older proton version (probably <5.13), with Proton: No compat data path?

[smcv]

So the issue seems to be that when launching a non-steam game, if Steam detects the appID of the started application is anything other than 0, it wont load the required proton variables.

OK, that makes sense: it's setting environment variables that would be appropriate for the Steam app-ID that the game says it has, instead of the set that would be appropriate for this specific non-Steam game.

@kisak-valve, I think we can move this to steam-for-linux and hand over to someone with Steam client code access?

[Okxa]

OK, that makes sense: it's setting environment variables that would be appropriate for the Steam app-ID that the game says it has, instead of the set that would be appropriate for this specific non-Steam game.

And I'll further add, it fails even if it's seemingly random bogus ID. (So Steam does not check specifcally that the ID is for valid game)