Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SteamCMD download error #389

Closed
voruti opened this issue Jan 17, 2025 · 22 comments · Fixed by #394
Closed

SteamCMD download error #389

voruti opened this issue Jan 17, 2025 · 22 comments · Fixed by #394
Labels
bug Something isn't working environment issue Issue caused by environmental factors, not by the image itself

Comments

@voruti
Copy link
Contributor

voruti commented Jan 17, 2025

The game won't start (see logs below).

Docker Compose File

name: satisfactory
services:
  satisfactory:
    deploy:
      resources:
        limits:
          memory: "17179869184"
        reservations:
          memory: "4294967296"
    environment:
      AUTOPAUSE: "false"
      AUTOSAVEINTERVAL: "1200"
      MAXPLAYERS: "4"
      PGID: "1000"
      PUID: "1000"
      ROOTLESS: "true"
      STEAMBETA: "false"
      TIMEOUT: "60"
      TZ: Europe/Berlin
      VMOVERRIDE: "false"
    hostname: satisfactory-server
    image: wolveix/satisfactory-server:v1.9.1
    labels:
      wud.tag.include: ^v\d+\.\d+\.\d+$$
    networks:
      default: null
    ports:
      - mode: ingress
        target: 7777
        published: "7777"
        protocol: udp
      - mode: ingress
        target: 7777
        published: "7777"
        protocol: tcp
    restart: always
    user: 1000:1000
    volumes:
      - type: bind
        source: /etc/localtime
        target: /etc/localtime
        read_only: true
        bind:
          create_host_path: true
      - type: bind
        source: /etc/timezone
        target: /etc/timezone
        read_only: true
        bind:
          create_host_path: true
      - type: bind
        source: /home/user/composefiles/gaming/satisfactory/data
        target: /config/saved
        bind:
          create_host_path: true
      - type: volume
        source: satisfactory-gamefiles
        target: /config/gamefiles
        volume: {}
networks:
  default:
    name: satisfactory_default
volumes:
  satisfactory-gamefiles:
    name: satisfactory_satisfactory-gamefiles

Debug Output

satisfactory-1  | ===== Satisfactory Server 1.9.1 =====
satisfactory-1  | https://github.com/wolveix/satisfactory-server
satisfactory-1  |
satisfactory-1  | Debugging enabled (the container will exit after printing the debug info)
satisfactory-1  |
satisfactory-1  | Printing environment variables:
satisfactory-1  | declare -x AUTOPAUSE="false"
satisfactory-1  | declare -x AUTOSAVEINTERVAL="1200"
satisfactory-1  | declare -x AUTOSAVENUM="5"
satisfactory-1  | declare -x DEBIAN_FRONTEND="noninteractive"
satisfactory-1  | declare -x DEBUG="true"
satisfactory-1  | declare -x DISABLESEASONALEVENTS="false"
satisfactory-1  | declare -x GAMECONFIGDIR="/config/gamefiles/FactoryGame/Saved"
satisfactory-1  | declare -x GAMESAVESDIR="/home/steam/.config/Epic/FactoryGame/Saved/SaveGames"
satisfactory-1  | declare -x HOME="/home/steam"
satisfactory-1  | declare -x HOSTNAME="satisfactory-server"
satisfactory-1  | declare -x LANG="en_US.UTF-8"
satisfactory-1  | declare -x LANGUAGE="en_US:en"
satisfactory-1  | declare -x LOG="false"
satisfactory-1  | declare -x MAXOBJECTS="2162688"
satisfactory-1  | declare -x MAXPLAYERS="4"
satisfactory-1  | declare -x MAXTICKRATE="30"
satisfactory-1  | declare -x OLDPWD
satisfactory-1  | declare -x PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
satisfactory-1  | declare -x PGID="1000"
satisfactory-1  | declare -x PUID="1000"
satisfactory-1  | declare -x PWD="/config"
satisfactory-1  | declare -x ROOTLESS="true"
satisfactory-1  | declare -x SERVERGAMEPORT="7777"
satisfactory-1  | declare -x SERVERSTREAMING="true"
satisfactory-1  | declare -x SHLVL="1"
satisfactory-1  | declare -x SKIPUPDATE="false"
satisfactory-1  | declare -x STEAMAPPID="1690800"
satisfactory-1  | declare -x STEAMBETA="false"
satisfactory-1  | declare -x TIMEOUT="60"
satisfactory-1  | declare -x TZ="Europe/Berlin"
satisfactory-1  | declare -x USER="steam"
satisfactory-1  | declare -x VERSION="1.9.1"
satisfactory-1  | declare -x VMOVERRIDE="false"
satisfactory-1  |
satisfactory-1  | System info:
satisfactory-1  | OS:  Linux satisfactory-server 6.6.71 #1-NixOS SMP PREEMPT_DYNAMIC Fri Jan 10 13:31:36 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
satisfactory-1  | CPU: 12th Gen Intel(R) Core(TM) i5-12500
satisfactory-1  | RAM: 38GB/56GB
satisfactory-1  | HDD: 152GB/295GB (54% used)
satisfactory-1  |
satisfactory-1  | Current version:
satisfactory-1  | 1.9.1
satisfactory-1  | Current user:
satisfactory-1  | uid=1000(steam) gid=1000(steam) groups=1000(steam)
satisfactory-1  | Proposed user:
satisfactory-1  | uid=1000(?) gid=1000(?) groups=1000(?)
satisfactory-1  |
satisfactory-1  | Exiting...
satisfactory-1 exited with code 1

System Specs

===== START ISSUE REPORT =====
OS:  Linux hostname 6.6.71 #1-NixOS SMP PREEMPT_DYNAMIC Fri Jan 10 13:31:36 UTC 2025 x86_64 GNU/Linux
CPU: 12th Gen Intel(R) Core(TM) i5-12500
RAM: 38GB/56GB
HDD: 152GB/295GB (54% used)
===== END ISSUE REPORT =====

Logs

satisfactory-1  | ===== Satisfactory Server 1.9.1 =====
satisfactory-1  | https://github.com/wolveix/satisfactory-server
satisfactory-1  |
satisfactory-1  | Checking available memory: 38GB detected
satisfactory-1  | Clearing old Satisfactory logs (set LOG=true to disable this)
satisfactory-1  | Setting autosave number to 5
satisfactory-1  | Setting max objects to 2162688
satisfactory-1  | Setting max tick rate to 30
satisfactory-1  | Setting server streaming to 1
satisfactory-1  | Setting timeout to 60
satisfactory-1  | Setting max players to 4
satisfactory-1  | Checking available storage: 129GB detected
satisfactory-1  |
satisfactory-1  | Downloading the latest version of the game...
satisfactory-1  | Redirecting stderr to '/home/steam/.steam/steam/logs/stderr.txt'
satisfactory-1  | ILocalize::AddFile() failed to load file "public/steambootstrapper_english.txt".
satisfactory-1  | [  0%] Checking for available update...
satisfactory-1  | KeyValues Error: LoadFromBuffer: missing {   (current key: '<!DOCTYPE') in file manifest [offset: 15]
satisfactory-1  |
satisfactory-1  | ../tier1/KeyValues.cpp (2925) : Assertion Failed: Error while parsing text KeyValues for resource manifest
satisfactory-1  | [----] Verifying installation...
satisfactory-1  | [  0%] Downloading Update...
satisfactory-1  | [  0%] Checking for available update...
satisfactory-1  | KeyValues Error: LoadFromBuffer: missing {   (current key: '<!DOCTYPE') in file manifest [offset: 15]
satisfactory-1  |
satisfactory-1  | ../tier1/KeyValues.cpp (2925) : Assertion Failed: Error while parsing text KeyValues for resource manifest
satisfactory-1  | [----] !!! Fatal Error: Steamcmd needs to be online to update.         Please confirm your network connection and try again.
satisfactory-1  | threadtools.cpp (3294) : Assertion Failed: Illegal termination of worker thread 'Thread(0x0x5782d650/0x0xf7724a'
satisfactory-1 exited with code 0

Docker Version

Docker version 27.3.1, build v27.3.1
Docker Compose version 2.30.3

Additional Context (/ what I already tried)

  1. The error occured with Satisfactory Server image version v1.8.9 (was already running that version for a while, but did Linux system updates recently). I also tried v1.8.7 and now v1.9.1, which also don't work.
  2. I tried deleting the satisfactory-gamefiles volume (and re-setting the owner to id 1000) - didn't help.
  3. The ping media.steampowered.com and add to /etc/hosts hack didn't help.
  4. I previously was on QEMU CPU and tried to fix it by switching to Proxmox type host CPU, which didn't help (logs/debug from above are collected after switching).
  5. I already tried with network_mode: host - same result.
@voruti
Copy link
Contributor Author

voruti commented Jan 18, 2025

I did some more testing:

v1.7.4 works, as some other people described in #306. The solution for that issue was the host CPU in Proxmox, but that doesn't work for me (like I already described above).

I'm now able to play by downloading the game on image v1.7.4 and then playing on v1.9.1 with SKIPUPDATE: "true". But this is no long-term solution.

@wolveix
Copy link
Owner

wolveix commented Jan 20, 2025

@voruti Does v1.8.0 work?

@voruti
Copy link
Contributor Author

voruti commented Jan 20, 2025

Does v1.8.0 work?

No.

@wolveix
Copy link
Owner

wolveix commented Jan 20, 2025

Weird, so it's a unique issue with using later versions of Ubuntu on your system. I'm not too sure what to suggest unfortunately, as this is a SteamCMD issue rather than an issue with this container. Please try running the following:

docker run --name=steam --rm steamcmd/steamcmd:ubuntu-18 steamcmd +login anonymous +app_update 1690800 validate +quit

@voruti
Copy link
Contributor Author

voruti commented Jan 20, 2025

Weird, so it's a unique issue with using later versions of Ubuntu on your system. I'm not too sure what to suggest unfortunately, as this is a SteamCMD issue rather than an issue with this container. Please try running the following:

docker run --name=steam --rm steamcmd/steamcmd:ubuntu-18 steamcmd +login anonymous +app_update 1690800 validate +quit

That works without issues with all of the following images:

steamcmd/steamcmd:ubuntu-18
steamcmd/steamcmd:ubuntu-20
steamcmd/steamcmd:ubuntu-22
steamcmd/steamcmd:ubuntu-24

Interesting... 🤔

@wolveix
Copy link
Owner

wolveix commented Jan 20, 2025

Now that is surprising! Could you try this, please? It's a direct rip from what the container runs:

docker run --name=steam --rm  steamcmd/steamcmd:ubuntu-18 steamcmd +force_install_dir /config/gamefiles +login anonymous +app_update 1690800 -beta public validate +quit

I suspect it could be a difference in how filesystem permissions may be handled between different versions of Ubuntu. So if this exact command does work, please try it with /config/gamefiles mapped to something.

@voruti
Copy link
Contributor Author

voruti commented Jan 20, 2025

Now that is surprising! Could you try this, please? It's a direct rip from what the container runs:

docker run --name=steam --rm  steamcmd/steamcmd:ubuntu-18 steamcmd +force_install_dir /config/gamefiles +login anonymous +app_update 1690800 -beta public validate +quit

I suspect it could be a difference in how filesystem permissions may be handled between different versions of Ubuntu. So if this exact command does work, please try it with /config/gamefiles mapped to something.

Also successful.


I checked out the repo (on main branch) and did some testing with the init.sh script.
With the following diff, the download works (running the container with just docker run localbuiltimage):

diff --git a/init.sh b/init.sh
index 78d2987..284cc2e 100755
--- a/init.sh
+++ b/init.sh
@@ -5,7 +5,6 @@ set -e
 printf "===== Satisfactory Server %s =====\\nhttps://github.com/wolveix/satisfactory-server\\n\\n" "$VERSION"

 CURRENTUID=$(id -u)
-HOME="/home/steam"
 MSGERROR="\033[0;31mERROR:\033[0m"
 MSGWARNING="\033[0;33mWARNING:\033[0m"
 NUMCHECK='^[0-9]+$'
@@ -111,7 +110,7 @@ echo "Satisfactory logs can be found in /config/gamefiles/FactoryGame/Saved/Logs

 if [[ "${ROOTLESS,,}" != "true" ]]; then
   chown -R "$PUID":"$PGID" /config /home/steam /tmp/dumps
-  exec gosu "$USER" "/home/steam/run.sh" "$@"
+  exec "/home/steam/run.sh" "$@"
 else
   exec "/home/steam/run.sh" "$@"
 fi

@wolveix
Copy link
Owner

wolveix commented Jan 20, 2025

The exec change you highlight shouldn't make a difference, as you're using ROOTLESS anyway, no?

In which case, the issue seems to be with setting $HOME, right? I just tested the default value of $HOME from the SteamCMD Docker image, and it's /root. I don't think this is the problem, I think you're just running it as rootful where you normally run it as rootless, which is where the issue most likely resides.

@voruti
Copy link
Contributor Author

voruti commented Jan 20, 2025

I'm sorry for the confusion.

running the container with just docker run localbuiltimage

I ran the locally built image without any parameters at all (just like docker run wolveix/satisfactory-server:v1.9.1), so ROOTLESS is at it's default value of false in that case and I still got the same error. But then with the diff from above, the download works.

@wolveix
Copy link
Owner

wolveix commented Jan 20, 2025

Could you try (keep going until success):

  1. Run the image with no changes, but with ROOTLESS=true
  2. Run the image with just the HOME environment variable removed
  3. Run the image with just the if statement change?

@voruti
Copy link
Contributor Author

voruti commented Jan 20, 2025

I crafted a minimal "non-working" example. Setting another user or staying on root doesn't change the error.

FROM steamcmd/steamcmd:ubuntu-22

ENV HOME=/customhome

# when running as another user, correct permissions must be set for the directory:
#RUN mkdir -p           $HOME
#RUN chown -R 1000:1000 $HOME
#USER 1000

# the following fails:
CMD "steamcmd +login anonymous +app_update 1690800 validate +quit"

When keeping the default value of HOME=/root, everything works.


Could you try (keep going until success):
1. Run the image with no changes, but with ROOTLESS=true
2. Run the image with just the HOME environment variable removed
3. Run the image with just the if statement change?

Oh, just saw your message. I think the above also helps.

@voruti
Copy link
Contributor Author

voruti commented Jan 20, 2025

Could you try (keep going until success):
1. Run the image with no changes, but with ROOTLESS=true
2. Run the image with just the HOME environment variable removed
3. Run the image with just the if statement change?

1. problem persists
2. problem persists
1. + 2. (ROOTLESS=true and remove HOME) works!

@voruti
Copy link
Contributor Author

voruti commented Jan 21, 2025

I crafted a minimal "non-working" example. Setting another user or staying on root doesn't change the error.

FROM steamcmd/steamcmd:ubuntu-22

ENV HOME=/customhome

# when running as another user, correct permissions must be set for the directory:
#RUN mkdir -p           $HOME
#RUN chown -R 1000:1000 $HOME
#USER 1000

# the following fails:
CMD "steamcmd +login anonymous +app_update 1690800 validate +quit"

When keeping the default value of HOME=/root, everything works.

With FROM steamcmd/steamcmd:ubuntu-18 it also works, despite the changed HOME.

@wolveix
Copy link
Owner

wolveix commented Jan 21, 2025

So to summarise, the current image works with ROOTLESS=true, as long as HOME isn't set?

@voruti
Copy link
Contributor Author

voruti commented Jan 23, 2025

So to summarise, the current image works with ROOTLESS=true, as long as HOME isn't set?

Yes. docker run -e ROOTLESS=true local-image-with-unset-home

But after looking at the ROOTLESS variable again, I'm a little bit confused about the way it works:

diff --git a/init.sh b/init.sh
index 78d2987..b224b5d 100755
--- a/init.sh
+++ b/init.sh
@@ -54,20 +54,20 @@ if [[ "${LOG,,}" != "true" ]]; then
 fi

 # check if the user and group IDs have been set
-if [[ "$CURRENTUID" -ne "0" ]] && [[ "${ROOTLESS,,}" != "true" ]]; then
+if [[ "$CURRENTUID" -ne "0" ]] && [[ "${ROOTLESS,,}" != "true" ]]; then # CURRENTUID is 0, skipping this
     printf "${MSGERROR} Current user (%s) is not root (0)\\nPass your user and group to the container using the PGID and PUID environment variables\\nDo not use the --user flag (or user: field in Docker Compose) without setting ROOTLESS=true\\n" "$CURRENTUID"
     exit 1
 fi

-if ! [[ "$PGID" =~ $NUMCHECK ]] ; then
+if ! [[ "$PGID" =~ $NUMCHECK ]] ; then # PGID is at default value 1000, skipping
     printf "${MSGWARNING} Invalid group id given: %s\\n" "$PGID"
     PGID="1000"
-elif [[ "$PGID" -eq 0 ]]; then
+elif [[ "$PGID" -eq 0 ]]; then # 1000 != 0, skipping
     printf "${MSGERROR} PGID/group cannot be 0 (root)\\n"
     exit 1
 fi

-if ! [[ "$PUID" =~ $NUMCHECK ]] ; then
+if ! [[ "$PUID" =~ $NUMCHECK ]] ; then # same as PGID
     printf "${MSGWARNING} Invalid user id given: %s\\n" "$PUID"
     PUID="1000"
 elif [[ "$PUID" -eq 0 ]]; then
@@ -75,7 +75,7 @@ elif [[ "$PUID" -eq 0 ]]; then
     exit 1
 fi

-if [[ "${ROOTLESS,,}" != "true" ]]; then
+if [[ "${ROOTLESS,,}" != "true" ]]; then # ROOTLESS is true, skipping
   if [[ $(getent group $PGID | cut -d: -f1) ]]; then
       usermod -a -G "$PGID" steam
   else
@@ -109,9 +109,9 @@ mkdir -p \

 echo "Satisfactory logs can be found in /config/gamefiles/FactoryGame/Saved/Logs" > /config/logs/satisfactory-path.txt

-if [[ "${ROOTLESS,,}" != "true" ]]; then
+if [[ "${ROOTLESS,,}" != "true" ]]; then # ROOTLESS is true, skipping
   chown -R "$PUID":"$PGID" /config /home/steam /tmp/dumps
   exec gosu "$USER" "/home/steam/run.sh" "$@"
 else
-  exec "/home/steam/run.sh" "$@"
+  exec "/home/steam/run.sh" "$@" # starting Steamcmd here, as root, while ROOTLESS is true ?!?
 fi

@wolveix
Copy link
Owner

wolveix commented Jan 23, 2025

The whole point of ROOTLESS was to permit people supplying their own UID and GID to the container using Docker's --user 1000:1000 mechanism. Since you're supplying your own user which cannot be root, it's impossible to escalate to another using using gosu, nor it is possible to usermod. So the changes you highlight were necessary to facilitate non-root users.

I just pushed a new revision to ghcr.io/wolveix/satisfactory-server:389, please test it and let me know whether it resolves your issue :)

@voruti
Copy link
Contributor Author

voruti commented Jan 23, 2025

It doesn't fix it. I think I will take a look into Steamcmd and why it behaves this way. I'll open a PR afterwards.

But I appreciate the removal (see following excerpt) of the ROOTLESS variable, I think it simplifies the configuration:

diff --git a/init.sh b/init.sh
index 78d2987..29e3aff 100755
--- a/init.sh
+++ b/init.sh
@@ -54,9 +54,8 @@ if [[ "${LOG,,}" != "true" ]]; then
 fi

 # check if the user and group IDs have been set
-if [[ "$CURRENTUID" -ne "0" ]] && [[ "${ROOTLESS,,}" != "true" ]]; then
-    printf "${MSGERROR} Current user (%s) is not root (0)\\nPass your user and group to the container using the PGID and PUID environment variables\\nDo not use the --user flag (or user: field in Docker Compose) without setting ROOTLESS=true\\n" "$CURRENTUID"
-    exit 1
+if [[ "$CURRENTUID" -ne "0" ]]; then
+    printf "${MSGWARNING} Current user (%s) is not root (0).\\nNo permissions will be adjusted as we're running within a rootless environment.\\n" "$CURRENTUID"
 fi

 if ! [[ "$PGID" =~ $NUMCHECK ]] ; then
@@ -75,7 +74,7 @@ elif [[ "$PUID" -eq 0 ]]; then
     exit 1
 fi

-if [[ "${ROOTLESS,,}" != "true" ]]; then
+if [[ "$CURRENTUID" -eq "0" ]]; then
     if [[ $(getent group $PGID | cut -d: -f1) ]]; then
         usermod -a -G "$PGID" steam
     else
@@ -109,9 +108,10 @@ mkdir -p \

 echo "Satisfactory logs can be found in /config/gamefiles/FactoryGame/Saved/Logs" > /config/logs/satisfactory-path.txt

-if [[ "${ROOTLESS,,}" != "true" ]]; then
+if [[ "$CURRENTUID" -eq "0" ]]; then
     chown -R "$PUID":"$PGID" /config /home/steam /tmp/dumps
     exec gosu "$USER" "/home/steam/run.sh" "$@"
 else
+    # running within a rootless environment
     exec "/home/steam/run.sh" "$@"
 fi

@wolveix
Copy link
Owner

wolveix commented Jan 23, 2025

Yeah, as I was reminding myself of what the hell the variable did in the first place (prompted by your query), I realised it was totally unnecessary anyway.

I'm curious why that didn't work, when it's effectively the same as the version you said should have. Hm.

wolveix added a commit that referenced this issue Jan 23, 2025
…user` directly to the container (#389)

Increase healthcheck timeout (#392)
@wolveix
Copy link
Owner

wolveix commented Jan 24, 2025

Copying my message from #393, I don't really see a way around the rootless permissions issues, outside of requiring people who wish to run it rootless to build it with new build arguments 🤷‍♂ If you're running with any UID/GID that's not 1000:1000, you're going to encounter permissions issues.

wolveix added a commit that referenced this issue Jan 24, 2025
@wolveix
Copy link
Owner

wolveix commented Jan 24, 2025

I've adjusted the rootless design to follow my previous message. Please take a look at the new README, and let me know how you get on :)

@wolveix wolveix changed the title Steamcmd download error Rootless SteamCMD download error Jan 24, 2025
voruti added a commit to voruti/satisfactory-server that referenced this issue Jan 24, 2025
Copy some Steam files to the newly created user's home directory.
@voruti voruti changed the title Rootless SteamCMD download error SteamCMD download error Jan 24, 2025
@voruti
Copy link
Contributor Author

voruti commented Jan 24, 2025

I created a PR.

I don't really understand why the added copy statement is necessary for my machine/server, but with this it works.

wolveix pushed a commit that referenced this issue Jan 24, 2025
Copy some Steam files to the newly created user's home directory.
@wolveix
Copy link
Owner

wolveix commented Jan 24, 2025

How peculiar! Thanks :)

@wolveix wolveix added bug Something isn't working environment issue Issue caused by environmental factors, not by the image itself labels Jan 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working environment issue Issue caused by environmental factors, not by the image itself
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants