Pulling features from ghcr.io breaks when GHE token is set

[gpiper14]

There is an issue where pulling a feature from ghcr.io will fail with a 403 if you have a GITHUB_TOKEN env var set for a GitHub Enterprise instance. The issue appears to stem from here as it assumes any GITHUB_TOKEN env var is a github.com token.

Possible solution

A possible solution could be to check for the existence of a GITHUB_HOST env var, and if it is set, only use the GITHUB_TOKEN for ghcr.io requests if it is set to github.com

[joshspicer]

Agreed, GITHUB_TOKEN has the highest precedence - this is especially useful for codespaces where that token can be granted additional scopes through the dev container config.

GITHUB_HOST seems like a nice approach, thanks for suggesting that. Even better, we might be able to stop looking at the GITHUB_TOKEN all together and rely on the standard docker credential files.

[joshspicer]

@gpiper14, in your opinion (from the GitHub Enterprise scenario) is it ok behavior to assume that if no GITHUB_HOST is set that the GITHUB_TOKEN is for the public github.com instance?

I'm thinking to bump GITHUB_TOKEN to have the least precedence (looking at docker config/cred helpers first), and for back compat continuing to fall back to use GITHUB_TOKEN unless GITHUB_HOST is set and not github.com (see: https://github.com/devcontainers/cli/pull/482/files#diff-00403d33ea4c9f9520b612ddfc2028d85401eb237f1d42edf13a6b258c421fadR176)

Is GITHUB_HOST documented somewhere that you could share with me, or is this env var convention of some sort?

[gpiper14]

That seems like a decent assumption to me, but would likely want to make the behavior clear in some sort of documentation.

Im not sure if there is any "official" guidance from github for GITHUB_HOST. The closest thing I have is the hub cli produced by github, which is why i have that variable set in the first place. See docs here

Thanks for the quick reply on this!

[joshspicer]

Thanks for putting this on our radar!

[brcarp]

I'm glad I stumbled on this issue, since I just got burned by this and have been digging around trying to find a workaround. The issue is that we had been using docker/login-action@v2 in GitHub Actions to log in to the GitHub Container Registry with username and password set, and separately cloning repositories via SSH, but for various reasons we needed to switch to cloning using HTTPS, which necessitated passing GITHUB_TOKEN in as an environment variable. Except as soon as I do that, access to ghcr.io breaks with errors like this:

  [timestamp] * Processing feature: ghcr.io/{org}/features/{service}
  
  [timestamp] (!) WARNING: Falling back to deprecated GitHub Release syntax. See https://github.com/devcontainers/spec/blob/main/proposals/devcontainer-features.md#referencing-a-feature for updated specification.
  [timestamp] Github feature.
  
  [timestamp] Could not resolve Feature 'ghcr.io/{org}/features/{service}'.  Ensure the Feature is published and accessible from your current environment.
  
  Error: Failed to process feature ghcr.io/{org}/features/{service}
      at jie (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:1844:3124)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async Jm (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:1844:2243)
      at async mh (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:1847:1644)
      at async gD (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:1864:4822)
      at async Eae (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:2059:27846)
      at async gae (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:2059:25119)
  
  Error: Dev container build failed: Failed to process feature ghcr.io/{org}/features/{service} (exit code: undefined)
  An error occurred building the container.
  Error: Failed to process feature ghcr.io/{org}/features/{service}

Anyone know of any immediate workarounds that we can take advantage of?

[joshspicer]

This change will be updating the precedence levels so the docker login creds is higher than the environment's GITHUB_TOKEN. That isn't shipped yet (can start to cut a release today), but something to watch :)

Is your Feature scoped to be publicly accessible in GHCR? (We have a bit of detail on doing that here if appropriate for your Feature) If not, it's likely this action run's GITHUB_TOKEN doesn't have the necessary scopes to read this private or internal Feature.

Is your Feature published to the same organization that you're trying to use it in? Where are you getting the GITHUB_TOKEN from? (Is it the one the actions provides or a PAT?)

[joshspicer]

The order of precedence (moving GITHUB_TOKEN after using docker cred stores) has shipped in CLI version 0.39.0.