No such image: oxsecurity/megalinter-<flavor>:latest

[Kurt-von-Laven]

Describe the bug
[email protected] abruptly started failing when the MegaLinter Docker image isn't explicitly tagged latest locally. The issue began at some point within the last 20 hours without pertinent changes on our end. Both mega-linter-runner and MegaLinter are pinned to v6.22.2.

To Reproduce
Steps to reproduce the behavior:

1. Configure the megalinter-full pre-commit hook in your .pre-commit-config.yaml:

   repos:
     - repo: https://github.com/oxsecurity/megalinter
       rev: v6.22.2
       hooks:
         - id: megalinter-full
           args:
             - [email protected] # Pin mega-linter-runner for maximum reproducibility.
             - --containername
             - megalinter-full
             - --remove-container
             - --fix
             - --env
             - "'APPLY_FIXES=all'"
             - --env
             - "'CLEAR_REPORT_FOLDER=true'"
             - --env
             - "'LOG_LEVEL=warning'"
             - --flavor
             - python # Any other flavor should work just as well.
             - --release
             - v6.22.2

2. Run docker rmi oxsecurity/megalinter-python:latest to verify that you don't have oxsecurity/megalinter-python:latest installed locally.

3. Run the hook with: pre-commit run megalinter-full --all-files --hook-stage push.

4. The hook fails with the following output:

   Run MegaLinter................................................................Failed
   - hook id: megalinter-full
   - exit code: 2
   
   Pulling docker image oxsecurity/megalinter-python:v6.22.2 ... 
   INFO: this operation can be long during the first use of mega-linter-runner
   The next runs, it will be immediate (thanks to docker cache !)
   v6.22.2: Pulling from oxsecurity/megalinter-python
   Digest: sha256:5adb1e95fca62b4d509d004bd43d8b2bdfa0ad3f211d447f679b582b5049059b
   Status: Image is up to date for oxsecurity/megalinter-python:v6.22.2
   No such image: oxsecurity/megalinter-python:latest
   

Expected behavior
I expected the hook to pass like it did yesterday.

Additional context
This could be related to #2646. I encountered the issue on Linux using rootless Docker both locally and in GitHub Actions. I was not able to reproduce the issue in WSL (Windows Subsystem for Linux), which only supports rootful Docker. Running docker pull oxsecurity/megalinter-python:latest works around the issue even though I already had pulled oxsecurity/megalinter-python:v6.22.2, which is currently the same image. The issue reproduces on all of the flavors I tried: documentation, dotnet, javascript, and python. I'm confused as to what changed since I didn't have the latest tag present locally before.

[btilford]

Same thing started happening today with mega-linter-runner@^6.19.0

[nvuillam]

hmmm what about a big docker system prune --all ? :/

[Kurt-von-Laven]

If that were going to fix it, then I don't see how it could possibly be broken in GitHub Actions as well? There's also the fact that it broke abruptly without any Docker operations on our end.

[btilford]

Dropping the caret in mega-linter-runner@^6.19.0 did fix it. But we've pinned to this version for the m1 Mac users.

[Kurt-von-Laven]

Neither docker system prune --all nor npm cache clean --force had an effect on the bug. The bug reproduces for mega-linter-runner v6.20.1, v6.21.0, v6.22.0, v6.22.1, and v6.22.2, but not v6.20.0. It also reproduces outside of pre-commit (e.g., npx -- [email protected] --container-name megalinter-full --remove-container --fix --env "'APPLY_FIXES=all'" --env "'CLEAR_REPORT_FOLDER=true'" --env "'LOG_LEVEL=warning'" --flavor python --release v6.22.2).

[echoix]

Does pre commit call npx mega linter runner too? Should we check on the runner side? The Docker side? Our images? Is there a new version of something elsewhere that affects?

[nvuillam]

All those tags exist on docker hub... I don't see what we could do on MegaLinter side :/

mega-linter-runner just does a docker pull then a docker run :/

Maybe something new on docker itself ?

[echoix]

Is this going on on GitHub actions too?

[Kurt-von-Laven]

Yeah, pre-commit simply calls npx mega-linter-runner in the manner specified above. I think we should probably see what is different in mega-linter-runner between v6.20.0 and v6.20.1. I am guessing there is something weird going on when --platform is passed to docker pull that was exposed by a Docker update. Yeah, it reproduces in GitHub Actions.

[echoix]

I vaguely remember that the version of node in the runners was about to change, is it in that timeframe?

[Kurt-von-Laven]

No, we pin Node.js both locally and in GitHub Actions, currently to v18.16.0, which hasn't changed since April.

[nvuillam]

I've not seen any issue of that type with the MegaLinter GitHub action

[Kurt-von-Laven]

No, I meant that running the pre-commit hook fails the same way in GitHub Actions; part of the point of pre-commit is to reuse the same configuration one uses locally in CI. I was able to reproduce the issue by running docker pull --platform linux/amd64 oxsecurity/megalinter-python:v6.22.2 directly, so I believe that indeed something changed in Docker or Docker Hub upstream that broke mega-linter-runner on certain platforms. Without --platform, docker pull still works with our images. I wonder if there is some step we are supposed to do to make our Docker images work correctly with respect to the new multi-platform feature. It makes sense that the official GitHub Action doesn't expose this bug, because it probably doesn't pass --platform to docker pull.

[nvuillam]

For the moment the --platform with default value linux/amd64 is just to force to use the linux/amd64 on platforms that are not, like M1

Maybe if we don't specify the platform it could work better ? We could add --platform none option that would remove the platform argument from docker pull... but not sure it would help a lot , I fear the issues are directly from docker itself... on specific platforms !

[Kurt-von-Laven]

The outage just resolved itself both locally and in GitHub Actions, so I'm guessing this was a bug in Docker Hub or something. Closing this for now. Others should feel free to comment if they experience the same issue, and I can look into it further.

[cam-barts]

Hi there, I'm definitely still having a similar issue. It appears to me that pulling the image is working fine, but when mega-linter-runner goes to run the image, it is attempting to run the latest tag.

I've attempted to run exactly the same way the default pre-commit configuration runs it, with:
npx mega-linter-runner --containername "megalinter-incremental" --remove-container --fix --env "'APPLY_FIXES=all'" --env "'CLEAR_REPORT_FOLDER=true'" --env "'LOG_LEVEL=warning'" --filesonly

Here is the output I get:

Pulling docker image oxsecurity/megalinter:v6 ... 
INFO: this operation can be long during the first use of mega-linter-runner
The next runs, it will be immediate (thanks to docker cache !)
v6: Pulling from oxsecurity/megalinter
Digest: sha256:7da83e77289ab3956e41f503b998858a5aebfcaf6725c25128508c622bc48395
Status: Downloaded newer image for oxsecurity/megalinter:v6
No such image: oxsecurity/megalinter:latest

I can verify that it pulled the correct v6 image to my machine at this point. I can work around by retagging the image with docker image tag oxsecurity/megalinter:v6 oxsecurity/megalinter:latest, but that feels like a hack.

Another work around I've discovered works for me, based on @btilford's comment above, is by tagging the megalinter-runner-version to 6.19.0, so:

npx [email protected] --containername "megalinter-incremental" --remove-container --fix --env "'APPLY_FIXES=all'" --env "'CLEAR_REPORT_FOLDER=true'" --env "'LOG_LEVEL=warning'" --filesonly works fine.

I can add that to my pre-commit config for now, but I can confirm that pre-commit will fail if it runs as the documentation currently suggests.

My currently working pre-commit hook in .pre-commit-config.yaml:

  - repo: https://github.com/oxsecurity/megalinter
    rev: v6.22.2 # Git tag specifying the hook, not mega-linter-runner, version
    hooks:
      - id: megalinter-incremental # Faster, less thorough
        stages:
          - commit
        args:
          - [email protected]
          - --containername
          - "megalinter-incremental"
          - --remove-container
          - --fix
          - --env
          - "'APPLY_FIXES=all'"
          - --env
          - "'CLEAR_REPORT_FOLDER=true'"
          - --env
          - "'LOG_LEVEL=warning'"
          - --filesonly
      - id: megalinter-full # Slower, more thorough
        stages:
          - push
        args:
          - [email protected]
          - --containername
          - "megalinter-full"
          - --remove-container
          - --fix
          - --env
          - "'APPLY_FIXES=all'"
          - --env
          - "'CLEAR_REPORT_FOLDER=true'"
          - --env
          - "'LOG_LEVEL=warning'"

[Kurt-von-Laven]

Can you reproduce the issue more simply with docker pull --platform linux/amd64 <MegaLinter image>? If so, does it reproduce with non-MegaLinter multiplatform Docker images?

[cam-barts]

@Kurt-von-Laven I'm having no issue pulling down the images with docker pull, both latest and v6.22.2, and I can pull down any other multiplatform image as needed. It seems to me the bug is the runner isn't attempting to run the version that the user requests. It pulls down the image the user requests fine, I can see that after removing all of the images and starting again. I've tested that with both the --release parameter and the full --image parameter, with each they pull what is expected, but with each they attempt to run latest after pulling.

[nvuillam]

@cam-barts what if you try with mega-linter-runner@beta ?

[nvuillam]

I think I have it, PR on the way

--platform argument was used for docker pull, but not for docker run

[Kurt-von-Laven]

Wild. That is actually a different bug with the same symptom, because I couldn't get past the docker pull when I experienced it. I get the impression Docker is making related breaking changes.