Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error: failed to solve: failed to compute cache key: failed to copy: invalid status response 416 #558

Closed
nmattia opened this issue Feb 8, 2022 · 29 comments
Labels
area/cache kind/upstream Changes need to be made on upstream project

Comments

@nmattia
Copy link

nmattia commented Feb 8, 2022

Behaviour

Got an error while computing the cache key:

error: failed to solve: failed to compute cache key: failed to copy: invalid status response 416 The range specified is invalid for the current size of the resource. for https://vth0acprodeus2file0.blob.core.windows.net/1665cc709f9c430f88b363d79c2f9f6f/35da92eddf84ec1194f60050f276280d?sv=2019-07-07&sr=b&sig=tAOoxQlbhCkNGNswTquFfnmtVgq3VCAP4Icc0bdnvlg%3D&se=2022-02-08T15%3A07%3A22Z&sp=r&rscl=x-e2eid-d10f7757-becb4d8c-94279385-0bd5086d
[4181](https://github.com/dfinity/internet-identity/runs/5110353933?check_suite_focus=true#step:4:4181)
Error: buildx failed with: error: failed to solve: failed to compute cache key: failed to copy: invalid status response 416 The range specified is invalid for the current size of the resource. for https://vth0acprodeus2file0.blob.core.windows.net/1665cc709f9c430f88b363d79c2f9f6f/35da92eddf84ec1194f60050f276280d?sv=2019-07-07&sr=b&sig=tAOoxQlbhCkNGNswTquFfnmtVgq3VCAP4Icc0bdnvlg%3D&se=2022-02-08T15%3A07%3A22Z&sp=r&rscl=x-e2eid-d10f7757-becb4d8c-94279385-0bd5086d

This was transient and I'm not sure how to reproduce it. Here is the GitHub actions run: https://github.com/dfinity/internet-identity/runs/5110353933?check_suite_focus=true

I reckon that might be a buildx issue more than a build-push-action issue; happy to migrate the ticket if you confirm.

@nmattia
Copy link
Author

nmattia commented Feb 11, 2022

Getting this somewhat regularly now, any idea what might be happening? https://github.com/dfinity/internet-identity/runs/5156407802?check_suite_focus=true#step:4:2065

@crazy-max crazy-max added area/cache kind/upstream Changes need to be made on upstream project labels Feb 11, 2022
@crazy-max
Copy link
Member

@nmattia Looks linked to the GitHub Cache API. @dhadka @chrispat We saw that one too few days ago with @tonistiigi. Any ideas?

@nmattia
Copy link
Author

nmattia commented Mar 15, 2022

It's gotten less frequent, but still running into it every now and then: https://github.com/dfinity/internet-identity/runs/5553785037?check_suite_focus=true

@Kailai-Wang
Copy link

@patrikholcak
Copy link

I’m also facing this issue across multiple projects 😞 this is the setup we’re using

      - uses: docker/build-push-action@v2
        with:
          context: .
          push: true
          tags: ghcr.io/org/repo:tag
          cache-from: type=gha
          cache-to: type=gha,mode=max

@tonistiigi
Copy link
Member

We updated the error message for this case in v0.10.1 with extra info. If you have received this error after that then please post the message here.

@nmattia
Copy link
Author

nmattia commented Apr 27, 2022

@tonistiigi to clarify, this is v0.10.1 of what?

@crazy-max
Copy link
Member

@nmattia BuildKit 0.10.1 which is the latest stable so it should be the one used in your workflow if you have not enforced a specific BuildKit image in setup-buildx-action.

@tonistiigi
Copy link
Member

@dhadka

Got this on buildkit CI.

error: failed to solve: failed to compute cache key: failed to copy: invalid status response 416 The range specified is invalid for the current size of the resource. for https://ki6cacprodeus1file5.blob.core.windows.net/2ff2ce0399e5464697755b110c91788d/dd15048612dcec11b65614cb655b2cad?sv=2019-07-07&sr=b&sig=n7RwHrvaJu%2FrMLyCzY3e%2FXnNBEImbs0XulVQstSARV8%3D&se=2022-05-26T01%3A48%3A36Z&sp=r&rscl=x-e2eid-e0b78c9e-7a194019-bf7b385e-d8b4c39f, range: bytes=115194599-

But the URL is already expired so I can't inspect what is wrong with that blob. Maybe you have better visibility to give some ideas. I wonder if bug could be that this is asking for the range exactly at the end of the file.

@mmdeniz
Copy link

mmdeniz commented May 28, 2022

Now I am unable to use either ADD or COPY in my Dockerfile so the build process fails.
I am receiving the errors listed below:

ERROR: failed to calculate checksum of ref 1gjng5kin5uk6yv60uy8tx1hs::w6ujc2j2e2mnienr66n7godbb: failed to walk /tmp/buildkit-mount296995550/tmp: lstat /tmp/buildkit-mount296995550/tmp: no such file or directory

error: failed to solve: failed to compute cache key: failed to calculate checksum of ref 1gjng5kin5uk6yv60uy8tx1hs::w6ujc2j2e2mnienr66n7godbb: failed to walk /tmp/buildkit-mount296995550/tmp: lstat /tmp/buildkit-mount296995550/tmp: no such file or directory
Error: buildx failed with: error: failed to solve: failed to compute cache key: failed to calculate checksum of ref 1gjng5kin5uk6yv60uy8tx1hs::w6ujc2j2e2mnienr66n7godbb: failed to walk /tmp/buildkit-mount296995550/tmp: lstat /tmp/buildkit-mount296995550/tmp: no such file or directory

I used Dockerfile file in the home folder or under another folder, nothing changed.

@willzhang
Copy link

willzhang commented Jun 14, 2022

same

root@VM-20-2-ubuntu:~/test# cat Dockerfile
FROM quay.io/skopeo/stable:latest as builder1
RUN for image in $(cat images.list); do skopeo sync --format=v2s2 --src docker --dest docker ${image} localhost:5000 --dest-tls-verify=false; do
ne

FROM registry:2
COPY --from=builder1 /tmp /tmp
COPY registry /var/lib/registry/
root@VM-20-2-ubuntu:~/test# ll
total 12
drwxr-xr-x  2 root root 4096 Jun 14 16:52 ./
drwx------ 12 root root 4096 Jun 14 16:52 ../
-rw-r--r--  1 root root  603 Jun 14 16:52 Dockerfile

build failed, must mkdir test/registry before build , but the test/registry will generate auto after build !!

root@VM-20-2-ubuntu:~/test# docker buildx build -t test .
[+] Building 1.4s (14/15)                                                                                                                       
 => [internal] load build definition from Dockerfile                                                                                       0.0s
 => => transferring dockerfile: 640B                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                          0.0s
 => => transferring context: 2B                                                                                                            0.0s
 => [internal] load metadata for docker.io/library/registry:2                                                                              1.0s
 => [internal] load metadata for quay.io/skopeo/stable:latest                                                                              0.3s
 => [internal] load metadata for quay.io/kubespray/kubespray:v2.19.0                                                                       0.0s
 => [builder0 1/3] FROM quay.io/kubespray/kubespray:v2.19.0                                                                                0.0s
 => [internal] load build context                                                                                                          0.0s
 => => transferring context: 2B                                                                                                            0.0s
 => CACHED [stage-2 1/3] FROM docker.io/library/registry:2@sha256:bedef0f1d248508fe0a16d2cacea1d2e68e899b2220e2258f1b604e1f327d475         0.4s
 => [builder1 1/3] FROM quay.io/skopeo/stable:latest@sha256:4dd7606ce9098541b10b99920da3bfd9740e004b9862f9822ed7d8d0bf6ea870               0.0s
 => CACHED [builder0 2/3] RUN cd /kubespray     && grep -rIl . | xargs sed -i "s/\r//g"  || true                                           0.0s
 => CACHED [builder0 3/3] RUN cd /kubespray/contrib/offline/     && bash generate_list.sh                                                  0.0s
 => CACHED [builder1 2/3] COPY --from=builder0 /kubespray/contrib/offline/temp/images.list .                                               0.0s
 => CANCELED [builder1 3/3] RUN for image in $(cat images.list); do skopeo sync --format=v2s2 --src docker --dest docker ${image} localho  0.4s
 => CACHED [stage-2 2/3] COPY --from=builder1 /tmp /tmp                                                                                    0.0s
 => ERROR [stage-2 3/3] COPY registry /var/lib/registry/                                                                                   0.0s
------
 > [stage-2 3/3] COPY registry /var/lib/registry/:
------
error: failed to solve: failed to compute cache key: "/registry" not found: not found
root@VM-20-2-ubuntu:~/test# 

@cyrus-za
Copy link

Also running into this, both locally and in github actions

@baocongchen
Copy link

I got the error as well. Any idea how to solve the error?

@baocongchen
Copy link

Never mind. I figured it out. My COPY command should be
COPY ./project/techtrends/ /app
instead of
COPY ./techtrends/ /app

because techtrends is under project directory

@archonic
Copy link

archonic commented Aug 5, 2022

I've never had issues with this COPY line before but it's happening consistently when building with load: true in GH actions.

Dockerfile:6
--------------------
   4 |     
   5 |     COPY --from=pandoc /usr/local/bin/pandoc* /usr/bin/
   6 | >>> COPY --from=pandoc /opt/texlive* /usr/bin/
   7 |     
   8 |     ARG BUILD_PACKAGES="build-essential yarn git-core ..."
--------------------
error: failed to solve: failed to compute cache key: failed to calculate checksum of ref xvy7rzjibaifonylswzxq9vel::ma4wp1egg53q7n0wq3kbi98qf: "/opt/texlive/texdir/texmf-dist/doc/man": not found
Error: buildx failed with: error: failed to solve: failed to compute cache key: failed to calculate checksum of ref xvy7rzjibaifonylswzxq9vel::ma4wp1egg53q7n0wq3kbi98qf: "/opt/texlive/texdir/texmf-dist/doc/man": not found

Update:

Seems this was just because COPY --from=pandoc /opt/texlive* /usr/bin/ does not maintain folder structure. Changing it to COPY --from=pandoc /opt/texlive/ /usr/bin/texlive/ fixed it. Details here: https://stackoverflow.com/questions/30215830/dockerfile-copy-keep-subdirectory-structure

@Maleware
Copy link

Maleware commented Sep 12, 2022

I've got the current problem that the COPY --from=builder ./app/stackable-kafka-operator/ ./stackable-kafka-operator where FROM docker.stackable.tech/stackable-experimental/ubi8-rust-builder AS builder

This error happened the first time, when switching from build to buildx to enable multi-arch builds (which is quite painful anyway with qemu).

I've tried several things to avoid that error:

  1. COPY --from=builder ./app/stackable-kafka-operator/ ./
  2. COPY --from=builder ./app/stackable-kafka-operator/ /
  3. COPY --from=builder /app/stackable-kafka-operator/ /
  4. COPY --from=builder ./app/stackable-kafka-operator /
  5. COPY --from=builder /app/stackable-kafka-operator /

Sadly none of those combination was working.

To compile the operator during building time of the image we use a ubi8-minimal based rust builder. Multi-Arch is tried to be archived leveraging cargo build --target=ARCH (played some tricks here to archive building under qemu, testing of the new things fails because the arm64 side of things (native) fails during build time at this line since it is way faster.

Do I overlook something or is this an actual problem?

It fails with:
error: failed to solve: failed to compute cache key: failed to calculate checksum of ref rykfgneaumnifqjl5s76cl050::fvg8fineqh20z2i8y7dx5t3a2: "/app/stackable-kafka-operator": not found

Nevermind: COPY ./app/stackable-kafka-operator* ./stackable-kafka-operator worked for me

aviggiano added a commit to aviggiano/hack that referenced this issue Sep 13, 2022
@devinrsmith
Copy link

I'm getting this same issue consistently, even with the latest buildkit:

moby/buildkit:buildx-stable-1 => buildkitd github.com/moby/buildkit v0.10.4 a2ba6869363812a210fcc3ded6926757ab780b5f

@tonistiigi
Copy link
Member

@Maleware There is no such path /app/stackable-kafka-operator anywhere in the docker.stackable.tech/stackable-experimental/ubi8-rust-builder image afaics.

@Lecon-a
Copy link

Lecon-a commented Oct 15, 2022

I am also facing this docker build command's issue. When I run: docker build -t myimage .

The following is what I got:

> [2/5] COPY . /app:
------
failed to compute cache key: failed to copy: httpReadSeeker: failed open: failed to do request: Get "https://d2glxqk2uabbnd.cloudfront.net/b16f79-305812972329-ecbbcc52-aa8e-da32-9e5b-1acde3109751/b8500d72-cb2f-4ffe-8e1d-314cfa1121af?Expires=1665868609&Signature=SfNnQoCyOEjU1~p9hpfd~6lMml1le-9SQWMv0xv9X53eCHjC9C~qFdlKY9ir7PQG7H2zYFQsv7uju61-9YOEFaT2BDTSVoBPxmrypZI-VRmZzVkGe5~nZG~~eFRIt3dcIXHyVChpVd1WcsEQqe4Mem08vS7hjVZoKqKJEqT7EVwqgGDn~rdoTuKRYpdK8lKacgS3HALiEIH3Tv~PU4c2HB61eoVqIBCVOnDmQCzCdwje-hhtUja5r8O2L2tr6ns1proFFFe5uqqQUai9Z3QZIRlL-~JRDKjZtg~eZCyJL7WBoD1~u7meZMgEOjR4XrQ0pZyvgRo7cY1lxpV2dZTnlg__&Key-Pair-Id=KSPLJWGOARK62": net/http: TLS handshake timeout

@Lecon-a
Copy link

Lecon-a commented Oct 15, 2022

Here is my Dockerfile

FROM public.ecr.aws/sam/build-python3.7:latest

COPY . /app
WORKDIR /app

RUN pip install --upgrade pip
RUN pip install -r requirements.txt

ENTRYPOINT ["gunicorn", "-b", ":8080", "main:APP"]

nicholac pushed a commit to nismod/infra-risk-vis that referenced this issue Oct 20, 2022
nicholac added a commit to nismod/infra-risk-vis that referenced this issue Oct 20, 2022
nicholac pushed a commit to nismod/infra-risk-vis that referenced this issue Oct 20, 2022
nicholac added a commit to nismod/infra-risk-vis that referenced this issue Oct 20, 2022
nicholac pushed a commit to nismod/infra-risk-vis that referenced this issue Oct 20, 2022
nicholac added a commit to nismod/infra-risk-vis that referenced this issue Oct 20, 2022
nicholac pushed a commit to nismod/infra-risk-vis that referenced this issue Oct 20, 2022
nicholac pushed a commit to nismod/infra-risk-vis that referenced this issue Oct 20, 2022
nicholac added a commit to nismod/infra-risk-vis that referenced this issue Oct 20, 2022
@xorinzor
Copy link

xorinzor commented Oct 23, 2022

Same issue here, building works fine locally.

Failed build: https://github.com/Shoutz0r/frontend/actions/runs/3308523917/jobs/5460967584
Dockerfile: https://github.com/Shoutz0r/frontend/blob/4d2d50968d0a222f512f4c16660547c7f6df0b78/Dockerfile

EDIT:
Turns out that the docker/build-push-action@v3 uses the git context by default for building the docker images, rather then the local directory. Had to set context: . to fix this. Documentation

@Lecon-a
Copy link

Lecon-a commented Oct 24, 2022 via email

@crazy-max
Copy link
Member

Closing this issue as many comments are not related to the initial report.

@nmattia If you still encounter this issue, can you provide another repro with also BuildKit logs? Thanks.

@nmattia
Copy link
Author

nmattia commented Oct 24, 2022

@crazy-max will do thanks!

@frank-pian
Copy link

I set the context,but I still have the error.

https://github.com/frank-pian/OCCT/blob/occ-api/.github/workflows/build.yml

Dockerfile:51
--------------------
  49 |     RUN git clone --depth 1 https://github.com/gkv311/tcl.git
  50 |     WORKDIR tcl
  51 | >>> ADD occ_tcl_build_wasm.sh .
  52 |     RUN /bin/bash occ_tcl_build_wasm.sh
  53 |     
--------------------
ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref lvqspldaya570v6mf2cna5bxq::up36el7zdf0pt6k9d9cmrl9tx: "/occ_tcl_build_wasm.sh": not found
Error: buildx failed with: ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref lvqspldaya570v6mf2cna5bxq::up36el7zdf0pt6k9d9cmrl9tx: "/occ_tcl_build_wasm.sh": not found

tomalrussell pushed a commit to nismod/irv-frontend that referenced this issue Mar 6, 2023
tomalrussell pushed a commit to nismod/irv-frontend that referenced this issue Mar 6, 2023
tomalrussell pushed a commit to nismod/irv-frontend that referenced this issue Mar 6, 2023
tomalrussell pushed a commit to nismod/irv-frontend that referenced this issue Mar 6, 2023
@mattparlane
Copy link

I just got this error tonight. If anyone stumbles across this in the future, adding DOCKER_BUILDKIT=0 before the build command worked for me.

@toriandsouza
Copy link

We are getting this error --- happens every once in a while in our GH Actions for whatever reason:

#22 [16/17] COPY . /app
#22 ERROR: blob not found
------
 > [16/17] COPY . /app:
------
Dockerfile:122
--------------------
 12*** |         fi
 121 |     
 122 | >>> COPY . /app
 123 |     RUN mkdir -p /app/temp_files
 124 |     EXPOSE 8*** 8***82
--------------------
error: failed to solve: failed to compute cache key: blob not found
Error: buildx failed with: error: failed to solve: failed to compute cache key: blob not found

We have not been able to solve it and our builds are stalled.

@theAkito
Copy link

Same here.

https://github.com/theAkito/docker-murmur/actions/runs/5627573725/job/15250407075#step:7:6824

What's the actual reason behind this issue?

@crazy-max
Copy link
Member

@theAkito

#24 [linux/amd64 stage-1 5/6] COPY --from=build /root/mumble/release/murmurd /usr/bin/murmurd
#24 ERROR: failed to calculate checksum of ref qgp9ucmtmblbgry83pja48n5y::8myxzhj2gtzqfcqjue0dzjw4i: failed to walk /tmp/buildkit-mount2545155800/root/mumble/release: lstat /tmp/buildkit-mount2545155800/root/mumble/release: no such file or directory

This does not look related, can you open a new issue please?

@docker docker locked as too heated and limited conversation to collaborators Jul 22, 2023
mcasperson added a commit to OctopusSolutionsEngineering/Octopub that referenced this issue Oct 15, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/cache kind/upstream Changes need to be made on upstream project
Projects
None yet
Development

No branches or pull requests