When the imageList docker image has dependencies, the dependent image pull failed

[init-object]

What happened:

docker logs -f sealer-registry :

What you expected to happen:

pull fmweb image success

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

• sealer version (use sealer version): v0.5.1
• Cloud provider or hardware configuration: 4 core 8g
• OS (e.g: cat /etc/os-release): CentOS Linux 7 (Core)
• Kernel (e.g. uname -a):Linux k8s-master01 3.10.0-1127.el7.x86_64 fix: ci lint for mod registry and image fix: ci lint for mod registry and image #1 SMP Tue Mar 31 23:36:51 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
• Install tools:
• Others:

[kakaZhou719]

@wangkun-king ,could you please check more details via "docs/site/src/docs/help/faq.md" . i think we lost some layers. you can try docker system prune -a clean your old images.

[init-object]

could you please check more details via "docs/site/src/docs/help/faq.md" . i think we lost some layers. you can try docker system prune -a clean your old images.

[root@k8s-node03 ~]# docker version
Client: Docker Engine - Community
Version: 19.03.14
API version: 1.40
Go version: go1.13.15
Git commit: 5eb3275
Built: Tue Dec 1 19:14:24 2020
OS/Arch: linux/amd64
Experimental: false

Server:
Engine:
Version: 19.03.14-sealer
API version: 1.40 (minimum version 1.12)
Go version: go1.13.15
Git commit: 711cc111cf
Built: Wed Jun 2 09:07:15 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.3.9
GitCommit: ea765aba0d05254012b0b9e595e995c09186427f
runc:
Version: 1.0.0-rc10
GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
docker-init:
Version: 0.18.0
GitCommit: fec3683

[root@k8s-node03 ~]# cat /etc/docker/daemon.json
{
"max-concurrent-downloads": 20,
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"insecure-registries": [
"sea.hub",
"sea.hub:5000"
],
"mirror-registries": [
{
"domain": "*",
"mirrors": ["http://sea.hub:5000"]
}
],
"data-root": "/var/lib/docker"
}

[root@sealer ~]# docker system prune -a
WARNING! This will remove:

• all stopped containers
• all networks not used by at least one container
• all images without at least one container associated to them
• all build cache

Are you sure you want to continue? [y/N] y
Deleted Images:
untagged: registry:2.7.1
untagged: registry:2.7.1
deleted: sha256:0d0107588605f17177f6d11bad900de6a7bcf2b7aa0f72f5e2dd04102cb34fda
deleted: sha256:a46656c7787a223c5f81952140e422d5eac3b23939e01bc98b9b104896a60cbb
deleted: sha256:4e426c176d3691b72c406fa95b3aefa77878ddbc0af004927eae36f4406d636e
deleted: sha256:b2d5eeeaba3a22b9b8aa97261957974a6bd65274ebd43e1d81d0a7b8b752b116
....
deleted: sha256:a84b4e945affc390a49fe16b1e47251671dfc628c111de3a4c5b2e0b09d2649a
deleted: sha256:4bc4490d37ab1d9664581bcbb9c198cd5c4e7eaf0c5b1607c5ec44a11eaf922f
deleted: sha256:21b5c6b4b2751d9e70a183749330b4b9e11687a0cb4995f33ee5812846128294
deleted: sha256:c2ddc1bc2645ab5d982c60434d8bbc6aecee1bd4e8eee0df7fd08c96df2d58bb

Total reclaimed space: 1.435GB

rebuild cluster image:

...

I think The layers of the dependent tomcat image are all cached

[kakaZhou719]

@wangkun-king ,nice work. whether the new image runs successfully？

[init-object]

@wangkun-king ,nice work. whether the new image runs successfully？

Unfortunately, still cannot pull successfully

Only pull tomcat image first and then pull fmweb image to succeed

[kakaZhou719]

@wangkun-king, do you mean only pull fmweb failed. if we pull tomcat first and then pull fmweb, will be successful?if so that a question. i guess the manifest of fmweb dont exist on the master-0, could please check it ?

[init-object]

@wangkun-king, do you mean only pull fmweb failed. if we pull tomcat first and then pull fmweb, will be successful?if so that a question. i guess the manifest of fmweb dont exist on the master-0, could please check it ?

how to do check it?

[init-object]

manifest of fmweb may be this

when execute docker pull xxxxx/fmweb:1.0.0 sealer-registry log as follows:

When I replaced fmweb with tomcat, curl succeeded

[kakaZhou719]

@wangkun-king , please check the difference of failed pull and success pull , diff the layers and the manifests to see which layers we lost in the blob.

examples like below :

docker image layer list include layers and manifests:

docker image blobs:

[init-object]

@kakaZhou719

[kakaZhou719]

@wangkun-king, looks like we actually lost the tomcat layers . this is a problem . you can temporarily circumvent the problem by adjusting the imageList order: fmweb first and set tomcat as secondary .

and i wrote the faq section yesterday ,that how to trouble shoot the lost layers using "docker inspect". please try it .

[init-object]

@kakaZhou719
But I have multiple images that depend on the tomcat image,
Adjusting the imageList order does not solve my problem

[fanux]

fixed in 0.6.1