Does porter archive or copy really need to consult index.docker.io for bundled images?

[jasmdk]

Is your feature request related to a problem? Please describe.
Due to the pull-rate-limitation imposed by docker.io, working with porter bundles often reach the limit on our CI setups.
Locally developers have a docker user and docker daemon on CI agents have been configured with a registry-mirror, but anyway when I archive or copy a bundle which includes bundle images, it still accesses the REST API on index.docker.io for images which originally came from docker hub. I would assume that once I published the bundle to my own registry, it contained all the information I need for an airgapped setup.
Publish works without any issues, but I cant archive the bundle to disc first in these scenarios.
The log indicates that during copy or archive, the referenced images are copied from the original source rather than the already bundled copy of the image.

Describe the solution you'd like
Eliminate the callbacks to the original docker registry for images which are already bundled. The information needed should already be present in the bundle.

Describe alternatives you've considered
At least document the limitations of using porter in airgapped scenarios.

Additional context
Output:
porter copy --source host.docker.internal:5443/library/airgapped:latest --destination registry.admin.synergy.local:21443/library/airgapped:copied2
Beginning bundle copy to registry.admin.synergy.local:21443/library/airgapped:copied2. This may take some time.
Starting to copy image host.docker.internal:5443/library/airgapped-installer:latest...
Completed image host.docker.internal:5443/library/airgapped-installer:latest copy
Starting to copy image busybox:latest...
Failed to copy image busybox:latest: failed to resolve or push image for service "busybox"
Error: unable to copy bundle to new location: error preparing the bundle with cnab-to-oci before pushing: failed to resolve or push image for service "busybox"

[carolynvs]

Hi @jasmdk, let me recap how to move a bundle across an airgap, and copy it between registryes, and then we can compare that to the commands that you used.

1. Publish the bundle to a registry, e.g. porter publish --registry docker.io/getporter. This bundle has references to other images on docker hub, e.g. docker.io/nginx.
2. Archive the bundle. This copies the bundle, and all referenced images, into a tgz file.
3. Move the tgz file across an airgap.
4. Publish the bundle archive to a registry on the other side of the airgap, e.g. porter publish --archive mybundle.tgz --reference localhost:5000/mybundle:v0.1.0. This bundle now has a reference to localhost:5000/nginx, not docker.io/nginx.
5. Copy the bundle to a second registry in the airgapped network. porter copy --source localhost:5000/mybundle:v0.1.0 --destination localhost:5001. This bundle now has a reference to localhost:5001/nginx.

[jasmdk]

Hi @carolynvs, I agree with the steps you outline.
In my scenario I do conceptually the same, and step5 executed with my network cable unplugged results in:
Error: error preparing artifacts: Get https://index.docker.io/v2/: dial tcp: lookup index.docker.io: no such host
In some scenario where I already copied the bundle to the registry under a different name does not always fail - but a freshly built bundle does.
Also the command porter archive mybundle.tgz --reference localhost:5000/mybundle:v0.1.0 fails with:

Starting to copy image busybox:latest...
Failed to copy image busybox:latest: failed to resolve or push image for service "busybox"
Error: unable to copy bundle to new location: error preparing the bundle with cnab-to-oci before pushing: failed to resolve or push image for service "busybox"

In neither case can I see the need for consulting docker.io to copy the image as it is already bundled, but some metadata probably still knows about the original location and uses that.

[carolynvs]

It sounds like you may have found a bug, but I am unclear on the reproduction steps. The steps I outlined should not hit the original registry once the bundle is relocated to another. Are you able to reproduce this with a test bundle, for example getporter/whalegap:v0.1.0?

If you can provide a set of steps for me to follow with a public bundle, then I can get a fix out for you.

[jasmdk]

To reproduce these airgapped scenarios, I assume you have a local (here insecure) registry running like:
docker run --name myregistry -p 5000:5000 --restart unless-stopped registry:2

To start with your "whalegap" bundle, do:
porter copy --source getporter/whalegap:v0.1.0 --destination localhost:5000/whalegap:v0.1.0

Now unplug your internet and copy the bundle already copied to the registry to another destination - here just another tag in the same registry:
porter copy --source localhost:5000/whalegap:v0.1.0 --destination localhost:5000/whalegap:v0.1.1
The copy now fails with:

Beginning bundle copy to localhost:5000/whalegap:v0.1.1. This may take some time.
Starting to copy image getporter/whalegap-installer:v0.1.0...
Failed to copy image getporter/whalegap-installer:v0.1.0: failed to resolve or push image for service "InvocationImage"
Error: unable to copy bundle to new location: error preparing the bundle with cnab-to-oci before pushing: failed to resolve or push image for service "InvocationImage"

If I plug in my internet again and repeat, the operation succeeds.

Another scenario is the "archive" action. Again unplug internet and do:
porter archive whalegap.tgz --reference localhost:5000/whalegap:v0.1.0 which produces:

Error: error preparing artifacts: Get https://index.docker.io/v2/: dial tcp: lookup index.docker.io: no such host

But succeeds if you have an internet connection.

Another thing worth noting is that for all of the above, I have configured my docker daemon with a registry mirror to minimize calls to docker.io, but none of these seem to care about my mirror settings - nor does "porter publish".

[carolynvs]

Thanks for the reproduction steps. I believe the bug is with copy and I have opened an issue against the cnab-to-oci library to track it. If you follow the steps that I outlined above in my original reply, you should be able to work around the bug until we have a fix, i.e. use archive+publish to move a bundle instead of copy.

[jasmdk]

Thanks - it does work as airgapped once the bundle has been exported.
Our current problem is more related to our CI which will pull bundles from a registry and install them during testing, but as our build agents do not have an valid docker hub user, then our complete set of build agents work with anonymous access against docker.io and our company exposes a single external IP so we often hit the docker pull rate limit.
That is why if "copy" or "archive" could work without contacting docker.io we would not hit this limit.
The only alternative is to host our bundles both in an OCI registry and on some shared disk storage for our CI builds to pick the already exported bundle archive.

[carolynvs]

I have an upstream PR to fix cnab-to-oci's copy functionality. I expect that to be merged soon and then I'll cut a release of Porter with the fix.

[jasmdk]

Thank you for your quick response - as usual :-)

[carolynvs]

This should now be fixed in v0.38.3