Store the whole Bundle.json and generate a Relocation map

[silvin-lubecki]

Context

During CNAB discussions about storing bundles into registry, some comments were raised about cnab-to-oci not storing the whole bundle.json in the registry but only a subset, reconstructing it during a pull.

Discussion around storing the complete bundle.json file (and duplicate the information about the reference images) vs. reconstructing it based on a configuration file (parameters, credentials, metadata - all other fields in bundle.json) and the referenced images. Whatever path we choose, the proposed changes to the OCI specification should allow both options. Same goes for signing - whether we choose to sign the complete bundle.json that is stored in the registry, or we sign the manifest generated by the registry, the proposed changes to the specification and the current CNAB security spec should allow for for both approaches.

Current behavior

When pushing a CNAB Bundle to a registry, cnab-to-oci produces the following:

• Push or Mount all the invocation and service images inside the same repository, and retrieve their digested reference
• Create a new config object, storing only a sub-set of the bundle.json (see the type here (with mostly parameters, actions and credentials) and retrieve its digest
  Ex:

{
  "schemaVersion": "v1.0.0-WD",
  "actions": {
    "com.docker.app.inspect": {
      "stateless": true
    },
    "com.docker.app.render": {
      "stateless": true
    },
    "io.cnab.status": {}
  },
  "definitions": {
    "port": {
      "default": "8080",
      "type": "string"
    },
    "text": {
      "default": "Hello, World!",
      "type": "string"
    }
  },
  "parameters": {
    "fields": {
      "port": {
        "definition": "port",
        "destination": {
          "env": "docker_param1"
        }
      },
      "text": {
        "definition": "text",
        "destination": {
          "env": "docker_param2"
        }
      }
    }
  },
  "credentials": {
    "docker.context": {
      "path": "/cnab/app/context.dockercontext"
    }
  }
} 

• Create an OCI Index (or Manifest List) with some informations found in the bundle.json like metadata (as annotations), or invocation/service images

{
  "schemaVersion": 2,
  "manifests": [
    {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "digest": "sha256:1f1470cc6d80bcf8057782bc38a2469a3d4c2833e804b98c19b6f4dedcddda4e",
      "size": 316,
      "annotations": {
        "io.cnab.manifest.type": "config"
      }
    },
    {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "digest": "sha256:720d8fa437af27e98a76ade5395b92c32a8e39721c62a6b4d8c3e4e1967d0caa",
      "size": 1363,
      "annotations": {
        "io.cnab.manifest.type": "invocation"
      }
    },
    {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "digest": "sha256:ba27d460cd1f22a1a4331bdf74f4fccbc025552357e8a3249c40ae216275de96",
      "size": 528,
      "annotations": {
        "io.cnab.component.name": "hello",
        "io.cnab.manifest.type": "component"
      }
    }
  ],
  "annotations": {
    "io.cnab.runtime_version": "v1.0.0-WD",
    "io.docker.app.format": "cnab",
    "io.docker.type": "app",
    "org.opencontainers.artifactType": "application/vnd.cnab.manifest.v1",
    "org.opencontainers.image.authors": "[{\"name\":\"user\",\"email\":\"[email protected]\"}]",
    "org.opencontainers.image.description": "Hello, World!",
    "org.opencontainers.image.title": "hello-world",
    "org.opencontainers.image.version": "0.1.0"
  }
} 

During the pull, we can reconstruct the bundle.json from the config object, the annotations and the images:

{
        "schemaVersion": "v1.0.0-WD",
        "name": "hello-world",
        "version": "0.1.0",
        "description": "Hello, World!",
        "maintainers": [
                {
                        "name": "user",
                        "email": "[email protected]"
                }
        ],
        "invocationImages": [
                {
                        "imageType": "docker",
                        "image": "slubecki/test1@sha256:720d8fa437af27e98a76ade5395b92c32a8e39721c62a6b4d8c3e4e1967d0caa",
                        "size": 1363,
                        "mediaType": "application/vnd.docker.distribution.manifest.v2+json"
                }
        ],
        "images": {
                "hello": {
                        "imageType": "docker",
                        "image": "slubecki/test1@sha256:ba27d460cd1f22a1a4331bdf74f4fccbc025552357e8a3249c40ae216275de96",
                        "size": 528,
                        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
                        "description": ""
                }
        },
        "actions": {
                "com.docker.app.inspect": {
                        "stateless": true
                },
                "com.docker.app.render": {
                        "stateless": true
                },
                "io.cnab.status": {}
        },
        "parameters": {
                "fields": {
                        "port": {
                                "definition": "port",
                                "destination": {
                                        "env": "docker_param1"
                                }
                        },
                        "text": {
                                "definition": "text",
                                "destination": {
                                        "env": "docker_param2"
                                }
                        }
                }
        },
        "credentials": {
                "docker.context": {
                        "path": "/cnab/app/context.dockercontext"
                }
        },
        "definitions": {
                "port": {
                        "default": "8080",
                        "type": "string"
                },
                "text": {
                        "default": "Hello, World!",
                        "type": "string"
                }
        }
}

This reconstruction was designed to tackle 2 things:

• avoiding duplication between information stored at OCI Index level (which can be directly parsed by registries)
• moving a CNAB from a repository to another means we need to update the bundle.json image references, and then to re-sign it, so moving would remove any guarantee seal. With this design moving an OCI index does not change anything, we can safely keep the signature.

Proposal

Since this design was implemented, the Image Relocation has been added to the CNAB specifications.

We propose to leverage this new feature in the Bundle Runtime so:

• whole bundle can be pushed as is, as relocating it does not need to patch the bundle but just generates locally a relocation map. cnab-to-oci can then pull the original bundle.json.
• cnab-to-oci generates the relocation map from the image references in the OCI Index.

Ex of Relocation map:

{
 "hashicorp/http-echo": "slubecki/test1@sha256:ba27d460cd1f22a1a4331bdf74f4fccbc025552357e8a3249c40ae216275de96"
}

This new design does not avoid annotation/metadata replication, but we think it is a minor issue, regarding the concern of storing a subset of the bundle.json.

[silvin-lubecki]

cc @jcsirot @radu-matei @glyn @caervs @chris-crone WDYT?

[caervs]

@silvin-lubecki neat! To be clear, the annotations that we have today would continue to exist, correct?

Maybe this is a separate conversation for us @chris-crone and @simonferquel (and maybe I need to re-read the spec), but I'd like to better understand the motivation for referencing images in external repos in the bundle.json in the first place. IMHO this is akin to referencing external resources in a website's HTML and something we could avoid by always mounting blobs into the target repo for a CNAB and referencing them in the bundle.json just by digest.

[silvin-lubecki]

To be clear, the annotations that we have today would continue to exist, correct?

@caervs absolutely, we don't change that part at all. The pushed OCI index is still the same with this proposal, only the config object changes.

[glyn]

I've been thinking about this and I'm not sure how the original image name would be obtained when pulling from or relocating from a bundle stored in a repository unless it is captured when pushing the bundle to the repository (and stored in the repository).

This issue crops up in the example above: where did "hashicorp/http-echo" come from in the relocation map?

[silvin-lubecki]

@glyn I think we have everything we need in the OCI index, we have an annotation with the component name, so we can link it to the original component in the bundle.json and get back the original image name to produce the relocation map.

Does that make sense? Am I missing something?

[glyn]

@silvin-lubecki Thanks. I understand now. I incorrectly assumed you were treating the original bundle.json as completely opaque, but of course you can get the original image names from it.

[radu-matei]

@caervs - the usage of annotations is still required, since an OCI index does not currently support a config object (where a concrete mediaType would be passed) - there have been proposals to add this to the OCI specification, but following that entire discussion is... let's say consuming.

As to the second part of your question, it has to do the following workflow:

• system A generates a bundle, pushes it to its own registry, and signs the content digest of bundle.json
• system B needs to use the bundle, but move it (and the images) to its own private registry. Because the container image names are part of bundle.json, rewriting them results in changing the bundle, thus changing the content digest of the bundle, resulting in invalidating the signature.

Constructing an image map that is used by runtimes (with the restriction that all images referenced in the image map must have the same digest with the original images) allows us to move bundles across registries (OCI compliant registries, that generate the same image digest for the same image) without invalidating the trust data generated when first building the bundle.

Does that make sense?

@silvin-lubecki, @glyn, @chris-crone - LGTM, I'm happy to pair sometime this / next week and go through the requirements and workflows one more time.

[silvin-lubecki]

@radu-matei happy to pair with you on this, let's schedule that on slack 👍