[Proposal] Add more instructions to enrich the Kubefile's capabilities

[starnop]

Issue Description

Type: feature request

Issue Description

Type: feature request

As we know, we can use the Kubefile to define a Sealer Image that can be shared and deployed, and the Kubefile has implemented the basic abilities. After that, we need to continue refining Kubefile and enriching its capabilities.

Let's first agree on what a complete Kubefile needs to contain. In my opinion, the following items need to be included:

• Get files and put them in place, such as FROM, COPY, APP
• Set default running commands and parameters, such as LAUNCH, CMDS
• Add some declarations, including supported capabilities and any information you want to inform the Sealer Image user. However, we do not currently support such instructions.

Combined with the design in this PR, the following suggestions are made for the Kubefile and Sealer Image Spec definition

cluster image:

FROM sealerio/kubernetes:v1.22.15

# Override or provide CNI plugins, the syntax is same with `APP`. In fact, we can think of CNI as a special kind of `APP`.
# And multiple `CNI` definitions can be specified. If the name is the same, the last one takes precedence.
# In addition, the CNI plugin is not only an app, but also reflected in the label `cluster.alpha.sealer.io/kube-cni-xxx`.
CNI calico local://install-calico.sh
CNI calico local://install-calico-new.sh
CNI flannel local://flannel.yaml

# Override or provide CRI plugins just like `CNI`.
# In addition, the CNI plugin is not only an app, but also reflected in the label `cluster.alpha.sealer.io/kube-csi-xxx`.
CSI alibaba-cloud-csi-driver local://install-alibaba-cloud-csi-driver.sh

# Add some metadata to an image just like `Dockerfile LABEL instruction`.
# An image can have more than one label. You can specify multiple labels on a single line.
# 
# NOTE: 
# 1. In general, a base image should declare supported cluster runtimes, container runtimes, etc.
# 2. Sealer reserves all label keys prefixed with `sealer.io`.
# 3. All labels will be inherited by child image.
LABEL <key>=<value> <key>=<value> <key>=<value> ...
LABEL <key>=<value> \
      <key>=<value> \
      <key>=<value> ...
LABEL "cluster.alpha.sealer.io/cluster-runtime-version"="v1.22.15"
LABEL "cluster.alpha.sealer.io/cluster-runtime-type"="kubernetes" # such as kubernetes, k0s, k3s, etc
LABEL "cluster.alpha.sealer.io/container-runtime-type"="docker"
LABEL "cluster.alpha.sealer.io/container-runtime-version"="20.10.14"

# NOTE: we need a clear launch `CNI`, `CSI` here just like launch `APP`.
LAUNCH ["calico"]

cluster image spec:

{
    "id": "bb75382891e7f04f192f1baeab18ef9c9f5503f4de8ac6dfc2a4d94f2164dde6",
    "name": "docker.io/sealerio/kubernetes:v1.22.15",
    "digest": "sha256:2f92b0149053ece9de6c683754f76fb9fd023a44540a9e33fc371afb8b76cc1b",
    "manifestv1": {
       ......
    },
    "ociv1": {
        ......
    },
    "buildClient": {
        "sealerVersion": "v0.9.0",
        "buildahVersion": "1.27.1"
    },
    "schemaVersion": "v1alpha1",
    "type": "kube-installer",
    "applications": [
        {
            "name": "calico",
            "type": "shell",
            "launchfiles": [
                "install-calico-new.sh"
            ],
            "version": "v1"
        }
        {
            "name": "flannel",
            "type": "kube",
            "launchfiles": [
                "flannel.yaml"
            ],
            "version": "v1"
        }
        {
            "name": "alibaba-cloud-csi-driver",
            "type": "shell",
            "launchfiles": [
                "install-alibaba-cloud-csi-driver.sh"
            ],
            "version": "v1"
        }
    ],
    "labels": {
      "cluster.alpha.sealer.io/cluster-runtime-version": "v1.22.15",
      "cluster.alpha.sealer.io/cluster-runtime-type": "kubernetes",
      "cluster.alpha.sealer.io/container-runtime-type": "docker",
      "cluster.alpha.sealer.io/container-runtime-version": "20.10.14",
      "cluster.alpha.sealer.io/kube-cni": ["calico", "flannel"],
      "cluster.alpha.sealer.io/kube-csi": ["alibaba-cloud-csi-driver"]
    },
    "launch": {
        "app_names": [
            "calico",
            "alibaba-cloud-csi-driver",
        ]
    }
}

app image:

FROM scratch

# A SemVer range of compatible Kubernetes versions by the applications.
# If there are multiple apps in the `Kubefile`, you should take the intersection of the cluster versions they support
# NOTE: This value will not be inherited.
# The label `app.alpha.sealer.io/supported-kube-version` will be gernetated with `KUBEVERSION`.
# For a detailed explanation of supported semver constraints see [Masterminds/semver](https://github.com/Masterminds/semver).
KUBEVERSION 1.22 - 1.24

# add redis app
# ["kubectl apply -f redis.yaml"] will be executed
APP redis local://redis.yaml

# add nginx app
# ["kubectl apply -f nginx.yaml -n nginx-namespace"] will be executed
APP nginx local://nginx.yaml
# `APPCMDS` specify the cmds for a specified app and the context is the dir of specified app.
# The `APPCMDS` instruction format: ["executable","param1","param2"]
# If there are more than one `APPCMDS` for a `APP` then only the last `APPCMDS` will take effect.
APPCMDS nginx ["kubectl apply -f nginx.yaml -n nginx-namespace"]

# add mysql app
# ["helm install mysql . -f values.yaml --set key=value"] will be executed
APP mysql local://charts/mysql/ local://values.yaml
# `APPCMDS` must be specified when the app has multiple files
APPCMDS mysql ["helm install mysql .",  "-f values.yaml",  "--set key=value"]

LABEL <key>=<value> <key>=<value> <key>=<value> ...
LABEL "created-by"="sealer.io"

# The following operation will be executed:
# ["kubectl apply -f nginx.yaml -n nginx-namespace", "helm install mysql . -f values.yaml --set key=value"]
LAUNCH ["nginx", "mysql"]

app image spec

{
  "id": "783d1c3814b9cb28dafd7c3ca34b734d5798a61dc523c0d30349636e7cd56cc6",
  "name": "cloud-image-registry.cn-shanghai.cr.aliyuncs.com/applications/1137465199671599:cnstack-acos-0.0.5-beta-07b8866-70f202",
  "digest": "sha256:ebdd1badbb67f0f3de27317252618759db11782928bd206e9b155d5cc81f2076",
  "manifestv1": {
    ......
  },
  "ociv1": {
    ......
  },
  "buildClient": {
    "sealerVersion": "0.9",
    "buildahVersion": "0.0.1"
  },
  "schemaVersion": "0.1",
  "type": "app-image",
  "applications": [
    {
      "name": "nginx",
      "type": "kube",
      "launchfiles": [
        "nginx.yaml"
      ],
      "version": "v1"
    },
    {
      "name": "mysql",
      "type": "helm",
      "launchfiles": [
        "mysql.tgz"
      ],
      "version": "v1"
    },
  ],
  "labels": {
    "app.alpha.sealer.io/supported-kube-version": "1.22 - 1.24",
    "created-by": "sealer.io"
  },
  "launch": {
    "cmds": [],
    "appConfigs": [
      {
        "name": "nginx",
        "launch": {
          "cmds": [
            "kubectl apply -f nginx.yaml -n nginx-namespace"
          ]
       }
      },
      {
        "name": "mysql",
        "launch": {
          "cmds": [
            "helm install mysql .",
            "-f values.yaml",
            "--set key=value"
          ]
         }
      },
    ],
    "app_names": [
      "redis",
      "nginx",
      "mysql",
    ]
  }
}

Describe what feature you want

Additional context

Add any other context or screenshots about the feature request here.

Describe what feature you want

Additional context

Add any other context or screenshots about the feature request here.

In my opinion, we have two remaining problems to resolve:

1. How to modify the container runtime configuration and cluster runtime configuration in the Sealer Kubefile? Currently, files in basefs can be overwritten by COPY instruction, however, it is not a very standard, user-friendly approach.
2. How to delete the app when we delete a Sealer image? So far the sealer delete is cluster-oriented, but now that we support running an app Sealer Image, we should support deleting an app Sealer Image as well. For helm and kube type apps, the deletion operation is also standard by helm uninstall or kubectl delete. However, how do we implement uninstallation operations for shell-type apps?

[justadogistaken]

Looks very professional. But I have one question.

APPCMDS

Q: I learn that there will be only one APPCMDS for an app, will there be the situation that users need more than one execuation commands for a single app?