Skip to content

Commit

Permalink
Add examples to user stories
Browse files Browse the repository at this point in the history 
Signed-off-by: Stefan Prodan <[email protected]>
  • Loading branch information
@stefanprodan
stefanprodan committed Mar 29, 2022
1 parent 746f389 commit 83e88a6
Showing 1 changed file with 154 additions and 93 deletions.
247 changes: 154 additions & 93 deletions rfcs/0006-oci-artifacts/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,62 +4,174 @@

**Creation date:** 2022-03-24

**Last update:** 2022-03-24
**Last update:** 2022-03-29

## Summary

We want to make it possible to reconcile workload from OCI Registries, the same we can reconcile today from git or Helm repositories. The targets are Helm charts and kubernetes manifests.
We want to make it possible to reconcile workloads from OCI Registries, the same we can reconcile today from Git or Helm repositories.
The targets are Helm charts and Kubernetes manifests packaged as OCI artifacts.

## Motivation

Registries are evolving into generic artifact stores. Many of our users would like to take advantage of this fact in order to store their kubernetes declaration and Helm charts in registries and apply them using Flux. Helm already supports oci registries to store Charts.
Registries are evolving into generic artifact stores.
Many of our users would like to take advantage of this fact in order to store their Kubernetes manifests
and Helm charts in OCI registries and reconcile them with Flux. Helm already supports OCI as Charts storage.

This [request](https://github.com/fluxcd/source-controller/issues/124) has been made to the Flux team to add support for Helm charts in OCI registries.
This [request](https://github.com/fluxcd/source-controller/issues/124) has been made to the Flux team
to add support for Helm charts stored in OCI registries.

### Goals

- Introduce a new source that can reconcile 2 OCI artifact types from OCI compliant registries, Helm artifact and OCI image.
- This new source must respect the existing source contract, i.e. it must produce the same outputs as the existing sources. this will enable existing consumers to use the new source with minimal effort.
- Introduce a new source that can reconcile artifacts from OCI compliant registries, Helm artifact and OCI image.
- This new source must respect the existing source contract, i.e. it must produce the same outputs as the existing sources.
This will enable existing consumers to use the new source with minimal effort.
- Pave the way for eventually supporting other OCI artifact types.

### Non-Goals

- Introduce a new OCI artifact type that can be used to store kubernetes manifests in OCI registries.
- Introduce a new OCI artifact type that can be used to store Kubernetes manifests in OCI registries.

## Proposal

Introduce 2 new Flux resources:
Introduce two Flux resources:
- `OCIRegistry`, which holds the registry information, i.e. URL and credentials.
- `OCIArtifact`, which holds the artifact information, mainly a reference to the desired tag, semantic version or digest.

An `OCIArtifact` can be used to reference a Helm Chart published with a Helm artifact media type or any tar package published with an OCI Image media type.
An `OCIArtifact` can be used to reference a Helm Chart published with a Helm artifact media type or any tar package
published with an OCI Image media type.
For any other artifact type, we will return an event stating that the OCI artifact type is not supported.

The OCI artifact reconciler will retrieve an artifact based on the url and credentials provided by the reference `OCIRegistry`, and store the artifact locally for dependent controllers to consume.

The Helm Chart reconciler will retrieve an artifact based on the url and credentials provided by the reference `OCIRegistry`, build the chart, and store the artifact locally for dependent controllers to consume.
The OCI artifact reconciler will retrieve an artifact based on the URL and credentials provided by the reference `OCIRegistry`,
and store the artifact locally for kustomize-controller to consume.

The Helm Chart reconciler will retrieve an artifact based on the URL and credentials provided by the reference `OCIRegistry`,
build the chart, and store the artifact locally for helm-controller to consume.

### User Stories

#### Story 1

As a developper I want to push my kubernetes manifests as a tar archive to an OCI Registry, and then use Flux to reconcile it in my cluster.
> As a developer I want to push my Kubernetes manifests as a tar archive to an OCI Registry,
and then use a Flux `Kustomization` to reconcile it in my cluster.


First you define an `OCIRegistry` and setup authentication:

```yaml
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: OCIRegistry
metadata:
name: ghcr-podinfo
namespace: flux-system
spec:
url: ghcr.io/stefanprodan
# required for private registries
auth:
# one of
secretRef:
name: regcred
serviceAccountName: reg
```
Then you define an `OCIArtifact` using the `OCIRegistry`:

```yaml
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: OCIArtifact
metadata:
name: podinfo
namespace: flux-system
spec:
repository: podinfo-deploy
ociRegistryRef:
name: ghcr-podinfo
ref:
# one of
tag: "latest"
digest: "sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2"
semver: "6.0.x"
ignore: ".md, .txt"
interval: 10m
timeout: 1m
```

And finally in the Flux `Kustomizations`, the `OCIArtifact` can be used instead of a `GitRepository`:

```yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
name: podinfo
namespace: flux-system
spec:
sourceRef:
kind: OCIArtifact
name: podinfo
path: ./
prune: true
targetNamespace: default
wait: true
interval: 60m
timeout: 2m
```

#### Story 2

As a developper I want to push my Helm charts using the Helm cli to an OCI Registry, and then use Flux to reconcile it in my cluster.
> As a developer I want to push my Helm charts using the Helm CLI to an OCI Registry,
and then use a Flux `HelmRelease` to reconcile it in my cluster.

First you define an `OCIRegistry` and setup authentication if needed:

```yaml
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: OCIRegistry
metadata:
name: ghcr-podinfo
namespace: flux-system
spec:
url: ghcr.io/stefanprodan
# required for private registries
auth:
# one of
secretRef:
name: regcred
serviceAccountName: reg
```

Then in Flux `HelmReleases`, the `OCIRegistry` can be used instead of a `HelmRepository`:

```yaml
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: podinfo
spec:
interval: 60m
chart:
spec:
chart: podinfo
version: '6.0.x'
sourceRef:
kind: OCIRegistry
name: ghcr-podinfo
interval: 1m
```

### Alternatives

We could keep the `OCIRepository` proposal and tie `OCIArtifact` and `OCIRegistry` specs. That is considered unpractical, as we would like to be able to reuse a registry spec for different artifact types with different reconcilers.
We could use the `OCIRepository` [proposal](https://github.com/fluxcd/source-controller/pull/450)
which bundles both an `OCIArtifact` and `OCIRegistry`. That is considered unpractical,
as we would like to be able to reuse a registry spec
for different artifact types with different reconcilers.

Instead of reusing the `oci image` mediaType to adress the need to store Kubernetes manifests declaration, we could register our own artifact type. This is also considered un practical, as declaring a type has to go through the IANA process and Flux is not the owner of those type as Helm is for Helm artifact for example.
Instead of reusing the `oci image` mediaType to address the need to store Kubernetes manifests declaration,
we could register our own artifact type. This is also considered un practical,
as declaring a type has to go through the IANA process and Flux is not the owner of those type
as Helm is for Helm artifact for example.

## Design Details

The new APIs:
API spec:

```go
type OCIRegistrySpec struct {
Expand Down Expand Up @@ -89,31 +201,29 @@ type OCIRepositoryAuth struct {
}

type OCIArtifactSpec struct {
// Name is the name of the artifact to pull.
// Repository is the name of the artifact repository.
// +required
Name string `json:"name"`
Repository string `json:"repository"`

// Pull artifacts using this provider.
// Pull artifacts using this provider.
// +required
OCIRegistryRef meta.LocalObjectReference `json:"ociRegistryRef"`

// The OCI reference to pull and monitor for changes, defaults to
// latest tag.
// The repository reference to pull and monitor for changes, defaults to
// the latest tag.
// +optional
Reference *OCIRepositoryRef `json:"ref,omitempty"`

// The interval at which to check for image updates.
// The interval at which to check for updates.
// +required
Interval metav1.Duration `json:"interval"`

// The timeout for remote OCI Repository operations like pulling, defaults to 20s.
// +kubebuilder:default="20s"
// The timeout for remote OCI Repository operations like pulling, defaults to 60s.
// +kubebuilder:default="60s"
// +optional
Timeout *metav1.Duration `json:"timeout,omitempty"`

// Ignore overrides the set of excluded patterns in the .sourceignore format
// (which is the same as .gitignore). If not provided, a default will be used,
// consult the documentation for your version to find out what those are.
// Ignore is a set of excluded patterns in the .gitignore format.
// +optional
Ignore *string `json:"ignore,omitempty"`

Expand All @@ -140,63 +250,6 @@ type OCIRepositoryRef struct {
}
```

An `OCIArtifact` declaration would be:

```yaml
apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: OCIRegistry
metadata:
name: ghcr-podinfo
namespace: flux-system
spec:
url: ghcr.io/stefanprodan
# one of secretRef or serviceAccountName
auth:
secretRef:
name: regcred
serviceAccountName: reg

---
apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: OCIArtifact
metadata:
name: podinfo-deploy
namespace: flux-system
spec:
name: podinfo-deploy
ociRegistryRef:
name: ghcr-podinfo
interval: 10m
timeout: 1m
url: ghcr.io/stefanprodan/podinfo-deploy
ref:
# one of
tag: "latest"
digest: "sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2"
semver: "1.x"
serviceAccountName: reg
ignore: ".git, .out"
suspend: true
```
for HelmChart, we would have something like:
```yaml
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: podinfo-deploy
spec:
interval: 5m
chart:
spec:
chart: podinfo-deploy
version: '4.0.x'
sourceRef:
kind: OCIRegistry
name: ghcr-podinfo
interval: 1m
```
We will introduce our own client based on the `oras` library. This will allow us to decide the supported artifact types and the way to retrieve them.

```go
Expand Down Expand Up @@ -258,7 +311,7 @@ type OCIRegistrySpec struct {
// +optional
Authentication *OCIRepositoryAuth `json:"auth,omitempty"`

// +optional
// +optional
AccessFrom *acl.AccessFrom `json:"accessFrom,omitempty"`
}

Expand All @@ -267,7 +320,7 @@ type OCIArtifactSpec struct {
// +required
Name string `json:"name"`

// Pull artifacts using this provider.
// Pull artifacts using this provider.
// +required
OCIRegistryRef meta.LocalObjectReference `json:"ociRegistryRef"`

Expand Down Expand Up @@ -295,7 +348,7 @@ type OCIArtifactSpec struct {
// +optional
Suspend bool `json:"suspend,omitempty"`

// +optional
// +optional
AccessFrom *acl.AccessFrom `json:"accessFrom,omitempty"`
}
```
Expand All @@ -304,15 +357,21 @@ type OCIArtifactSpec struct {

`The OCI Registry` client pulls all artifacts layers for supported types.

The `OCIArtifact` and `HelmChart` reconciler will expect `data` to be in the first layer of the artifact, and will discard all the other layers. The controllers will then resolve the artifact dependencies, as it is a domain specific problem.
The `OCIArtifact` and `HelmChart` reconciler will expect `data` to be in the first layer of the artifact,
and will discard all the other layers. The controllers will then resolve the artifact dependencies,
as it is a domain specific problem.

For example, if we have a Helm artifact, the dependent Helm Chart controller will pull the artifact, extract the Chart and then resolve the chart dependencies.
For example, if we have a Helm artifact, the dependent Helm Chart controller will pull the artifact,
extract the Chart and then resolve the chart dependencies.

### Enabling the feature

The feature is enabled by default. You can disable it by setting the `enabe-oci-registry` flag to false. Which will prevent registering the oci reconcilers in the controller manager. For Helm, the flag is checked in the `HelmChart` reconciler.
The feature is enabled by default.

The Helm chart reconciler will have a new source for downloading the charts, but we will keep using the `remoteChartBuilder` to build the charts. The `remoteChartBuilder` will be modified to accept a `Remote` interface, in order to support the OCI Registry.
The Helm chart reconciler will have a new source for downloading the charts,
but we will keep using the `remoteChartBuilder` to build the charts. The `remoteChartBuilder`
will be modified to accept a `Remote` interface,
in order to support the OCI Registry.

## Follow-up work

Expand All @@ -322,7 +381,9 @@ There will be a specific Flux command (built on top of ORAS) to help users push
⋊> ~ flux push localhost:5000/podinfo:v1 -path ./podinfo/kustomize --username souleb
```

The artifact will be of type `application/vnd.oci.image.config.v1+json`. The directory pointed to by `path` will be archived and compressed to `tar+gzip`. The layer media type will be `application/vnd.oci.image.layer.v1.tar+gzip`.
The artifact will be of type `application/vnd.oci.image.config.v1+json`.
The directory pointed to by `path` will be archived and compressed to `tar+gzip`.
The layer media type will be `application/vnd.oci.image.layer.v1.tar+gzip`.

This will ease adoption of this new feature.

Expand Down

0 comments on commit 83e88a6

Please sign in to comment.