Generating a Crossplane Provider

In this guide, we will generate a Crossplane provider based on an existing Terraform provider using Upjet.

We have chosen Terraform GitHub provider as an example, but the process will be quite similar for any other Terraform provider. We will use myorg as the example organization name to be used.

Generate

Generate a GitHub repository for the Crossplane provider by hitting the "Use this template" button in upjet-provider-template repository. The preferred repository name is provider-<name> (e.g. provider-github), which is assumed by the ./hack/prepare.sh script in step 3.
Clone the repository to your local and cd into the repository directory. Fetch the upbound/build submodule by running the following:
```
make submodules
```
Replace template with your provider name.
1. Run the ./hack/prepare.sh script from repo root to prepare the repo, e.g., to replace all occurrences of template with your provider name and upbound with your organization name:
```
./hack/prepare.sh
```
To configure the Terraform provider to generate from, update the following variables in Makefile:
```
export TERRAFORM_PROVIDER_SOURCE := integrations/github
export TERRAFORM_PROVIDER_REPO := https://github.com/integrations/terraform-provider-github
export TERRAFORM_PROVIDER_VERSION := 5.5.0
export TERRAFORM_PROVIDER_DOWNLOAD_NAME := terraform-provider-github
export TERRAFORM_NATIVE_PROVIDER_BINARY := terraform-provider-github_v5.5.0_x5
export TERRAFORM_DOCS_PATH := website/docs/r
```
- TERRAFORM_PROVIDER_SOURCE: You can find this variable in Terraform GitHub provider documentation by hitting the "USE PROVIDER" button.
- TERRAFORM_PROVIDER_REPO: You can find this variable in Terraform GitHub provider documentation by clicking the "Report an issue" link.
- TERRAFORM_PROVIDER_VERSION: You can find this variable in Terraform GitHub provider documentation by hitting the "USE PROVIDER" button.
- TERRAFORM_PROVIDER_DOWNLOAD_NAME: terraform-provider-<Lower case provider name (ex. github)>.
- TERRAFORM_NATIVE_PROVIDER_BINARY: terraform-provider-github_v<TERRAFORM_PROVIDER_VERSION>_x5
- TERRAFORM_DOCS_PATH: You can find this by going to the terraform provider repository => click website => click docs => click resources(where resource documentation is stored).
Check this line in controller Dockerfile to see how TERRAFORM_PROVIDER_SOURCE and TERRAFORM_PROVIDER_VERSION are used to build the provider plugin binary.

Please make sure your organization name in PROJECT_REPO is correct.

Implement ProviderConfig logic. In upjet-provider-template, there is already a boilerplate code in file internal/clients/github.go which takes care of properly fetching secret data referenced from ProviderConfig resource.

For our GitHub provider, we need to check Terraform documentation for provider configuration and provide the keys there:

const (
  ...
  keyBaseURL = "base_url"
  keyOwner = "owner"
  keyToken = "token"
)

func TerraformSetupBuilder(version, providerSource, providerVersion string) terraform.SetupFn {
  ...
  // set provider configuration
  ps.Configuration = map[string]any{}
  if v, ok := creds[keyBaseURL]; ok {
    ps.Configuration[keyBaseURL] = v
  }
  if v, ok := creds[keyOwner]; ok {
    ps.Configuration[keyOwner] = v
  }
  if v, ok := creds[keyToken]; ok {
    ps.Configuration[keyToken] = v
  }
  return ps, nil
}

Before generating all resources that the provider has, let's go step by step and only start with generating CRDs for github_repository and github_branch Terraform resources.

Only the resources with external name configuration should be generated. Let's add external name configurations for these two resources in config/external_name.go as an entry to the map called ExternalNameConfigs:

// ExternalNameConfigs contains all external name configurations for this
// provider.
var ExternalNameConfigs = map[string]config.ExternalName{
  ...
  // Name is a parameter and it is also used to import the resource.
  "github_repository": config.NameAsIdentifier,
  // The import ID consists of several parameters. We'll use branch name as
  // the external name.
  "github_branch": config.TemplatedStringAsIdentifier("branch", "{{ .parameters.repository }}:{{ .external_name }}:{{ .parameters.source_branch }}"),
}

Please take a look at the Configuring a Resource documentation for more information about external name configuration.

Finally, we would need to add some custom configurations for these two resources as follows:

# Create custom configuration directory for whole repository group
mkdir config/repository
# Create custom configuration directory for whole branch group
mkdir config/branch

cat <<EOF > config/repository/config.go
package repository

import "github.com/upbound/upjet/pkg/config"

// Configure configures individual resources by adding custom ResourceConfigurators.
func Configure(p *config.Provider) {
    p.AddResourceConfigurator("github_repository", func(r *config.Resource) {
        // We need to override the default group that upjet generated for
        // this resource, which would be "github"
        r.ShortGroup = "repository"
    })
}
EOF

# Note that you need to change `myorg/provider-github`.
cat <<EOF > config/branch/config.go
package branch

import "github.com/upbound/upjet/pkg/config"

func Configure(p *config.Provider) {
    p.AddResourceConfigurator("github_branch", func(r *config.Resource) {
        // We need to override the default group that upjet generated for
        // this resource, which would be "github"
        r.ShortGroup = "branch"

        // This resource need the repository in which branch would be created
        // as an input. And by defining it as a reference to Repository
        // object, we can build cross resource referencing. See
        // repositoryRef in the example in the Testing section below.
        r.References["repository"] = config.Reference{
            Type: "github.com/myorg/provider-github/apis/repository/v1alpha1.Repository",
        }
    })
}
EOF

And register custom configurations in config/provider.go:

import (
    ...

    ujconfig "github.com/upbound/upjet/pkg/config"

-   "github.com/myorg/provider-github/config/null"
+   "github.com/myorg/provider-github/config/branch"
+   "github.com/myorg/provider-github/config/repository"
 )

 func GetProvider() *tjconfig.Provider {
    ...
    for _, configure := range []func(provider *tjconfig.Provider){
            // add custom config functions
-           null.Configure,
+           repository.Configure,
+           branch.Configure,
    } {
            configure(pc)
    }

To learn more about custom resource configurations (in step 7), please see the Configuring a Resource document.

Now we can generate our Upjet Provider:

Before we run make generate ensure to install goimports
```
go install golang.org/x/tools/cmd/goimports@latest
```
```
make generate
```

Adding More Resources

See the guide here to add more resources.

Test

Now let's test our generated resources.

First, we will create example resources under the examples directory:

Create example directories for repository and branch groups:

mkdir examples/repository
mkdir examples/branch

# remove the sample directory which was an example in the template
rm -rf examples/null

Create a provider secret template:

cat <<EOF > examples/providerconfig/secret.yaml.tmpl
apiVersion: v1
kind: Secret
metadata:
  name: example-creds
  namespace: crossplane-system
type: Opaque
stringData:
  credentials: |
    {
      "token": "y0ur-t0k3n"
    }
EOF

Create example for repository resource, which will use upjet-provider-template repo as template for the repository to be created:

cat <<EOF > examples/repository/repository.yaml
apiVersion: repository.github.upbound.io/v1alpha1
kind: Repository
metadata:
  name: hello-crossplane
spec:
  forProvider:
    description: "Managed with Crossplane Github Provider (generated with Upjet)"
    visibility: public
    template:
      - owner: upbound
        repository: upjet-provider-template
  providerConfigRef:
    name: default
EOF

Create branch resource which refers to the above repository managed resource:

cat <<EOF > examples/branch/branch.yaml
apiVersion: branch.github.upbound.io/v1alpha1
kind: Branch
metadata:
  name: hello-upjet
spec:
  forProvider:
    repositoryRef:
      name: hello-crossplane
  providerConfigRef:
    name: default
EOF

In order to change the apiVersion, you can use WithRootGroup and WithShortName options in config/provider.go as arguments to ujconfig.NewProvider.

Generate a Personal Access Token for your Github account with repo/public_repo and delete_repo scopes.

Create examples/providerconfig/secret.yaml from examples/providerconfig/secret.yaml.tmpl and set your token in the file:

GITHUB_TOKEN=<your-token-here>
cat examples/providerconfig/secret.yaml.tmpl | sed -e "s/y0ur-t0k3n/${GITHUB_TOKEN}/g" > examples/providerconfig/secret.yaml

Apply CRDs:
```
kubectl apply -f package/crds
```
Run the provider:

Please make sure Terraform is installed before running the "make run" command, you can check this guide.
```
make run
```

Apply ProviderConfig and example manifests (In another terminal since the previous command is blocking):

# Create "crossplane-system" namespace if not exists
kubectl create namespace crossplane-system --dry-run=client -o yaml | kubectl apply -f -

kubectl apply -f examples/providerconfig/
kubectl apply -f examples/repository/repository.yaml
kubectl apply -f examples/branch/branch.yaml

Observe managed resources and wait until they are ready:

watch kubectl get managed

NAME                                                   READY   SYNCED   EXTERNAL-NAME                     AGE
branch.branch.github.upbound.io/hello-upjet   True    True     hello-crossplane:hello-upjet   89s

NAME                                                             READY   SYNCED   EXTERNAL-NAME      AGE
repository.repository.github.upbound.io/hello-crossplane   True    True     hello-crossplane   89s

Verify that repo hello-crossplane and branch hello-upjet created under your GitHub account.

You can check the errors and events by calling kubectl describe for either of the resources.

Cleanup

kubectl delete -f examples/branch/branch.yaml
kubectl delete -f examples/repository/repository.yaml

Verify that the repo got deleted once deletion is completed on the control plane.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

generating-a-provider.md

generating-a-provider.md

Generating a Crossplane Provider

Generate

Adding More Resources

Test

Files

generating-a-provider.md

Latest commit

History

generating-a-provider.md

File metadata and controls

Generating a Crossplane Provider

Generate

Adding More Resources

Test