Skip to content

Commit

Permalink
support reading configuration from Kubernetes Secrets
Browse files Browse the repository at this point in the history
The `kubernetes` extension automatically generates a RoleBinding
that refers to the `view` ClusterRole. This ClusterRole doesn't
allow access to secrets. This commit therefore adds a configuration
property which, when enabled, makes the `kubernetes` extension
generate a special Role `view-secrets` and a second RoleBinding
referring to that role. This configuration property is build-time
only and has no other effect.

With this configuration in place, there's nothing preventing
the application from reading Secrets directly from the API server.

For convenience, a warning is printed at runtime if configuration
is read from Secrets yet the property is disabled.
  • Loading branch information
Ladicek committed Jul 29, 2020
1 parent 2d1f4ee commit 29c6d11
Show file tree
Hide file tree
Showing 17 changed files with 374 additions and 33 deletions.
2 changes: 1 addition & 1 deletion bom/application/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@
<aws-alexa-sdk.version>2.31.0</aws-alexa-sdk.version>
<azure-functions-java-library.version>1.3.0</azure-functions-java-library.version>
<kotlin.version>1.3.72</kotlin.version>
<dekorate.version>0.12.6</dekorate.version>
<dekorate.version>0.12.7</dekorate.version>
<maven-artifact-transfer.version>0.10.0</maven-artifact-transfer.version>
<jline.version>2.14.6</jline.version>
<maven-invoker.version>3.0.1</maven-invoker.version>
Expand Down
25 changes: 17 additions & 8 deletions docs/src/main/asciidoc/kubernetes-config.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ https://github.com/quarkusio/quarkus/tree/master/docs/src/main/asciidoc
include::./attributes.adoc[]


Quarkus includes the `kubernetes-config` extension which allows developers to use Kubernetes https://cloud.google.com/kubernetes-engine/docs/concepts/configmap[ConfigMaps] as a configuration source, without having to mount the ConfigMaps into the https://kubernetes.io/docs/concepts/workloads/pods/pod/[Pod] running the Quarkus application.
Quarkus includes the `kubernetes-config` extension which allows developers to use Kubernetes https://cloud.google.com/kubernetes-engine/docs/concepts/configmap[ConfigMaps] and https://cloud.google.com/kubernetes-engine/docs/concepts/secret[Secrets] as a configuration source, without having to mount them into the https://kubernetes.io/docs/concepts/workloads/pods/pod/[Pod] running the Quarkus application.


== Configuration
Expand All @@ -33,19 +33,23 @@ This will add the following to your `pom.xml`:

== Usage

The extension works by reading ConfigMaps directly from the Kubernetes API server using the link:kubernetes-client[Kubernetes Client].
The extension works by reading ConfigMaps and Secrets directly from the Kubernetes API server using the link:kubernetes-client[Kubernetes Client].

The extension understands the following types of ConfigMaps as input sources:
The extension understands the following types of ConfigMaps and Secrets as input sources:

* ConfigMaps that contain literal data (see https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#create-configmaps-from-literal-values[this] for an example on how to create one)
* ConfigMaps created from files named, `application.properties`, `application.yaml` or `application.yml` (see https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#create-configmaps-from-files[this] for an example on how to create one).
* ConfigMaps and Secrets that contain literal data (see https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#create-configmaps-from-literal-values[this] for an example on how to create one)
* ConfigMaps and Secrets created from files named `application.properties`, `application.yaml` or `application.yml` (see https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#create-configmaps-from-files[this] for an example on how to create one).

To configure which ConfigMaps (from the namespace that the application runs in, or the explicitly configured namespace via `quarkus.kubernetes-client.namespace`), you can set the `quarkus.kubernetes-config.config-maps` property (in any of the usual Quarkus ways). If access to ConfigMaps from a specific namespace you can set the
`quarkus.kubernetes-config.namespace` property. Keep in mind however that you will also have to explicitly enable the retrieval of ConfigMaps by setting `quarkus.kubernetes-config.enabled=true` (the default is `false` in order to make it easy to test the application locally).
You have to explicitly enable the retrieval of ConfigMaps and Secrets by setting `quarkus.kubernetes-config.enabled=true`.
The default is `false` in order to make it easy to test the application locally.

Afterwards, set the `quarkus.kubernetes-config.configmaps` property to configure which ConfigMaps should be used.
Set the `quarkus.kubernetes-config.secrets` property to configure which Secrets should be used.
To access ConfigMaps and Secrets from a specific namespace, you can set the `quarkus.kubernetes-config.namespace` property.

=== Priority of obtained properties

The properties obtained from the ConfigMaps have a higher priority (i.e. they override) any properties of the same name that are found in `application.properties` (or the YAML equivalents), but they have lower priority than properties set via Environment Variables or Java System Properties.
The properties obtained from the ConfigMaps and Secrets have a higher priority than (i.e. they override) any properties of the same name that are found in `application.properties` (or the YAML equivalents), but they have lower priority than properties set via Environment Variables or Java System Properties.

=== Kubernetes Permissions

Expand All @@ -54,6 +58,11 @@ that is used to run the application needs to have the proper permissions for suc

Thankfully, when using the `kubernetes-config` extension along with the link:deploying-to-kubernetes[Kubernetes] extension, all the necessary Kubernetes resources to make that happen are automatically generated.

==== Secrets

By default, the link:deploying-to-kubernetes[Kubernetes] extension doesn't generate the necessary resources to allow accessing secrets.
Set `quarkus.kubernetes-config.secrets.enabled=true` to generate the necessary role and corresponding role binding.

== Configuration Reference

include::{generated-dir}/config/quarkus-kubernetes-config-kubernetes-config-source-config.adoc[opts=optional, leveloffset=+1]
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
import io.quarkus.deployment.util.JandexUtil;
import io.quarkus.jackson.deployment.IgnoreJsonDeserializeClassBuildItem;
import io.quarkus.kubernetes.client.runtime.KubernetesClientProducer;
import io.quarkus.kubernetes.spi.KubernetesRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;

public class KubernetesClientProcessor {

Expand All @@ -53,15 +53,15 @@ public class KubernetesClientProcessor {
BuildProducer<IgnoreJsonDeserializeClassBuildItem> ignoredJsonDeserializationClasses;

@Inject
BuildProducer<KubernetesRoleBuildItem> roleProducer;
BuildProducer<KubernetesRoleBindingBuildItem> roleBindingProducer;

@BuildStep
public void process(ApplicationIndexBuildItem applicationIndex, CombinedIndexBuildItem combinedIndexBuildItem,
BuildProducer<ExtensionSslNativeSupportBuildItem> sslNativeSupport,
BuildProducer<AdditionalBeanBuildItem> additionalBeanBuildItemBuildItem) {

featureProducer.produce(new FeatureBuildItem(Feature.KUBERNETES_CLIENT));
roleProducer.produce(new KubernetesRoleBuildItem("view"));
roleBindingProducer.produce(new KubernetesRoleBindingBuildItem("view", true));

// make sure the watchers fully (and not weakly) register Kubernetes classes for reflection
final Set<DotName> watchedClasses = new HashSet<>();
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
package io.quarkus.kubernetes.config.deployment;

import java.util.Arrays;
import java.util.Collections;

import org.jboss.logmanager.Level;

import io.quarkus.deployment.annotations.BuildProducer;
Expand All @@ -9,8 +12,11 @@
import io.quarkus.deployment.builditem.LogCategoryBuildItem;
import io.quarkus.deployment.builditem.RunTimeConfigurationSourceValueBuildItem;
import io.quarkus.kubernetes.client.runtime.KubernetesClientBuildConfig;
import io.quarkus.kubernetes.client.runtime.KubernetesConfigBuildTimeConfig;
import io.quarkus.kubernetes.client.runtime.KubernetesConfigRecorder;
import io.quarkus.kubernetes.client.runtime.KubernetesConfigSourceConfig;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBuildItem;

public class KubernetesConfigProcessor {

Expand All @@ -22,6 +28,25 @@ public RunTimeConfigurationSourceValueBuildItem configure(KubernetesConfigRecord
recorder.configSources(config, clientConfig));
}

@BuildStep
@Record(ExecutionTime.RUNTIME_INIT)
public void handleAccessToSecrets(KubernetesConfigSourceConfig config,
KubernetesConfigBuildTimeConfig buildTimeConfig,
BuildProducer<KubernetesRoleBuildItem> roleProducer,
BuildProducer<KubernetesRoleBindingBuildItem> roleBindingProducer,
KubernetesConfigRecorder recorder) {
if (buildTimeConfig.secretsEnabled) {
roleProducer.produce(new KubernetesRoleBuildItem("view-secrets", Collections.singletonList(
new KubernetesRoleBuildItem.PolicyRule(
Collections.singletonList(""),
Collections.singletonList("secrets"),
Arrays.asList("get", "list", "watch")))));
roleBindingProducer.produce(new KubernetesRoleBindingBuildItem("view-secrets", false));
}

recorder.warnAboutSecrets(config, buildTimeConfig);
}

// done in order to ensure that http logs aren't shown by default which happens because of the interplay between
// not yet setup logging (as the bootstrap config runs before logging is setup) and the configuration
// of the okhttp3.logging.HttpLoggingInterceptor by io.fabric8.kubernetes.client.utils.HttpClientUtils
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
package io.quarkus.kubernetes.client.runtime;

import io.quarkus.runtime.annotations.ConfigItem;
import io.quarkus.runtime.annotations.ConfigPhase;
import io.quarkus.runtime.annotations.ConfigRoot;

@ConfigRoot(name = "kubernetes-config", phase = ConfigPhase.BUILD_TIME)
public class KubernetesConfigBuildTimeConfig {
/**
* Whether or not configuration can be read from secrets.
* If set to {@code true}, Kubernetes resources allowing access to secrets (role and role binding) will be generated.
*/
@ConfigItem(name = "secrets.enabled", defaultValue = "false")
public boolean secretsEnabled;
}
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,17 @@ public RuntimeValue<ConfigSourceProvider> configSources(KubernetesConfigSourceCo
KubernetesClientUtils.createClient(clientConfig)));
}

public void warnAboutSecrets(KubernetesConfigSourceConfig config, KubernetesConfigBuildTimeConfig buildTimeConfig) {
if (config.enabled
&& config.secrets.isPresent()
&& !config.secrets.get().isEmpty()
&& !buildTimeConfig.secretsEnabled) {
log.warn("Configuration is read from Secrets " + config.secrets.get()
+ ", but quarkus.kubernetes-config.secrets.enabled is false."
+ " Check if your application's service account has enough permissions to read secrets.");
}
}

private RuntimeValue<ConfigSourceProvider> emptyRuntimeValue() {
return new RuntimeValue<>(new EmptyConfigSourceProvider());
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,15 @@ public class KubernetesConfigSourceConfig {
public Optional<List<String>> configMaps;

/**
* Secrets to look for in the namespace that the Kubernetes Client has been configured for
* Secrets to look for in the namespace that the Kubernetes Client has been configured for.
* If you use this, you probably want to enable {@code quarkus.kubernetes-config.secrets.enabled}.
*/
@ConfigItem
public Optional<List<String>> secrets;

/**
* Namespace to look for config maps and secrets. If this is not specified, then the namespace configured in the kubectl config context
* is used. If the value is specified and the namespace doesn't exist, the application will fail to start.
* Namespace to look for config maps and secrets. If this is not specified, then the namespace configured in the kubectl
* config context is used. If the value is specified and the namespace doesn't exist, the application will fail to start.
*/
@ConfigItem
public Optional<String> namespace;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ public Iterable<ConfigSource> getConfigSources(ClassLoader forClassLoader) {
result.addAll(getConfigMapConfigSources(config.configMaps.get()));
}
if (config.secrets.isPresent()) {
// TODO generate a role with permissions to read secrets, and a role binding for that role
result.addAll(getSecretConfigSources(config.secrets.get()));
}
return result;
Expand Down Expand Up @@ -119,7 +118,8 @@ private void logMissingOrFail(String name, String namespace, String type, boolea
String message = type + " '" + name + "' not found";
if (namespace == null) {
message = message
+ ". No Kubernetes namespace was set (most likely because the application is running outside the Kubernetes cluster). Consider setting 'quarkus.kubernetes-client.namespace=my-namespace' to specify the namespace in which to look up the " + type;
+ ". No Kubernetes namespace was set (most likely because the application is running outside the Kubernetes cluster). Consider setting 'quarkus.kubernetes-client.namespace=my-namespace' to specify the namespace in which to look up the "
+ type;
} else {
message = message + " in namespace '" + namespace + "'";
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
package io.quarkus.kubernetes.spi;

import io.quarkus.builder.item.MultiBuildItem;

/**
* Produce this build item to request the Kubernetes extension to generate
* a Kubernetes {@code RoleBinding} resource. The configuration here is limited;
* in particular, you can't specify subjects of the role binding. The role will always
* be bound to the application's service account.
* <p>
* Note that this can't be used to generate a {@code ClusterRoleBinding}.
*/
public final class KubernetesRoleBindingBuildItem extends MultiBuildItem {
/**
* Name of the generated {@code RoleBinding} resource.
* Can be {@code null}, in which case the resource name is autogenerated.
*/
private final String name;
/**
* Name of the bound role.
*/
private final String role;
/**
* If {@code true}, the binding refers to a {@code ClusterRole}, otherwise to a namespaced {@code Role}.
*/
private final boolean clusterWide;

public KubernetesRoleBindingBuildItem(String role, boolean clusterWide) {
this(null, role, clusterWide);
}

public KubernetesRoleBindingBuildItem(String name, String role, boolean clusterWide) {
this.name = name;
this.role = role;
this.clusterWide = clusterWide;
}

public String getName() {
return this.name;
}

public String getRole() {
return this.role;
}

public boolean isClusterWide() {
return clusterWide;
}
}
Original file line number Diff line number Diff line change
@@ -1,16 +1,79 @@
package io.quarkus.kubernetes.spi;

import java.util.List;

import io.quarkus.builder.item.MultiBuildItem;

/**
* Produce this build item to request the Kubernetes extension to generate
* a Kubernetes {@code Role} resource.
* <p>
* Note that this can't be used to generate a {@code ClusterRole}.
*/
public final class KubernetesRoleBuildItem extends MultiBuildItem {
/**
* Name of the generated {@code Role} resource.
*/
private final String name;
/**
* The {@code PolicyRule} resources for this {@code Role}.
*/
private final List<PolicyRule> rules;

public KubernetesRoleBuildItem(String name, List<PolicyRule> rules) {
this.name = name;
this.rules = rules;
}

private final String role;
public String getName() {
return name;
}

public KubernetesRoleBuildItem(String role) {
this.role = role;
public List<PolicyRule> getRules() {
return rules;
}

public String getRole() {
return this.role;
/**
* Corresponds directly to the Kubernetes {@code PolicyRule} resource.
*/
public static final class PolicyRule {
private final List<String> apiGroups;
private final List<String> nonResourceURLs;
private final List<String> resourceNames;
private final List<String> resources;
private final List<String> verbs;

public PolicyRule(List<String> apiGroups, List<String> resources, List<String> verbs) {
this(apiGroups, null, null, resources, verbs);
}

public PolicyRule(List<String> apiGroups, List<String> nonResourceURLs, List<String> resourceNames,
List<String> resources, List<String> verbs) {
this.apiGroups = apiGroups;
this.nonResourceURLs = nonResourceURLs;
this.resourceNames = resourceNames;
this.resources = resources;
this.verbs = verbs;
}

public List<String> getApiGroups() {
return apiGroups;
}

public List<String> getNonResourceURLs() {
return nonResourceURLs;
}

public List<String> getResourceNames() {
return resourceNames;
}

public List<String> getResources() {
return resources;
}

public List<String> getVerbs() {
return verbs;
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
package io.quarkus.kubernetes.deployment;

import java.util.stream.Collectors;

import io.dekorate.deps.kubernetes.api.model.KubernetesListBuilder;
import io.dekorate.deps.kubernetes.api.model.ObjectMeta;
import io.dekorate.deps.kubernetes.api.model.rbac.PolicyRuleBuilder;
import io.dekorate.deps.kubernetes.api.model.rbac.RoleBuilder;
import io.dekorate.kubernetes.decorator.ResourceProvidingDecorator;
import io.quarkus.kubernetes.spi.KubernetesRoleBuildItem;

class AddRoleResourceDecorator extends ResourceProvidingDecorator<KubernetesListBuilder> {
private final KubernetesRoleBuildItem spec;

public AddRoleResourceDecorator(KubernetesRoleBuildItem buildItem) {
this.spec = buildItem;
}

public void visit(KubernetesListBuilder list) {
ObjectMeta meta = getMandatoryDeploymentMetadata(list);

if (contains(list, "rbac.authorization.k8s.io/v1", "Role", spec.getName())) {
return;
}

list.addToItems(new RoleBuilder()
.withNewMetadata()
.withName(spec.getName())
.withLabels(meta.getLabels())
.endMetadata()
.withRules(
spec.getRules()
.stream()
.map(it -> new PolicyRuleBuilder()
.withApiGroups(it.getApiGroups())
.withNonResourceURLs(it.getNonResourceURLs())
.withResourceNames(it.getResourceNames())
.withResources(it.getResources())
.withVerbs(it.getVerbs())
.build())
.collect(Collectors.toList())));
}
}
Loading

0 comments on commit 29c6d11

Please sign in to comment.