Skip to content
This repository has been archived by the owner on Jan 19, 2022. It is now read-only.

Commit

Permalink
Support auto-configuration of the Datastore emulator (#1661)
Browse files Browse the repository at this point in the history
- Support for `spring.cloud.gcp.datastore.emulator.enabled` property
- `spring.cloud.gcp.datastore.emulator-host` deprecated in favor of `spring.cloud.gcp.datastore.host`
- The autoconfiguration itself manage the lifecycle to start and stop the emulator

Fixes #1642.
  • Loading branch information
lucasoares authored and meltsufin committed May 16, 2019
1 parent db02ef2 commit 99f76da
Show file tree
Hide file tree
Showing 9 changed files with 451 additions and 11 deletions.
15 changes: 14 additions & 1 deletion docs/src/main/asciidoc/datastore.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,10 @@ The following configuration options are available:
| `spring.cloud.gcp.datastore.credentials.encoded-key` | Base64-encoded OAuth2 credentials for authenticating with the Google Cloud Datastore API, if different from the ones in the <<spring-cloud-gcp-core,Spring Cloud GCP Core Module>> | No |
| `spring.cloud.gcp.datastore.credentials.scopes` | https://developers.google.com/identity/protocols/googlescopes[OAuth2 scope] for Spring Cloud GCP Cloud Datastore credentials | No | https://www.googleapis.com/auth/datastore
| `spring.cloud.gcp.datastore.namespace` | The Cloud Datastore namespace to use | No | the Default namespace of Cloud Datastore in your GCP project
| `spring.cloud.gcp.datastore.emulator-host` | The `hostname:port` of the https://cloud.google.com/datastore/docs/tools/datastore-emulator[Datastore Emulator] to connect to | No |
| `spring.cloud.gcp.datastore.host` | The `hostname:port` of the datastore service or emulator to connect to. Can be used to connect to a manually started https://cloud.google.com/datastore/docs/tools/datastore-emulator[Datastore Emulator]. If the autoconfigured emulator is enabled, this property will be ignored and `localhost:<emulator_port>` will be used. | No |
| `spring.cloud.gcp.datastore.emulator.enabled` | To enable the auto configuration to start a local instance of the Datastore Emulator. | No | `false`
| `spring.cloud.gcp.datastore.emulator.port` | The local port to use for the Datastore Emulator | No | `8081`
| `spring.cloud.gcp.datastore.emulator.consistency` | The https://cloud.google.com/sdk/gcloud/reference/beta/emulators/datastore/start?#--consistency[consistency] to use for the Datastore Emulator instance | No | `0.9`
|===

==== Repository settings
Expand All @@ -88,6 +91,16 @@ Our Spring Boot autoconfiguration creates the following beans available in the S
- an instance of all user defined repositories extending `CrudRepository`, `PagingAndSortingRepository`, and `DatastoreRepository` (an extension of `PagingAndSortingRepository` with additional Cloud Datastore features) when repositories are enabled
- an instance of `Datastore` from the Google Cloud Java Client for Datastore, for convenience and lower level API access

==== Datastore Emulator Autoconfiguration

This Spring Boot autoconfiguration can also configure and start a local Datastore Emulator server if enabled by property.

It is useful for integration testing, but not for production.

When enabled, the `spring.cloud.gcp.datastore.host` property will be ignored and the Datastore autoconfiguration itself will be forced to connect to the autoconfigured local emulator instance.

It will create an instance of `LocalDatastoreHelper` as a bean that stores the `DatastoreOptions` to get the `Datastore` client connection to the emulator for convenience and lower level API for local access.
The emulator will be properly stopped after the Spring application context shutdown.

=== Object Mapping

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
/*
* Copyright 2017-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.springframework.cloud.gcp.autoconfigure.datastore;

/**
* Properties for configuring Cloud Datastore Emulator.
*
* @author Lucas Soares
*
* @since 1.2
*/
public class EmulatorSettings {
/**
* If enabled the Datastore client will connect to an local datastore emulator.
*/
private boolean enabled;

/**
* Is the datastore emulator port. Default: {@code 8081}
*/
private int port = 8081;

/**
* Consistency to use creating the Datastore server instance. Default: {@code 0.9}
*/
private double consistency = 0.9D;

public boolean isEnabled() {
return enabled;
}

public void setEnabled(boolean enabled) {
this.enabled = enabled;
}

public int getPort() {
return port;
}

public void setPort(int port) {
this.port = port;
}

public double getConsistency() {
return consistency;
}

public void setConsistency(double consistency) {
this.consistency = consistency;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
import com.google.cloud.NoCredentials;
import com.google.cloud.datastore.Datastore;
import com.google.cloud.datastore.DatastoreOptions;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
Expand Down Expand Up @@ -60,13 +62,15 @@
@EnableConfigurationProperties(GcpDatastoreProperties.class)
public class GcpDatastoreAutoConfiguration {

private static final Log LOGGER = LogFactory.getLog(GcpDatastoreEmulatorAutoConfiguration.class);

private final String projectId;

private final String namespace;

private final Credentials credentials;

private final String emulatorHost;
private final String host;

GcpDatastoreAutoConfiguration(GcpDatastoreProperties gcpDatastoreProperties,
GcpProjectIdProvider projectIdProvider,
Expand All @@ -77,7 +81,13 @@ public class GcpDatastoreAutoConfiguration {
: projectIdProvider.getProjectId();
this.namespace = gcpDatastoreProperties.getNamespace();

if (gcpDatastoreProperties.getEmulatorHost() == null) {
String hostToConnect = gcpDatastoreProperties.getHost();
if (gcpDatastoreProperties.getEmulator().isEnabled()) {
hostToConnect = "localhost:" + gcpDatastoreProperties.getEmulator().getPort();
LOGGER.info("Connecting to a local datastore emulator.");
}

if (hostToConnect == null) {
this.credentials = (gcpDatastoreProperties.getCredentials().hasKey()
? new DefaultCredentialsProvider(gcpDatastoreProperties)
: credentialsProvider).getCredentials();
Expand All @@ -87,7 +97,7 @@ public class GcpDatastoreAutoConfiguration {
this.credentials = NoCredentials.getInstance();
}

this.emulatorHost = gcpDatastoreProperties.getEmulatorHost();
this.host = hostToConnect;
}

@Bean
Expand All @@ -101,8 +111,8 @@ public Datastore datastore() {
builder.setNamespace(this.namespace);
}

if (emulatorHost != null) {
builder.setHost(emulatorHost);
if (this.host != null) {
builder.setHost(this.host);
}

return builder.build().getService();
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
/*
* Copyright 2017-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.springframework.cloud.gcp.autoconfigure.datastore;

import com.google.cloud.datastore.testing.LocalDatastoreHelper;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import org.springframework.boot.autoconfigure.AutoConfigureBefore;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.SmartLifecycle;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
* If spring.cloud.gcp.datastore.emulator.enabled is set to true the emulator will be
* started as a local datastore server using the
* {@link com.google.cloud.datastore.testing.LocalDatastoreHelper}.
*
* @author Lucas Soares
*
* @since 1.2
*/
@Configuration
@ConditionalOnProperty("spring.cloud.gcp.datastore.emulator.enabled")
@AutoConfigureBefore(GcpDatastoreAutoConfiguration.class)
@EnableConfigurationProperties(GcpDatastoreProperties.class)
@ConditionalOnMissingBean(LocalDatastoreHelper.class)
public class GcpDatastoreEmulatorAutoConfiguration implements SmartLifecycle {

private static final Log LOGGER = LogFactory.getLog(GcpDatastoreEmulatorAutoConfiguration.class);

private LocalDatastoreHelper helper;

private volatile boolean running;

@Bean
public LocalDatastoreHelper createDatastoreHelper(
GcpDatastoreProperties datastoreProperties) {
EmulatorSettings settings = datastoreProperties.getEmulator();

this.helper = LocalDatastoreHelper.create(settings.getConsistency(),
settings.getPort());

return this.helper;
}

/** Stops the instance of the emulator. */
@Override
public void stop() {
if (!isRunning()) {
LOGGER.warn("The datastore emulator is not running.");

return;
}

try {
LOGGER.info("Stopping datastore emulator.");

this.helper.stop();

LOGGER.info("Datastore emulator stopped.");

this.running = false;
}
catch (Exception e) {
LOGGER.warn("Failed to stop datastore instance.", e);
}
}

/**
* Checks if the instance is running. This will be <code>true</code> after a successful
* execution of the method {@link #start()} and <code>false</code> after a successful
* execution of the method {@link #stop()}. method is called.
*/
@Override
public boolean isRunning() {
return this.running;
}

/**
* Starts the instance of the emulator.
*/
@Override
public void start() {
if (isRunning()) {
LOGGER.warn("The datastore emulator is already running.");
return;
}

try {
LOGGER.info("Starting datastore emulator.");

this.helper.start();

LOGGER.info("Datastore emulator started.");

this.running = true;
}
catch (Exception e) {
LOGGER.error("Error constructing datastore instance.");
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,23 @@ public class GcpDatastoreProperties implements CredentialsSupplier {
private final Credentials credentials = new Credentials(GcpScope.DATASTORE.getUrl());

/**
* The host and port of the local running emulator. If provided, this will setup the
* client to connect against a running pub/sub emulator.
* Properties to auto configure a local Datastore Emulator.
*/
@NestedConfigurationProperty
private final EmulatorSettings emulator = new EmulatorSettings();

/**
* @deprecated use <code>spring.cloud.gcp.datastore.host</code> instead.
* @see #host
*/
@Deprecated
private String emulatorHost;

/**
* The host and port of a Datastore emulator as the following example: localhost:8081.
*/
private String host;

private String projectId;

private String namespace;
Expand All @@ -51,6 +63,10 @@ public Credentials getCredentials() {
return this.credentials;
}

public EmulatorSettings getEmulator() {
return this.emulator;
}

public String getProjectId() {
return this.projectId;
}
Expand All @@ -67,10 +83,28 @@ public void setNamespace(String namespace) {
this.namespace = namespace;
}

public String getHost() {
return this.host;
}

public void setHost(String host) {
this.host = host;
}

/**
* @return the old emulator host parameter
* @deprecated use {@link #getHost()} instead
*/
@Deprecated
public String getEmulatorHost() {
return emulatorHost;
return this.emulatorHost;
}

/**
* @param emulatorHost the emulator post
* @deprecated use {@link #setHost(String)} instead
*/
@Deprecated
public void setEmulatorHost(String emulatorHost) {
this.emulatorHost = emulatorHost;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ org.springframework.cloud.gcp.autoconfigure.trace.StackdriverTraceAutoConfigurat
org.springframework.cloud.gcp.autoconfigure.datastore.DatastoreRepositoriesAutoConfiguration,\
org.springframework.cloud.gcp.autoconfigure.spanner.SpannerRepositoriesAutoConfiguration,\
org.springframework.cloud.gcp.autoconfigure.security.IapAuthenticationAutoConfiguration,\
org.springframework.cloud.gcp.autoconfigure.vision.CloudVisionAutoConfiguration
org.springframework.cloud.gcp.autoconfigure.vision.CloudVisionAutoConfiguration,\
org.springframework.cloud.gcp.autoconfigure.datastore.GcpDatastoreEmulatorAutoConfiguration

org.springframework.cloud.bootstrap.BootstrapConfiguration=\
org.springframework.cloud.gcp.autoconfigure.config.GcpConfigBootstrapConfiguration
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ public class GcpDatastoreAutoConfigurationTests {
.withUserConfiguration(TestConfiguration.class)
.withPropertyValues("spring.cloud.gcp.datastore.project-id=test-project",
"spring.cloud.gcp.datastore.namespace=testNamespace",
"spring.cloud.gcp.datastore.emulator-host=localhost:8081",
"spring.cloud.gcp.datastore.host=localhost:8081",
"management.health.datastore.enabled=false");

@Test
Expand Down
Loading

0 comments on commit 99f76da

Please sign in to comment.