-
Notifications
You must be signed in to change notification settings - Fork 25k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DOC] Repository GCS ADC not supported #33238
Merged
albertzaharovits
merged 3 commits into
elastic:master
from
albertzaharovits:doc-gcs-plugin-credentials-unsupported
Aug 30, 2018
Merged
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -10,10 +10,11 @@ include::install_remove.asciidoc[] | |
[[repository-gcs-usage]] | ||
==== Getting started | ||
|
||
The plugin uses the https://cloud.google.com/storage/docs/json_api/[Google Cloud Storage JSON API] (v1) | ||
to connect to the Storage service. If this is the first time you use Google Cloud Storage, you first | ||
need to connect to the https://console.cloud.google.com/[Google Cloud Platform Console] and create a new | ||
project. Once your project is created, you must enable the Cloud Storage Service for your project. | ||
The plugin uses the https://github.com/GoogleCloudPlatform/google-cloud-java/tree/master/google-cloud-clients/google-cloud-storage[Google Cloud Java Client for Storage] | ||
to connect to the Storage service. If this is the first time you use https://cloud.google.com/storage/[Google Cloud Storage] | ||
, you first need to connect to the https://console.cloud.google.com/[Google Cloud Platform Console] | ||
and create a new project. Once your project is created, you must enable the | ||
Cloud Storage Service for your project. | ||
|
||
[[repository-gcs-creating-bucket]] | ||
===== Creating a Bucket | ||
|
@@ -34,38 +35,30 @@ To create a new bucket: | |
7. Select a location | ||
8. Click the "Create" button | ||
|
||
The bucket should now be created. | ||
The bucket should now be created. For more detailed instructions follow the | ||
https://cloud.google.com/storage/docs/quickstart-console#create_a_bucket[oficial guide]. | ||
|
||
[[repository-gcs-service-authentication]] | ||
===== Service Authentication | ||
|
||
The plugin supports two authentication modes: | ||
|
||
* The built-in <<repository-gcs-using-compute-engine, Compute Engine authentication>>. This mode is | ||
recommended if your Elasticsearch node is running on a Compute Engine virtual machine. | ||
|
||
* Specifying <<repository-gcs-using-service-account, Service Account>> credentials. | ||
|
||
[[repository-gcs-using-compute-engine]] | ||
===== Using Compute Engine | ||
When running on Compute Engine, the plugin use Google's built-in authentication mechanism to | ||
authenticate on the Storage service. Compute Engine virtual machines are usually associated to a | ||
default service account. This service account can be found in the VM instance details in the | ||
https://console.cloud.google.com/compute/[Compute Engine console]. | ||
|
||
This is the default authentication mode and requires no configuration. | ||
|
||
NOTE: The Compute Engine VM must be allowed to use the Storage service. This can be done only at VM | ||
creation time, when "Storage" access can be configured to "Read/Write" permission. Check your | ||
instance details at the section "Cloud API access scopes". | ||
The plugin needs to authenticate the requests it is making to the Google Cloud | ||
Storage service. In this respect, it is common for google client libraries | ||
to employ a strategy named https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application[Application Default Credentials]. | ||
This, however, is **not** supported because the plugin operates under the | ||
Elasticsearch process which runs with the security manager enabled. This | ||
obstructs the "automatic" credential discovery. | ||
You will have to configure <<repository-gcs-using-service-account, Service Account>> | ||
credentials, even if running in an environment that normally is not requiring | ||
this configuration, such as Compute Engine, Kubernetes Engine or App Engine. | ||
|
||
[[repository-gcs-using-service-account]] | ||
===== Using a Service Account | ||
If your Elasticsearch node is not running on Compute Engine, or if you don't want to use Google's | ||
built-in authentication mechanism, you can authenticate on the Storage service using a | ||
https://cloud.google.com/iam/docs/overview#service_account[Service Account] file. | ||
You have to obtain and provide https://cloud.google.com/iam/docs/overview#service_account[service account credentials] | ||
manually. | ||
|
||
To create a service account file: | ||
The authoritative guide on generating JSON service account files is https://cloud.google.com/storage/docs/authentication?hl=en#service_accounts[here]. | ||
Note that the PKCS12 format is not supported by this plugin. | ||
Here is a concise breakdown of the guide: | ||
|
||
1. Connect to the https://console.cloud.google.com/[Google Cloud Platform Console] | ||
2. Select your project | ||
|
@@ -74,7 +67,7 @@ To create a service account file: | |
5. Click on "Create service account" | ||
6. Once created, select the new service account and download a JSON key file | ||
|
||
A service account file looks like this: | ||
A JSON service account file looks like this: | ||
|
||
[source,js] | ||
---- | ||
|
@@ -84,19 +77,25 @@ A service account file looks like this: | |
"private_key_id": "...", | ||
"private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n", | ||
"client_email": "service-account-for-your-repository@your-project-id.iam.gserviceaccount.com", | ||
"client_id": "..." | ||
"client_id": "...", | ||
"auth_uri": "https://accounts.google.com/o/oauth2/auth", | ||
"token_uri": "https://accounts.google.com/o/oauth2/token", | ||
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", | ||
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/[email protected]" | ||
} | ||
---- | ||
// NOTCONSOLE | ||
|
||
This file must be stored in the {ref}/secure-settings.html[elasticsearch keystore], under a setting name | ||
of the form `gcs.client.NAME.credentials_file`, where `NAME` is the name of the client configuration. | ||
The default client name is `default`, but a different client name can be specified in repository | ||
settings using `client`. | ||
To provide this file to the plugin, it must be stored in the {ref}/secure-settings.html[elasticsearch keystore] | ||
, under a setting name of the form `gcs.client.NAME.credentials_file`, where `NAME` | ||
is the name of the client configuration for the repository. The implicit client | ||
name is `default`, but a different client name can be specified in the | ||
repository settings with the `client` key. For emphasis, passing the file path | ||
via the GOOGLE_APPLICATION_CREDENTIALS environment variable is **not** supported. | ||
|
||
For example, if specifying the credentials file in the keystore under | ||
`gcs.client.my_alternate_client.credentials_file`, you can configure a repository to use these | ||
credentials like this: | ||
`gcs.client.my_alternate_client.credentials_file`, you can configure a | ||
repository to use these credentials like this: | ||
|
||
[source,js] | ||
---- | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think "google" should probably be capitalized here too.