Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REST high-level client: add clear cache API #28866

Merged
merged 10 commits into from
Mar 20, 2018
Merged

REST high-level client: add clear cache API #28866

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
271da22
REST high-level client: add clear cache API
javanna Mar 1, 2018
c0dd793
Merge branch 'master' into enhancement/rest_hl_client_clear_cache
javanna Mar 2, 2018
22cb53e
Merge branch 'master' into enhancement/rest_hl_client_clear_cache
javanna Mar 6, 2018
cddc2e0
address some of the review comments
javanna Mar 6, 2018
6814616
Merge branch 'master' into enhancement/rest_hl_client_clear_cache
javanna Mar 16, 2018
8acef57
prevent NPEs if indices are null, add tests and skip encoding of know…
javanna Mar 19, 2018
f848991
Merge branch 'master' into enhancement/rest_hl_client_clear_cache
javanna Mar 19, 2018
1611f65
remove unnecessary null check
javanna Mar 19, 2018
0ee3c95
iter
javanna Mar 19, 2018
ee5d389
fix get index issue
javanna Mar 19, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions client/rest-high-level/src/main/java/org/elasticsearch/client/IndicesClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest;
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesResponse;
import org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest;
import org.elasticsearch.action.admin.indices.cache.clear.ClearIndicesCacheRequest;
import org.elasticsearch.action.admin.indices.cache.clear.ClearIndicesCacheResponse;
import org.elasticsearch.action.admin.indices.close.CloseIndexRequest;
import org.elasticsearch.action.admin.indices.close.CloseIndexResponse;
import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
Expand Down Expand Up @@ -259,6 +261,29 @@ public void flushAsync(FlushRequest flushRequest, ActionListener<FlushResponse>
listener, emptySet(), headers);
}

/**
* Clears the cache of one or more indices using the Clear Cache API
* <p>
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-clearcache.html">
* Clear Cache API on elastic.co</a>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit : do we need this \t here? ( we did not use it in the rest of the javadocs )

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need it here otherwise the line is too long

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @olcbean talks about the indent of the new line here (line 268), to remove it and align with other lines :) like in other comments.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh right the tab :) that we don't need

*/
public ClearIndicesCacheResponse clearCache(ClearIndicesCacheRequest clearIndicesCacheRequest, Header... headers) throws IOException {
return restHighLevelClient.performRequestAndParseEntity(clearIndicesCacheRequest, Request::clearCache,
ClearIndicesCacheResponse::fromXContent, emptySet(), headers);
}

/**
* Asynchronously clears the cache of one or more indices using the Clear Cache API
* <p>
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-clearcache.html">
* Clear Cache API on elastic.co</a>
*/
public void clearCacheAsync(ClearIndicesCacheRequest clearIndicesCacheRequest, ActionListener<ClearIndicesCacheResponse> listener,
Header... headers) {
restHighLevelClient.performRequestAsyncAndParseEntity(clearIndicesCacheRequest, Request::clearCache,
ClearIndicesCacheResponse::fromXContent, listener, emptySet(), headers);
}

/**
* Checks if the index (indices) exists or not.
* <p>
Expand Down
12 changes: 12 additions & 0 deletions client/rest-high-level/src/main/java/org/elasticsearch/client/Request.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
import org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsRequest;
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest;
import org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest;
import org.elasticsearch.action.admin.indices.cache.clear.ClearIndicesCacheRequest;
import org.elasticsearch.action.admin.indices.close.CloseIndexRequest;
import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
Expand Down Expand Up @@ -234,6 +235,17 @@ static Request flush(FlushRequest flushRequest) {
return new Request(HttpPost.METHOD_NAME, endpoint, parameters.getParams(), null);
}

static Request clearCache(ClearIndicesCacheRequest clearIndicesCacheRequest) {
String endpoint = endpoint(clearIndicesCacheRequest.indices(), "_cache", "clear");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this result a NPE if there are no indices specified ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great catch, this is true for every broadcast response

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed this

Params parameters = Params.builder();
parameters.withIndicesOptions(clearIndicesCacheRequest.indicesOptions());
parameters.putParam("query", Boolean.toString(clearIndicesCacheRequest.queryCache()));
parameters.putParam("fielddata", Boolean.toString(clearIndicesCacheRequest.fieldDataCache()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit : isn't field_data the preferred name ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the \rest-api-spec\src\main\resources\rest-api-spec\api\indices.clear_cache.json as well? ( do we need to specify all supported params, or only the preferred ones. There is also a recycler flag in the rest specs which I do not see in the code )

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the fielddata thing is a mistake in the way we use ParseField to read parameters. Looking at the history, they are just synonyms, we never deprecated one in favour of the other. We use fielddata in our docs though. You are welcome to send a PR to remove support for the other deprecated params against master though ;)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I only went with the code ( without going into details ).

However, I just called GET _cache/clear?fielddata in Kibana and got

#! Deprecation: Deprecated field [fielddata] used, expected [field_data] instead
{
  "_shards": {
    "total": 2,
    "successful": 1,
    "failed": 0
  }
}

The deprecation seems to be wrapped in the ParseField#match

We use fielddata in our docs though.

Good catch! Maybe it would be better to have a separate PR for the ref docs ( so that we can track when the preferred name has been changed and since when a deprecation msg is logged ) ( I can take this one over if you want )

You are welcome to send a PR to remove support for the other deprecated params against master though ;)

Which other params do you mean? And remove them from the the rest api specs and the ref docs ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which other params do you mean? And remove them from the the rest api specs and the ref docs ?

I meant filter, filter_cache, and request_cache. I think we can remove them now in master.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fielddata is the preferred name as of my merging #28943 today.

parameters.putParam("request", Boolean.toString(clearIndicesCacheRequest.requestCache()));
parameters.putParam("fields", String.join(",", clearIndicesCacheRequest.fields()));
return new Request(HttpPost.METHOD_NAME, endpoint, parameters.getParams(), null);
}

static Request info() {
return new Request(HttpGet.METHOD_NAME, "/", Collections.emptyMap(), null);
}
Expand Down
33 changes: 31 additions & 2 deletions client/rest-high-level/src/test/java/org/elasticsearch/client/IndicesClientIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest.AliasActions;
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesResponse;
import org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest;
import org.elasticsearch.action.admin.indices.cache.clear.ClearIndicesCacheRequest;
import org.elasticsearch.action.admin.indices.cache.clear.ClearIndicesCacheResponse;
import org.elasticsearch.action.admin.indices.close.CloseIndexRequest;
import org.elasticsearch.action.admin.indices.close.CloseIndexResponse;
import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
Expand Down Expand Up @@ -431,9 +433,36 @@ public void testFlush() throws IOException {
{
String nonExistentIndex = "non_existent_index";
assertFalse(indexExists(nonExistentIndex));
FlushRequest refreshRequest = new FlushRequest(nonExistentIndex);
FlushRequest flushRequest = new FlushRequest(nonExistentIndex);
ElasticsearchException exception = expectThrows(ElasticsearchException.class,
() -> execute(refreshRequest, highLevelClient().indices()::flush, highLevelClient().indices()::flushAsync));
() -> execute(flushRequest, highLevelClient().indices()::flush, highLevelClient().indices()::flushAsync));
assertEquals(RestStatus.NOT_FOUND, exception.status());
}
}

public void testClearCache() throws IOException {
{
String index = "index";
Settings settings = Settings.builder()
.put("number_of_shards", 1)
.put("number_of_replicas", 0)
.build();
createIndex(index, settings);
ClearIndicesCacheRequest clearCacheRequest = new ClearIndicesCacheRequest(index);
ClearIndicesCacheResponse clearCacheResponse =
execute(clearCacheRequest, highLevelClient().indices()::clearCache, highLevelClient().indices()::clearCacheAsync);
assertThat(clearCacheResponse.getTotalShards(), equalTo(1));
assertThat(clearCacheResponse.getSuccessfulShards(), equalTo(1));
assertThat(clearCacheResponse.getFailedShards(), equalTo(0));
assertThat(clearCacheResponse.getShardFailures(), equalTo(BroadcastResponse.EMPTY));
}
{
String nonExistentIndex = "non_existent_index";
assertFalse(indexExists(nonExistentIndex));
ClearIndicesCacheRequest clearCacheRequest = new ClearIndicesCacheRequest(nonExistentIndex);
ElasticsearchException exception = expectThrows(ElasticsearchException.class,
() -> execute(clearCacheRequest, highLevelClient().indices()::clearCache,
highLevelClient().indices()::clearCacheAsync));
assertEquals(RestStatus.NOT_FOUND, exception.status());
}
}
Expand Down
35 changes: 35 additions & 0 deletions client/rest-high-level/src/test/java/org/elasticsearch/client/RequestTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest;
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest.AliasActions;
import org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest;
import org.elasticsearch.action.admin.indices.cache.clear.ClearIndicesCacheRequest;
import org.elasticsearch.action.admin.indices.close.CloseIndexRequest;
import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
Expand Down Expand Up @@ -580,6 +581,40 @@ public void testFlush() {
assertThat(request.getMethod(), equalTo(HttpPost.METHOD_NAME));
}

public void testClearCache() {
String[] indices = randomIndicesNames(0, 5);
ClearIndicesCacheRequest clearIndicesCacheRequest = new ClearIndicesCacheRequest(indices);
Map<String, String> expectedParams = new HashMap<>();
setRandomIndicesOptions(clearIndicesCacheRequest::indicesOptions, clearIndicesCacheRequest::indicesOptions, expectedParams);
if (randomBoolean()) {
clearIndicesCacheRequest.queryCache(randomBoolean());
}
expectedParams.put("query", Boolean.toString(clearIndicesCacheRequest.queryCache()));
if (randomBoolean()) {
clearIndicesCacheRequest.fieldDataCache(randomBoolean());
}
expectedParams.put("fielddata", Boolean.toString(clearIndicesCacheRequest.fieldDataCache()));
if (randomBoolean()) {
clearIndicesCacheRequest.requestCache(randomBoolean());
}
expectedParams.put("request", Boolean.toString(clearIndicesCacheRequest.requestCache()));
if (randomBoolean()) {
clearIndicesCacheRequest.fields(randomIndicesNames(1, 5));
expectedParams.put("fields", String.join(",", clearIndicesCacheRequest.fields()));
}

Request request = Request.clearCache(clearIndicesCacheRequest);
StringJoiner endpoint = new StringJoiner("/", "/", "");
if (indices.length > 0) {
endpoint.add(String.join(",", indices));
}
endpoint.add("_cache/clear");
assertThat(request.getEndpoint(), equalTo(endpoint.toString()));
assertThat(request.getParameters(), equalTo(expectedParams));
assertThat(request.getEntity(), nullValue());
assertThat(request.getMethod(), equalTo(HttpPost.METHOD_NAME));
}

public void testUpdate() throws IOException {
XContentType xContentType = randomFrom(XContentType.values());

Expand Down
87 changes: 87 additions & 0 deletions ...el/src/test/java/org/elasticsearch/client/documentation/IndicesClientDocumentationIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest.AliasActions;
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesResponse;
import org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest;
import org.elasticsearch.action.admin.indices.cache.clear.ClearIndicesCacheRequest;
import org.elasticsearch.action.admin.indices.cache.clear.ClearIndicesCacheResponse;
import org.elasticsearch.action.admin.indices.close.CloseIndexRequest;
import org.elasticsearch.action.admin.indices.close.CloseIndexResponse;
import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
Expand Down Expand Up @@ -769,6 +771,91 @@ public void onFailure(Exception e) {
}
}

public void testClearCache() throws Exception {
RestHighLevelClient client = highLevelClient();

{
createIndex("index1", Settings.EMPTY);
}

{
// tag::clear-cache-request
ClearIndicesCacheRequest request = new ClearIndicesCacheRequest("index1"); // <1>
ClearIndicesCacheRequest requestMultiple = new ClearIndicesCacheRequest("index1", "index2"); // <2>
ClearIndicesCacheRequest requestAll = new ClearIndicesCacheRequest(); // <3>
// end::clear-cache-request

// tag::clear-cache-request-indicesOptions
request.indicesOptions(IndicesOptions.lenientExpandOpen()); // <1>
// end::clear-cache-request-indicesOptions

// tag::clear-cache-request-query
request.queryCache(true); // <1>
// end::clear-cache-request-query

// tag::clear-cache-request-request
request.requestCache(true); // <1>
// end::clear-cache-request-request

// tag::clear-cache-request-fielddata
request.fieldDataCache(true); // <1>
// end::clear-cache-request-fielddata

// tag::clear-cache-request-fields
request.fields("field1", "field2", "field3"); // <1>
// end::clear-cache-request-fields

// tag::clear-cache-execute
ClearIndicesCacheResponse clearCacheResponse = client.indices().clearCache(request);
// end::clear-cache-execute

// tag::clear-cache-response
int totalShards = clearCacheResponse.getTotalShards(); // <1>
int successfulShards = clearCacheResponse.getSuccessfulShards(); // <2>
int failedShards = clearCacheResponse.getFailedShards(); // <3>
DefaultShardOperationFailedException[] failures = clearCacheResponse.getShardFailures(); // <4>
// end::clear-cache-response

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe check the returned values ?

// tag::clear-cache-execute-listener
ActionListener<ClearIndicesCacheResponse> listener = new ActionListener<ClearIndicesCacheResponse>() {
@Override
public void onResponse(ClearIndicesCacheResponse clearCacheResponse) {
// <1>
}

@Override
public void onFailure(Exception e) {
// <2>
}
};
// end::clear-cache-execute-listener

// Replace the empty listener by a blocking listener in test
final CountDownLatch latch = new CountDownLatch(1);
listener = new LatchedActionListener<>(listener, latch);

// tag::clear-cache-execute-async
client.indices().clearCacheAsync(request, listener); // <1>
// end::clear-cache-execute-async

assertTrue(latch.await(30L, TimeUnit.SECONDS));
}

{
// tag::clear-cache-notfound
try {
ClearIndicesCacheRequest request = new ClearIndicesCacheRequest("does_not_exist");
client.indices().clearCache(request);
} catch (ElasticsearchException exception) {
if (exception.status() == RestStatus.NOT_FOUND) {
// <1>
}
}
// end::clear-cache-notfound
}
}


public void testCloseIndex() throws Exception {
RestHighLevelClient client = highLevelClient();

Expand Down
109 changes: 109 additions & 0 deletions docs/java-rest/high-level/indices/clear_cache.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
[[java-rest-high-clear-cache]]
=== Clear Cache API

[[java-rest-high-clear-cache-request]]
==== Clear Cache Request

A `ClearIndicesCacheRquest` can be applied to one or more indices, or even on
`_all` the indices:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[clear-cache-request]
--------------------------------------------------
<1> Clears the cache of one index
<2> Clears the cache of multiple indices
<3> Clears the cache of all the indices

==== Optional arguments

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[clear-cache-request-indicesOptions]
--------------------------------------------------
<1> Setting `IndicesOptions` controls how unavailable indices are resolved and
how wildcard expressions are expanded

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[clear-cache-request-query]
--------------------------------------------------
<1> Set the `query` flag to `true`

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[clear-cache-request-fielddata]
--------------------------------------------------
<1> Set the `fielddata` flag to `true`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit : s/fielddata/field_data/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fielddata is correct now!


["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[clear-cache-request-request]
--------------------------------------------------
<1> Set the `request` flag to `true`

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[clear-cache-request-fields]
--------------------------------------------------
<1> Set the `fields` parameter

[[java-rest-high-clear-cache-sync]]
==== Synchronous Execution

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[clear-cache-execute]
--------------------------------------------------

[[java-rest-high-clear-cache-async]]
==== Asynchronous Execution

The asynchronous execution of a clear cache request requires both the `ClearIndicesCacheRequest`
instance and an `ActionListener` instance to be passed to the asynchronous
method:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[clear-cache-execute-async]
--------------------------------------------------
<1> The `ClearIndicesCacheRequest` to execute and the `ActionListener` to use when
the execution completes

The asynchronous method does not block and returns immediately. Once it is
completed the `ActionListener` is called back using the `onResponse` method
if the execution successfully completed or using the `onFailure` method if
it failed.

A typical listener for `ClearIndicesCacheResponse` looks like:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[clear-cache-execute-listener]
--------------------------------------------------
<1> Called when the execution is successfully completed. The response is
provided as an argument
<2> Called in case of failure. The raised exception is provided as an argument

[[java-rest-high-clear-cache-response]]
==== Clear Cache Response

The returned `ClearIndicesCacheResponse` allows to retrieve information about the
executed operation as follows:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[clear-cache-response]
--------------------------------------------------
<1> Total number of shards hit by the clear cache request
<2> Number of shards where the clear cache has succeeded
<3> Number of shards where the clear cache has failed
<4> A list of failures if the operation failed on one or more shards

By default, if the indices were not found, an `ElasticsearchException` will be thrown:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[clear-cache-notfound]
--------------------------------------------------
<1> Do something if the indices to be cleared were not found
2 changes: 2 additions & 0 deletions docs/java-rest/high-level/supported-apis.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ Index Management::
* <<java-rest-high-split-index>>
* <<java-rest-high-refresh>>
* <<java-rest-high-flush>>
* <<java-rest-high-clear-cache>>
* <<java-rest-high-rollover-index>>

Mapping Management::
Expand All @@ -72,6 +73,7 @@ include::indices/shrink_index.asciidoc[]
include::indices/split_index.asciidoc[]
include::indices/refresh.asciidoc[]
include::indices/flush.asciidoc[]
include::indices/clear_cache.asciidoc[]
include::indices/rollover.asciidoc[]
include::indices/put_mapping.asciidoc[]
include::indices/update_aliases.asciidoc[]
Expand Down
Loading