[BUG] Compatibility of 2.0 client with 1.x OpenSearch

[dblock]

What is the bug?

Coming from opensearch-project/opensearch-clients#17 and looking at 02a313f compatibility with 1.x client has been removed. Re-add it.

I expect that a 2.0 REST client should work against 1.x assuming you're not using any new features that exist only in 2.0.

What is the expected behavior?
opensearch-java 2.0 works against OpenSearch 1.x

[rursprung]

i think 2.x is a special case where it shouldn't only support OpenSearch 1.x but also be compatible with ES 7.10.2 as there will be many which'll use a direct upgrade path from ES 7.10.2 to OpenSearch 2.x (e.g. the reasons for us are missing features in 1.x which only landed in 2.0). however, as OpenSearch 1.x and ES 7.10.2 have a compatible API this shouldn't be a big issue as long as there's either no API version validation or it considers this fact and treats ES 7.10.2 as OpenSearch 1.x

[dblock]

@nknize do you have any opinions about this? what should users of opensearch-java on 1.x do to upgrade to 2.x without downtime?

[nknize]

Looks like the commit just removes any support for _types REST endpoints which was optional anyway? Do we have a specific blocker that concretely shows a 1.x client failing to connect to the 2.0 cluster? Seems like clients should still be able to use the 1.x version they just need to change their API calls to not ever call the builder.type("mytype"); method.

[dblock]

I'm working on #154, however what is the tl;dr for users that are upgrading now that type is gone? is it safe to just remove it in all cases?

[dblock]

Looks like the commit just removes any support for _types REST endpoints which was optional anyway? Do we have a specific blocker that concretely shows a 1.x client failing to connect to the 2.0 cluster? Seems like clients should still be able to use the 1.x version they just need to change their API calls to not ever call the builder.type("mytype"); method.

This happens to work in this case, but feels backwards. We can't retroactively always ensure that 1.x clients work against all future versions of OpenSearch, but we can ensure that newer version of the client do. Since this is a strongly typed client, as a user I'd prefer to 1) find all upgrade issues at compile time by upgrading the client to 2.0, 2) upgrade server to 2.0, 3) start using new features in 2.0.

[nknize]

We can't retroactively always ensure that 1.x clients work against all future versions of OpenSearch, but we can insure that newer version of the client do.

This is the point of the REST Versioning API (which should be released in 2.x). We can add a REST http header check, if the API version is null by the client it assumes < 2.0.0 and exposes the deprecated type endpoints. If anyone wants to live on the edge they could even explore this for GA. (Bias for action)

[dlvenable]

I agree with the goal of having the 2.x client work against 1.x. Data Prepper is a good use-case of this. We should be able to support Elasticsearch 7.x clusters through OpenSearch 2.x clusters since most of our interactions happen through a small set of APIs (mostly the _bulk API). But, the classic High Level Client (from OpenSearch core) makes some assertions in 1.x that are invalid against 2.x clusters.

If this new OpenSearch Java client can support both, it would be a good solution.

[dblock]

Tests against 1.x were re-added in #154. So the client will work against 1.x as long as you don't use any features removed in 2.0 (types). This creates an upgrade path that will tell you what is broken at compile time.

[dblock]

Closing this, see above.