[skip ci] Data access service

[lukasolson]

Summary

Relies on #37563.

This PR introduces the new data access service API. It's the first step towards #34022. Specifically, this PR introduces a new server endpoint (/api/search) which accepts raw Elasticsearch search DSL and sends it to Elasticsearch.

This PR also introduces both a client- and server-side tool for making such requests.

Client-side API

  // Simple example
  const index = 'twitter';
  const body = { query: { match_all: {} } };
  const results = await search({
    searchParams: { index, body }
  }).toPromise();

  // Example with aborting/handling multiple responses/progress
  const controller = new AbortController();
  window.addEventListener('unload', () => controller.abort());
  const index = 'twitter';
  const body = { query: { match_all: {} } };
  const results$ = search({
    searchParams: { index, body },
    signal
  });
  results$.subscribe({
    next: ({ _shards: { successful, failed, skipped, total }}),  => {
      console.log('percent complete: ' (successful + failed + skipped) / total);
    },
    complete: response => {
      console.log('got response: ', response);
    },
    error: error => {
      console.log('got error: ', error);
    }
  });

Server-side API

The same as the client-side API, except that the calls must also take the request that initiated the search, to forward credentials to Elasticsearch:

  const index = 'twitter';
  const body = { query: { match_all: {} } };
  const results = await search(request, {
    searchParams: { index, body }
  }).toPromise();

Eventually we intend this service/API to be a replacement for the existing Courier service.

Checklist

• [ ] This was checked for cross-browser compatibility, including a check against IE11
• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• [ ] This was checked for keyboard-only and screenreader accessibility

For maintainers

• This was checked for breaking API changes and was labeled appropriately
• [ ] This includes a feature addition or change that requires a release note and was labeled appropriately

[elasticmachine]

Pinging @elastic/kibana-app-arch

[elasticmachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request

[elasticmachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request

[ppisljar]

apart from some minor comments this looks good, needs test coverage still.

[ppisljar]

should we allow overriding any of the advanced settings here ?

[ppisljar]

add a getter rather than exposing whole 'registry'

[ppisljar]

nit: i would prefer exporting searchStrategy function here and doing registration init

[stacey-gammon]

I wonder if this can get moved to the x-pack search strategy. Maybe getSearchParams should be part of the search strategy? e.g.

export interface ISearchStrategy {
  search: (request: Request, searchArguments: SearchArguments) => Observable<SearchResponse<any>>;
  getSearchParams: (
    request: Request,
    searchParams: SearchParams,
    options: SearchOptions
  ) => SearchParams;
}

Then the x-pack implementation can use the default one plus incorporate some additional settings that are commercial only?

[stacey-gammon]

Any idea why maxConcurrentShardRequests isn't in the SearchParams type? Is it an oversight in the typing or is this a kibana specific param?

[stacey-gammon]

actually I see it's only part of msearch. I guess for search endpoint it's irrelevant.

[stacey-gammon]

this is returning any type because of config.get. If we know the shape of return value, maybe it can be typed out explicitly here?

[stacey-gammon]

ah, I see this comment now!

[lukasolson]

Superseded by #43371.