[ML] Allow datafeed and job configs for datafeed preview API #70836

benwtrent · 2021-03-24T19:09:11Z

Previously, a datafeed and job must already exist for the _preview API to work.

With this change, users can get an accurate preview of the data that will be sent to the anomaly detection job
without creating either of them.

Example:

POST _ml/datafeeds/_preview
{
            "datafeed_config": {
              "indexes":"airline-data"
            },
            "job_config": {
              "analysis_config" : {
                "bucket_span": "1h",
                "detectors" :[{"function":"sum","field_name":"responsetime","by_field_name":"airline"}]
              },
              "data_description" : {
                "time_field":"time"
              }
            }
          }

closes #70264

…gs in datafeed preview

elasticmachine · 2021-03-24T19:09:14Z

Pinging @elastic/ml-core (Team:ML)

docs/reference/ml/anomaly-detection/apis/preview-datafeed.asciidoc

….com:benwtrent/elasticsearch into feature/ml-allow-configs-in-datafeed-preview

…idoc Co-authored-by: Lisa Cawley <[email protected]>

davidkyle

LGTM

Nice, I thought this would be a lot more complicated

davidkyle · 2021-03-25T17:16:21Z

...t/rest-high-level/src/test/java/org/elasticsearch/client/ml/PreviewDatafeedRequestTests.java

@@ -27,6 +30,6 @@ protected PreviewDatafeedRequest doParseInstance(XContentParser parser) throws I

    @Override
    protected boolean supportsUnknownFields() {
-        return true;
+        return false;


++ requests parsering shouldn't be lenient. Can you also change the parser in PreviewDatafeedRequest to have ignoreUnknownFields = false

davidkyle · 2021-03-25T17:32:07Z

.../plugin/core/src/main/java/org/elasticsearch/xpack/core/ml/action/PreviewDatafeedAction.java

+                        datafeedBuilder.setJobId("preview_job_id");
+                    }
+                }
+                if (jobBuilder != null) {


Is it valid to have a jobBuilder but no datafeedBuilder a null datafeedId? i.e. just the job config

Is it valid to have a jobBuilder but no datafeedBuilder a null datafeedId?

no, it isn't. The API MUST have either a datafeedBuilder or a datafeedId. The restrictive ctors ensure this (with null checks).

davidkyle · 2021-03-25T17:35:23Z

...lugin/ml/src/main/java/org/elasticsearch/xpack/ml/action/TransportPreviewDatafeedAction.java

+            listener::onFailure
+        );
+        if (request.getDatafeedConfig() != null) {
+            datafeedConfigActionListener.onResponse(request.getDatafeedConfig());


Well if I'd known it was that simple...

davidkyle · 2021-03-25T17:38:51Z

...in/ml/src/main/java/org/elasticsearch/xpack/ml/rest/datafeeds/RestPreviewDatafeedAction.java

@@ -35,8 +39,10 @@ public String getName() {

    @Override
    protected RestChannelConsumer prepareRequest(RestRequest restRequest, NodeClient client) throws IOException {
-        String datafeedId = restRequest.param(DatafeedConfig.ID.getPreferredName());
-        PreviewDatafeedAction.Request request = new PreviewDatafeedAction.Request(datafeedId);
+        String datafeedId = restRequest.param(DatafeedConfig.ID.getPreferredName(), null);


The defaultValue of null is unnecessary but I prefer this usage it makes the intent more explicit

Gotcha, I will only grab this param if hasContent is false. Also, I won't provide a default value that way it blows up expecting it.

benwtrent · 2021-03-26T12:09:08Z

run elasticsearch-ci/bwc

benwtrent · 2021-03-26T13:13:56Z

run elasticsearch-ci/2

benwtrent · 2021-03-26T16:09:16Z

@elasticmachine update branch

…#70836) Previously, a datafeed and job must already exist for the `_preview` API to work. With this change, users can get an accurate preview of the data that will be sent to the anomaly detection job without creating either of them. closes elastic#70264

…#70927) Previously, a datafeed and job must already exist for the `_preview` API to work. With this change, users can get an accurate preview of the data that will be sent to the anomaly detection job without creating either of them. closes #70264

[ML] this adds the ability for users to supply job and datafeed confi…

8936ff4

…gs in datafeed preview

benwtrent added >enhancement :ml Machine learning v8.0.0 v7.13.0 labels Mar 24, 2021

elasticmachine added the Team:ML Meta label for the ML team label Mar 24, 2021

[DOCS] Edits preview datafeeds API

4b488fd

lcawl reviewed Mar 24, 2021

View reviewed changes

docs/reference/ml/anomaly-detection/apis/preview-datafeed.asciidoc Outdated Show resolved Hide resolved

benwtrent and others added 4 commits March 25, 2021 07:01

fixing bwc serialization

2a0b1a0

Merge branch 'feature/ml-allow-configs-in-datafeed-preview' of github…

45284e9

….com:benwtrent/elasticsearch into feature/ml-allow-configs-in-datafeed-preview

making api a tad stricter

5fa7965

Update docs/reference/ml/anomaly-detection/apis/preview-datafeed.asci…

f7af8de

…idoc Co-authored-by: Lisa Cawley <[email protected]>

davidkyle approved these changes Mar 25, 2021

View reviewed changes

addressing pr comments

a2854cf

fixing parsing

72b91a6

Merge branch 'master' into feature/ml-allow-configs-in-datafeed-preview

4fdd157

benwtrent merged commit b796632 into elastic:master Mar 26, 2021

benwtrent deleted the feature/ml-allow-configs-in-datafeed-preview branch March 26, 2021 16:52

This was referenced Mar 26, 2021

[7.x] [ML] Allow datafeed and job configs for datafeed preview API (#70836) #70927

Merged

[ML] adjusting bwc for datafeed _preview #70928

Merged

benwtrent mentioned this pull request Apr 7, 2021

[ML] Anomaly detection wizard validation should run a datafeed _preview elastic/kibana#93771

Closed

stevejgordon mentioned this pull request Apr 21, 2021

7.13.0 Meta Ticket elastic/elasticsearch-net#5584

Closed

62 tasks

jgowdyelastic mentioned this pull request Apr 23, 2021

ML datafeed_preview needs updating elastic/elasticsearch-specification#377

Closed

jgowdyelastic mentioned this pull request Jun 9, 2021

[ML] Switching to new datafeed preview elastic/kibana#101780

Merged

3 tasks

jakelandis added v8.0.0-alpha1 and removed v8.0.0 labels Jul 26, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Allow datafeed and job configs for datafeed preview API #70836

[ML] Allow datafeed and job configs for datafeed preview API #70836

benwtrent commented Mar 24, 2021 •

edited

Loading

elasticmachine commented Mar 24, 2021

davidkyle left a comment

davidkyle Mar 25, 2021

benwtrent Mar 25, 2021

davidkyle Mar 25, 2021

benwtrent Mar 25, 2021

davidkyle Mar 25, 2021

davidkyle Mar 25, 2021

benwtrent Mar 25, 2021

benwtrent commented Mar 26, 2021

benwtrent commented Mar 26, 2021

benwtrent commented Mar 26, 2021

[ML] Allow datafeed and job configs for datafeed preview API #70836

[ML] Allow datafeed and job configs for datafeed preview API #70836

Conversation

benwtrent commented Mar 24, 2021 • edited Loading

elasticmachine commented Mar 24, 2021

davidkyle left a comment

Choose a reason for hiding this comment

davidkyle Mar 25, 2021

Choose a reason for hiding this comment

benwtrent Mar 25, 2021

Choose a reason for hiding this comment

davidkyle Mar 25, 2021

Choose a reason for hiding this comment

benwtrent Mar 25, 2021

Choose a reason for hiding this comment

davidkyle Mar 25, 2021

Choose a reason for hiding this comment

davidkyle Mar 25, 2021

Choose a reason for hiding this comment

benwtrent Mar 25, 2021

Choose a reason for hiding this comment

benwtrent commented Mar 26, 2021

benwtrent commented Mar 26, 2021

benwtrent commented Mar 26, 2021

benwtrent commented Mar 24, 2021 •

edited

Loading