Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Add new custom rule action to force time shift #188710

Merged
merged 7 commits into from
Jul 26, 2024

Conversation

valeriy42
Copy link
Contributor

This PR follows up elastic/elasticsearch#110974 and extends the schema for custom rules.

A valid schema looks like this:

POST _ml/anomaly_detectors/my_job/_update
{
	"detectors": {
		"detector_index": 0,
		"custom_rules": [
			// update the detector with a custom rule that forces a time shift of 1 hour back starting now
			{
				"actions": [
					"force_time_shift"
				],
				"params": {
					"force_time_shift": {
"time_shift_amount": 3600}
					}
				},
				"conditions": [{
					"applies_to": "time",
					"operator": "gt",
					"value": "now"
				},
{
					"applies_to": "time",
					"operator": "lt",
					"value": "now+bucket_span"
				},

]
			},
			...

@valeriy42
Copy link
Contributor Author

@elastic/ml-ui This works as far as validation in the Advanced Wizard goes, but I noticed that custom rule actions are wired in a few more places (Customer rule editor UI among other things). Could someone please pick this PR up and extend the missing places?

@valeriy42 valeriy42 marked this pull request as ready for review July 24, 2024 14:25
@valeriy42 valeriy42 requested a review from a team as a code owner July 24, 2024 14:25
@valeriy42
Copy link
Contributor Author

@elasticmachine merge upstream

@valeriy42
Copy link
Contributor Author

@elastic/ml-ui Can someone please review my PR?

@peteharverson peteharverson added the Feature:Anomaly Detection ML anomaly detection label Jul 25, 2024
@valeriy42 valeriy42 requested a review from jgowdyelastic July 25, 2024 14:35
valeriy42 added a commit to elastic/elasticsearch that referenced this pull request Jul 25, 2024
This PR extends the schema of the detection rule API by adding a new parametrizable action `force_time_shift` and a new property `params` to hold the parameter `total_shift_amount` as signed long in seconds to control the behavior of the time shift.

A valid schema looks like this:
```http
POST _ml/anomaly_detectors/my_job/_update
{
	"detectors": {
		"detector_index": 0,
		"custom_rules": [
			// update the detector with a custom rule that forces a time shift of 1 hour back starting now
			{
				"actions": [
					"force_time_shift"
				],
				"params": {
					"force_time_shift": {
"time_shift_amount": 3600}
					}
				},
				"conditions": [{
					"applies_to": "time",
					"operator": "gt",
					"value": "now"
				},
{
					"applies_to": "time",
					"operator": "lt",
					"value": "now+bucket_span"
				},

]
			},
			...
```

Execution of the detection rule action `force_time_shift` will shift the time inside the anomaly detector by a specified amount. This is useful, e.g. to quickly adapt to the daylight saving time events that are known beforehand. 

This PR is accompanied by changes in Kibana elastic/kibana#188710 and the native ml-cpp code elastic/ml-cpp#2690.

The integration tests between Java and C++ parts of the force-time shift action will be implemented in a separate PR.
Copy link
Member

@jgowdyelastic jgowdyelastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
ml 4.6MB 4.6MB +228.0B
Unknown metric groups

API count

id before after diff
@kbn/ml-anomaly-utils 207 209 +2

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @valeriy42

@valeriy42 valeriy42 merged commit bd843dd into elastic:main Jul 26, 2024
20 checks passed
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Jul 26, 2024
@valeriy42 valeriy42 deleted the force-time-shift branch July 26, 2024 07:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting Feature:Anomaly Detection ML anomaly detection :ml release_note:enhancement v8.16.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants