Elasticsearch 9 support #1626
nsano-rururu
started this conversation in
Ideas
Elasticsearch 9 support
#1626
Replies: 1 comment 3 replies
-
|
Elasticsearch 7.17.x reaches end of support nine months after 9.0.0 is released |
Beta Was this translation helpful? Give feedback.
3 replies
-
|
elastalert.yaml es_host: elasticsearch
es_port: 9200
rules_folder: rules
run_every:
seconds: 60
buffer_time:
minutes: 1
writeback_index: praeco_elastalert_status
alert_time_limit:
days: 2
skip_invalid: Truea.yaml alert:
- "slack"
alert_subject: "Test {0}\_\n\n123\n\_b{1}\_aa\n\nfoo\n\nbar\n\n--baz"
alert_text: "Test {0}\_\n\n123\n\_b{1}\_aa\n\nfoo\n\nbar\n\n--baz"
filter:
- query:
query_string:
query: "message:Quit"
import: "BaseRule.config"
index: "mariadb-log-*"
is_enabled: true
match_enhancements: []
name: "a"
num_events: 1
realert:
minutes: 5
slack_webhook_url: 'xxxxxxxxxxxxxxxxxxxx'
slack_channel_override:
- "#abc"
slack_emoji_override: ":kissing_cat:"
slack_msg_color: "warning"
slack_parse_override: "none"
slack_timeout: 10
slack_title_link: "http://localhost:8080/rules/a"
slack_username_override: "elastalert"
terms_size: 50
timeframe:
minutes: 5
timestamp_field: "@timestamp"
timestamp_type: "iso"
type: "frequency"
use_strftime_index: falseelastalert-server:20250220 (include elastalert 2.23.0) 14:42:48.860Z INFO elastalert-server: Config: No config.dev.json file was found in /opt/elastalert-server/config/config.dev.json.
14:42:48.863Z INFO elastalert-server: Config: Proceeding to look for normal config file.
14:42:48.867Z INFO elastalert-server: Config: A config file was found in /opt/elastalert-server/config/config.json. Using that config.
14:42:48.903Z INFO elastalert-server: Router: Listening for GET request on /.
14:42:48.903Z INFO elastalert-server: Router: Listening for GET request on /status.
14:42:48.904Z INFO elastalert-server: Router: Listening for GET request on /status/errors.
14:42:48.904Z INFO elastalert-server: Router: Listening for GET request on /rules.
14:42:48.917Z INFO elastalert-server: Router: Listening for GET request on /rules/:id*.
14:42:48.921Z INFO elastalert-server: Router: Listening for POST request on /rules/:id*.
14:42:48.922Z INFO elastalert-server: Router: Listening for DELETE request on /rules/:id*.
14:42:48.922Z INFO elastalert-server: Router: Listening for GET request on /templates.
14:42:48.935Z INFO elastalert-server: Router: Listening for GET request on /templates/:id*.
14:42:48.936Z INFO elastalert-server: Router: Listening for POST request on /templates/:id*.
14:42:48.941Z INFO elastalert-server: Router: Listening for DELETE request on /templates/:id*.
14:42:48.941Z INFO elastalert-server: Router: Listening for PUT request on /folders/:type/:path*.
14:42:48.942Z INFO elastalert-server: Router: Listening for DELETE request on /folders/:type/:path*.
14:42:48.942Z INFO elastalert-server: Router: Listening for POST request on /test.
14:42:48.942Z INFO elastalert-server: Router: Listening for POST request on /silence/:path*.
14:42:48.942Z INFO elastalert-server: Router: Listening for GET request on /config.
14:42:48.945Z INFO elastalert-server: Router: Listening for POST request on /config.
14:42:48.945Z INFO elastalert-server: Router: Listening for GET request on /metadata/elastalert.
14:42:48.945Z INFO elastalert-server: Router: Listening for GET request on /metadata/elastalert_status.
14:42:48.945Z INFO elastalert-server: Router: Listening for GET request on /metadata/silence.
14:42:48.945Z INFO elastalert-server: Router: Listening for GET request on /metadata/elastalert_error.
14:42:48.945Z INFO elastalert-server: Router: Listening for GET request on /metadata/past_elastalert.
14:42:48.945Z INFO elastalert-server: Router: Listening for GET request on /indices.
14:42:48.946Z INFO elastalert-server: Router: Listening for GET request on /mapping/:index.
14:42:48.946Z INFO elastalert-server: Router: Listening for POST request on /search/:index.
14:42:48.946Z INFO elastalert-server: Router: Listening for GET request on /config.
14:42:48.971Z INFO elastalert-server: ProcessController: Starting ElastAlert
14:42:48.971Z INFO elastalert-server: ProcessController: Creating index
14:42:55.364Z INFO elastalert-server:
ProcessController: Reading Elastic 8 index mappings:
Reading index mapping 'es_mappings/8/silence.json'
Reading index mapping 'es_mappings/8/elastalert_status.json'
Reading index mapping 'es_mappings/8/elastalert.json'
Reading index mapping 'es_mappings/8/past_elastalert.json'
Reading index mapping 'es_mappings/8/elastalert_error.json'
New index praeco_elastalert_status created
Done!
14:42:55.364Z INFO elastalert-server: ProcessController: Index create exited with code 0
14:42:55.365Z INFO elastalert-server: ProcessController: Starting elastalert with arguments [none]
14:42:55.373Z INFO elastalert-server: ProcessController: Started Elastalert (PID: 415)
14:42:55.395Z INFO elastalert-server: Server: Server listening on port 3030
14:42:55.405Z INFO elastalert-server: Server: Websocket listening on port 3333
14:42:55.406Z INFO elastalert-server: Server: Server started
14:43:02.841Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:43:33.146Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:43:42.046Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:43:42.061Z INFO elastalert-server: Routes: Successfully handled GET request for '/rules/:id'.
14:43:42.154Z INFO elastalert-server: Routes: Successfully handled GET request for '/config'.
14:43:42.158Z INFO elastalert-server: Routes: Successfully handled GET request for '/config'.
14:44:01.288Z INFO elastalert-server: Routes: Successfully handled GET request for '/rules/:id'.
14:44:01.304Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:44:01.366Z INFO elastalert-server: Routes: Successfully handled GET request for '/config'.
14:44:01.369Z INFO elastalert-server: Routes: Successfully handled GET request for '/config'.
14:44:03.323Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:44:33.449Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:45:03.672Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:45:33.815Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:46:03.932Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:46:34.021Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:47:04.220Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:47:31.963Z INFO elastalert-server: Routes: Successfully handled GET request for '/rules'.
14:47:34.473Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:47:42.282Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:47:42.306Z INFO elastalert-server: Routes: Successfully handled GET request for '/config'.
14:47:42.309Z INFO elastalert-server: Routes: Successfully handled GET request for '/config'.
14:48:04.658Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:48:14.804Z INFO elastalert-server: Routes: Successfully handled GET request for '/rules'.
14:48:17.069Z INFO elastalert-server: Routes: Successfully handled GET request for '/rules/:id'.
14:48:34.802Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:49:04.908Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
14:49:35.028Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
elastalert-server:20241227 (include elastalert 2.22.0) 12:26:48.683Z INFO elastalert-server: Config: A config file was found in /opt/elastalert-server/config/config.json. Using that config.
12:26:48.709Z INFO elastalert-server: Router: Listening for GET request on /.
12:26:48.709Z INFO elastalert-server: Router: Listening for GET request on /status.
12:26:48.710Z INFO elastalert-server: Router: Listening for GET request on /status/errors.
12:26:48.710Z INFO elastalert-server: Router: Listening for GET request on /rules.
12:26:48.714Z INFO elastalert-server: Router: Listening for GET request on /rules/:id*.
12:26:48.716Z INFO elastalert-server: Router: Listening for POST request on /rules/:id*.
12:26:48.717Z INFO elastalert-server: Router: Listening for DELETE request on /rules/:id*.
12:26:48.717Z INFO elastalert-server: Router: Listening for GET request on /templates.
12:26:48.718Z INFO elastalert-server: Router: Listening for GET request on /templates/:id*.
12:26:48.720Z INFO elastalert-server: Router: Listening for POST request on /templates/:id*.
12:26:48.722Z INFO elastalert-server: Router: Listening for DELETE request on /templates/:id*.
12:26:48.722Z INFO elastalert-server: Router: Listening for PUT request on /folders/:type/:path*.
12:26:48.722Z INFO elastalert-server: Router: Listening for DELETE request on /folders/:type/:path*.
12:26:48.723Z INFO elastalert-server: Router: Listening for POST request on /test.
12:26:48.723Z INFO elastalert-server: Router: Listening for POST request on /silence/:path*.
12:26:48.723Z INFO elastalert-server: Router: Listening for GET request on /config.
12:26:48.726Z INFO elastalert-server: Router: Listening for POST request on /config.
12:26:48.727Z INFO elastalert-server: Router: Listening for GET request on /metadata/elastalert.
12:26:48.727Z INFO elastalert-server: Router: Listening for GET request on /metadata/elastalert_status.
12:26:48.727Z INFO elastalert-server: Router: Listening for GET request on /metadata/silence.
12:26:48.727Z INFO elastalert-server: Router: Listening for GET request on /metadata/elastalert_error.
12:26:48.727Z INFO elastalert-server: Router: Listening for GET request on /metadata/past_elastalert.
12:26:48.727Z INFO elastalert-server: Router: Listening for GET request on /indices.
12:26:48.727Z INFO elastalert-server: Router: Listening for GET request on /mapping/:index.
12:26:48.727Z INFO elastalert-server: Router: Listening for POST request on /search/:index.
12:26:48.727Z INFO elastalert-server: Router: Listening for GET request on /config.
12:26:48.738Z INFO elastalert-server: ProcessController: Starting ElastAlert
12:26:48.741Z INFO elastalert-server: ProcessController: Creating index
12:26:55.078Z INFO elastalert-server:
ProcessController: Reading Elastic 8 index mappings:
Reading index mapping 'es_mappings/8/silence.json'
Reading index mapping 'es_mappings/8/elastalert_status.json'
Reading index mapping 'es_mappings/8/elastalert.json'
Reading index mapping 'es_mappings/8/past_elastalert.json'
Reading index mapping 'es_mappings/8/elastalert_error.json'
New index praeco_elastalert_status created
Done!
12:26:55.078Z INFO elastalert-server: ProcessController: Index create exited with code 0
12:26:55.079Z INFO elastalert-server: ProcessController: Starting elastalert with arguments [none]
12:26:55.086Z INFO elastalert-server: ProcessController: Started Elastalert (PID: 400)
12:26:55.094Z INFO elastalert-server: Server: Server listening on port 3030
12:26:55.096Z INFO elastalert-server: Server: Websocket listening on port 3333
12:26:55.097Z INFO elastalert-server: Server: Server started
12:27:08.126Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
12:27:38.381Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
12:27:57.336Z ERROR elastalert-server:
ProcessController: WARNING:elasticsearch:POST http://elasticsearch:9200/praeco_elastalert_status/_search?size=1000 [status:400 request:0.014s]
12:27:57.351Z ERROR elastalert-server:
ProcessController: ERROR:elastalert:Error finding recent pending alerts: RequestError(400, 'x_content_parse_exception', '[range] query does not support [from]') {'query': {'bool': {'must': {'query_string': {'query': '!_exists_:aggregate_id AND alert_sent:false'}}, 'filter': {'range': {'alert_time': {'from': '2025-02-18T12:27:57.317843Z', 'to': '2025-02-20T12:27:57.317883Z'}}}}}, 'sort': {'alert_time': {'order': 'asc'}}}
Traceback (most recent call last):
File "/opt/elastalert/elastalert/elastalert.py", line 1529, in find_recent_pending_alerts
res = self.writeback_es.search(index=self.writeback_index, body=query, size=1000)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/client/utils.py", line 152, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/elastalert/elastalert/__init__.py", line 147, in search
results = self.transport.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/transport.py", line 392, in perform_request
raise e
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/transport.py", line 358, in perform_request
status, headers_response, data = connection.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/connection/http_requests.py", line 199, in perform_request
self._raise_error(response.status_code, raw_data)
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/connection/base.py", line 315, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
elasticsearch.exceptions.RequestError: RequestError(400, 'x_content_parse_exception', '[range] query does not support [from]')
12:28:08.561Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
12:28:38.718Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
12:28:48.294Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
12:28:48.459Z INFO elastalert-server: Routes: Successfully handled GET request for '/config'.
12:28:48.471Z INFO elastalert-server: Routes: Successfully handled GET request for '/config'.
12:28:57.323Z ERROR elastalert-server:
ProcessController: WARNING:elasticsearch:POST http://elasticsearch:9200/praeco_elastalert_status/_search?size=1000 [status:400 request:0.006s]
12:28:57.327Z ERROR elastalert-server:
ProcessController: ERROR:elastalert:Error finding recent pending alerts: RequestError(400, 'x_content_parse_exception', '[range] query does not support [from]') {'query': {'bool': {'must': {'query_string': {'query': '!_exists_:aggregate_id AND alert_sent:false'}}, 'filter': {'range': {'alert_time': {'from': '2025-02-18T12:28:57.316194Z', 'to': '2025-02-20T12:28:57.316226Z'}}}}}, 'sort': {'alert_time': {'order': 'asc'}}}
Traceback (most recent call last):
File "/opt/elastalert/elastalert/elastalert.py", line 1529, in find_recent_pending_alerts
res = self.writeback_es.search(index=self.writeback_index, body=query, size=1000)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/client/utils.py", line 152, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/elastalert/elastalert/__init__.py", line 147, in search
results = self.transport.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/transport.py", line 392, in perform_request
raise e
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/transport.py", line 358, in perform_request
status, headers_response, data = connection.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/connection/http_requests.py", line 199, in perform_request
self._raise_error(response.status_code, raw_data)
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/connection/base.py", line 315, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
elasticsearch.exceptions.RequestError: RequestError(400, 'x_content_parse_exception', '[range] query does not support [from]')
12:29:08.889Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
12:29:21.263Z INFO elastalert-server: Routes: Successfully handled GET request for '/rules'.
12:29:22.391Z INFO elastalert-server: Routes: Successfully handled GET request for '/rules/:id'.
12:29:39.003Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
12:29:57.320Z ERROR elastalert-server:
ProcessController: WARNING:elasticsearch:POST http://elasticsearch:9200/praeco_elastalert_status/_search?size=1000 [status:400 request:0.004s]
12:29:57.321Z ERROR elastalert-server:
ProcessController: ERROR:elastalert:Error finding recent pending alerts: RequestError(400, 'x_content_parse_exception', '[range] query does not support [from]') {'query': {'bool': {'must': {'query_string': {'query': '!_exists_:aggregate_id AND alert_sent:false'}}, 'filter': {'range': {'alert_time': {'from': '2025-02-18T12:29:57.316052Z', 'to': '2025-02-20T12:29:57.316075Z'}}}}}, 'sort': {'alert_time': {'order': 'asc'}}}
Traceback (most recent call last):
File "/opt/elastalert/elastalert/elastalert.py", line 1529, in find_recent_pending_alerts
res = self.writeback_es.search(index=self.writeback_index, body=query, size=1000)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/client/utils.py", line 152, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/elastalert/elastalert/__init__.py", line 147, in search
results = self.transport.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/transport.py", line 392, in perform_request
raise e
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/transport.py", line 358, in perform_request
status, headers_response, data = connection.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/connection/http_requests.py", line 199, in perform_request
self._raise_error(response.status_code, raw_data)
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/connection/base.py", line 315, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
elasticsearch.exceptions.RequestError: RequestError(400, 'x_content_parse_exception', '[range] query does not support [from]')
12:30:09.172Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
12:30:39.308Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
12:30:57.327Z ERROR elastalert-server:
ProcessController: WARNING:elasticsearch:POST http://elasticsearch:9200/praeco_elastalert_status/_search?size=1000 [status:400 request:0.009s]
12:30:57.329Z ERROR elastalert-server:
ProcessController: ERROR:elastalert:Error finding recent pending alerts: RequestError(400, 'x_content_parse_exception', '[range] query does not support [from]') {'query': {'bool': {'must': {'query_string': {'query': '!_exists_:aggregate_id AND alert_sent:false'}}, 'filter': {'range': {'alert_time': {'from': '2025-02-18T12:30:57.316220Z', 'to': '2025-02-20T12:30:57.316247Z'}}}}}, 'sort': {'alert_time': {'order': 'asc'}}}
Traceback (most recent call last):
File "/opt/elastalert/elastalert/elastalert.py", line 1529, in find_recent_pending_alerts
res = self.writeback_es.search(index=self.writeback_index, body=query, size=1000)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/client/utils.py", line 152, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/elastalert/elastalert/__init__.py", line 147, in search
results = self.transport.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/transport.py", line 392, in perform_request
raise e
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/transport.py", line 358, in perform_request
status, headers_response, data = connection.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/connection/http_requests.py", line 199, in perform_request
self._raise_error(response.status_code, raw_data)
File "/home/node/.local/lib/python3.12/site-packages/elasticsearch/connection/base.py", line 315, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
elasticsearch.exceptions.RequestError: RequestError(400, 'x_content_parse_exception', '[range] query does not support [from]')
12:31:09.432Z INFO elastalert-server: Routes: Successfully handled GET request for '/'.
12:31:39.556Z INFO elastalert-server: Routes: Successfully handled GET request for '/'. |
Beta Was this translation helpful? Give feedback.
-
|
elasticsearch9.0.0 beta1 There seems to be no problem with elastalert2 2.23.0 The operation of kibana Discover etc. has not yet been confirmed |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for performing the initial check. |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
https://github.com/elastic/elasticsearch/releases/tag/v9.0.0-beta1
https://hub.docker.com/r/elastic/elasticsearch/tags
Once the official version of elasticsearch 9 is released, it will be necessary to check its operation.
Beta Was this translation helpful? Give feedback.
All reactions