Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect timezone reported with logstash and elasticsearch Filebeat modules #9756

Closed
ycombinator opened this issue Dec 21, 2018 · 4 comments

Comments

@ycombinator
Copy link
Contributor

Timestamps in neither Elasticsearch nor Logstash logs contain timezone information. Currently the elasticsearch and logstash Filebeat modules simply index these timestamps as-is (without any timezone information), causing Kibana to interpret them as being in UTC.

Like the system Filebeat module, the elasticsearch and logstash Filebeat modules should support the var.convert_timezone configuration setting.

@elasticmachine
Copy link
Collaborator

Pinging @elastic/stack-monitoring

@ruflin
Copy link
Contributor

ruflin commented Dec 27, 2018

@ycombinator I think we should investigate why LS and ES don't log the timezone in the first place too.

@ycombinator
Copy link
Contributor Author

@ruflin said:

I think we should investigate why LS and ES don't log the timezone in the first place too.

Elasticsearch is getting structured logging in the future. This would be a good opportunity to include the time zone in their log timestamps. I started a conversation about this on the structured logging PR here: https://github.com/elastic/elasticsearch/pull/36833/files#r244225243

For Logstash, I created this issue: elastic/logstash#10238

ycombinator added a commit that referenced this issue Dec 28, 2018
This PR updates the following filesets in the `elasticsearch` Filebeat module to accept a `var.convert_timezone` configuration setting:

* [x] server
* [x] audit
* [x] slowlog
* [x] deprecation

The `gc` fileset was not modified because Elasticsearch GC logs already emit timestamps with timezone information in them.

Fixes partially #9756. Related: #9797.
@ycombinator
Copy link
Contributor Author

ycombinator commented Jan 8, 2019

Resolved via #9797 and #9761.

ycombinator added a commit that referenced this issue Jan 8, 2019
…earch module filesets (#9830)

Cherry-pick of PR #9761 to 6.x branch. Original message: 

This PR updates the following filesets in the `elasticsearch` Filebeat module to accept a `var.convert_timezone` configuration setting:

* [x] server
* [x] audit
* [x] slowlog
* [x] deprecation

The `gc` fileset was not modified because Elasticsearch GC logs already emit timestamps with timezone information in them.

Fixes partially #9756. Related: #9797.
ycombinator added a commit that referenced this issue Jan 30, 2019
… module filesets (#10407)

Cherry-pick of PR #9797 to 6.6 branch. Original message: 

This PR updates the following filesets in the `logstash` Filebeat module to accept a `var.convert_timezone` configuration setting:

* [x] log
* [x] slowlog

Fixes partially #9756. Related: #9761
ycombinator added a commit that referenced this issue Jan 30, 2019
…earch module filesets (#10403)

* Allow users to convert timezone in elasticsearch module filesets (#9761)

This PR updates the following filesets in the `elasticsearch` Filebeat module to accept a `var.convert_timezone` configuration setting:

* [x] server
* [x] audit
* [x] slowlog
* [x] deprecation

The `gc` fileset was not modified because Elasticsearch GC logs already emit timestamps with timezone information in them.

Fixes partially #9756. Related: #9797.

(cherry picked from commit 3559e58)

* Regenerating generated files

* Fixing duplicated header in CHANGELOG

* Updating generated file
leweafan pushed a commit to leweafan/beats that referenced this issue Apr 28, 2023
…ogstash module filesets (elastic#10407)

Cherry-pick of PR elastic#9797 to 6.6 branch. Original message: 

This PR updates the following filesets in the `logstash` Filebeat module to accept a `var.convert_timezone` configuration setting:

* [x] log
* [x] slowlog

Fixes partially elastic#9756. Related: elastic#9761
leweafan pushed a commit to leweafan/beats that referenced this issue Apr 28, 2023
…lasticsearch module filesets (elastic#10403)

* Allow users to convert timezone in elasticsearch module filesets (elastic#9761)

This PR updates the following filesets in the `elasticsearch` Filebeat module to accept a `var.convert_timezone` configuration setting:

* [x] server
* [x] audit
* [x] slowlog
* [x] deprecation

The `gc` fileset was not modified because Elasticsearch GC logs already emit timestamps with timezone information in them.

Fixes partially elastic#9756. Related: elastic#9797.

(cherry picked from commit 43d976e)

* Regenerating generated files

* Fixing duplicated header in CHANGELOG

* Updating generated file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants