You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Tsdb data streams are designed to accept live metric data. So when a tsdb data stream gets created by default it will accept documents with @timestamp field that have a value from $now-2h to $now+2h. This seems to work well in most cases.
However when metric data is queued up this may not be ideal. This can lead to a large volume of metrics not being indexed because it was older than 2 hours. The 4 hour time window is configurable, but changing that isn't ideal. For example it can lead to very large first backing index, because the older data also gets indexed into the same backing index.
Note that this is only an issue at the time a tsdb data stream is created. After it has been created queued metrics can be indexed without an issue.
This issue is a place holder to keep track of how less recent data prior to the creation of a tsdb data stream can be indexed correctly (and not being dropped).
Note that the issue (#91340) for handling historic metric data is somewhat related to this issue, but it is a different use case, that likely needs a different solution.
The text was updated successfully, but these errors were encountered:
Tsdb data streams are designed to accept live metric data. So when a tsdb data stream gets created by default it will accept documents with
@timestamp
field that have a value from $now-2h to $now+2h. This seems to work well in most cases.However when metric data is queued up this may not be ideal. This can lead to a large volume of metrics not being indexed because it was older than 2 hours. The 4 hour time window is configurable, but changing that isn't ideal. For example it can lead to very large first backing index, because the older data also gets indexed into the same backing index.
Note that this is only an issue at the time a tsdb data stream is created. After it has been created queued metrics can be indexed without an issue.
This issue is a place holder to keep track of how less recent data prior to the creation of a tsdb data stream can be indexed correctly (and not being dropped).
Note that the issue (#91340) for handling historic metric data is somewhat related to this issue, but it is a different use case, that likely needs a different solution.
The text was updated successfully, but these errors were encountered: