reject metric event with invalid timestamp for sls serializer && fix serialization error with empty event #2066
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
henryzhx8
commented
Feb 7, 2025
•
henryzhx8
commented
- 对于metric event，强制要求时间戳为10位，否则直接丢弃该事件；
- event group里log event和raw event部分为空时，会出现序列化异常的情况，对该问题进行了修复；
- 为了便于排查问题，增加一段临时的debug代码，等序列化功能稳定后会删除。
henryzhx8
changed the title
yyuuttaaoo
reviewed
Feb 8, 2025
| @@ -116,13 +119,19 @@ bool SLSEventGroupSerializer::Serialize(BatchedEvents&& group, string& res, stri | |||
| case PipelineEvent::Type::METRIC: { | |||
| for (size_t i = 0; i < group.mEvents.size(); ++i) { | |||
| const auto& e = group.mEvents[i].Cast<MetricEvent>(); | |||
| if (e.GetTimestamp() < 1e9) { | |||
| LOG_WARNING(sLogger, | |||
There was a problem hiding this comment.
这种丢数据的，不能只有本地日志吧，得上报alarm
There was a problem hiding this comment.
数据面要梳理下哪些需要上报alarm，得自己在发布过程中提前发现问题
There was a problem hiding this comment.
alarm需要先把错误码的规范明确一下，这个涉及报表制作，现有的错误码太随意了，动不动就随便加一个，所以暂时性没加alarm，只有warning，但是这个事情也需要尽快启动。
yyuuttaaoo
approved these changes
Feb 8, 2025
messixukejia
approved these changes
Feb 8, 2025
| @@ -116,13 +119,19 @@ bool SLSEventGroupSerializer::Serialize(BatchedEvents&& group, string& res, stri | |||
| case PipelineEvent::Type::METRIC: { | |||
| for (size_t i = 0; i < group.mEvents.size(); ++i) { | |||
| const auto& e = group.mEvents[i].Cast<MetricEvent>(); | |||
| if (e.GetTimestamp() < 1e9) { | |||
| LOG_WARNING(sLogger, | |||
There was a problem hiding this comment.
数据面要梳理下哪些需要上报alarm，得自己在发布过程中提前发现问题
| if (!logGroup.ParseFromString(res)) { | ||
| JsonEventGroupSerializer ser(const_cast<Flusher*>(mFlusher)); | ||
| string jsonStr; | ||
| ser.DoSerialize(std::move(group), jsonStr, errorMsg); |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.