Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serialize MDC under top level #67

Merged
merged 3 commits into from
Mar 31, 2020
Merged

Conversation

felixbarny
Copy link
Member

Makes it easier to set ECS fields via MDC.
Previously, you'd have to configure topLevelLabels for each ECS field added via MDC.
Now, if you want to have a field nested under labels, you have to explicitly prefix it with labels..

This increases the chance of conflicts, for example when the user adds an @timestamp MDC value. But it also gets out of the way for legitimate use cases.

Having to whitelist all MDC values that should be serialized at the top level via topLevelLabels is cumbersome and feels inconsistent. I think it's overly protective.

Makes it easier to set ECS fields via MDC.
Previously, you'd have to configure topLevelLabels for each ECS field added via MDC.
Now, if you want to have a field nested under labels, you have to explicitly prefix it with `labels.`.
@felixbarny felixbarny requested a review from xeraa March 30, 2020 06:48
@felixbarny
Copy link
Member Author

@pgomulka would this break anything in Elasticsearch?

Copy link

@xeraa xeraa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a quick look and I think this makes a lot of sense (also in documenting how to use this)

@felixbarny felixbarny merged commit d036082 into elastic:master Mar 31, 2020
@felixbarny felixbarny deleted the mdc-top-level branch March 31, 2020 07:23
@pgomulka
Copy link
Contributor

@pgomulka would this break anything in Elasticsearch?

@felixbarny ES is not using MDC, so should be safe.

@lanore78
Copy link

lanore78 commented Jul 7, 2020

With this change, I'm having a problem with collecting log with Filebeat.

It's because some library produce 'url' MDC which is one of ECS key, and the formate is not compatible with ECS(or Elasticsearch)

Here is example value of 'url' MDC, and I got error when I collect this log with Filebeat.
Is there any good idea to solve this problem?

{ ...
"url" : "jdbc:sqlserver://xxx.xxx.xxx.xxx:61051;authenticationScheme=nativeAuthentication;xopenStates=false;sendTimeAsDatetime=true;trustServerCertificate=false;sendStringParametersAsUnicode=false;selectMethod=direct;responseBuffering=adaptive;packetSize=8000;multiSubnetFailover=false;loginTimeout=20;lockTimeout=-1;lastUpdateCount=true;encrypt=false;disableStatementPooling=true;databaseName=USER;applicationName=Microsoft JDBC Driver for SQL Server;applicationIntent=readwrite",
...
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants