Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix InfluxDB UDP line splitting (#5438) #5439

Merged
merged 2 commits into from
Feb 19, 2019

Conversation

oplehto
Copy link
Contributor

@oplehto oplehto commented Feb 15, 2019

Required for all PRs:

  • Signed CLA.
  • Associated README.md updated.
  • Has appropriate unit tests.

Fixes #5438

  • Set MaxLineBytes correctly
  • Split the data to utilize as much of the datagram as possible (even with MaxLineBytes set correctly each metric that did not fit into the first datagram was split up separately)
  • Split the byte slice from serializer to separate UDP datagrams if there are multiple lines

@danielnelson danielnelson self-assigned this Feb 15, 2019
@danielnelson danielnelson added this to the 1.10.0 milestone Feb 15, 2019
scanner.Split(scanLines)
for scanner.Scan() {
_, err = c.conn.Write(scanner.Bytes())
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For 1.10, I think I will experiment with doing a metric splitter before the Serialize step, so we don't need to parse the serialized data. There might be some duplication of the serialize code but I think it will be worth it, and also string fields can contain unescaped newlines in line protocol so there is a potential flaw with this (#4942).

I think we are going to do a 1.9.5, and this will be a more appropriate change for it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I completely agree doing the change you proposed is a sensible approach. This was just a stopgap to get things working with minimal modifications to the existing code.

plugins/serializers/influx/influx_test.go Show resolved Hide resolved
@danielnelson danielnelson modified the milestones: 1.10.0, 1.9.5 Feb 19, 2019
@danielnelson danielnelson merged commit 5dfa3fa into influxdata:master Feb 19, 2019
danielnelson pushed a commit that referenced this pull request Feb 19, 2019
otherpirate pushed a commit to otherpirate/telegraf that referenced this pull request Mar 15, 2019
otherpirate pushed a commit to otherpirate/telegraf that referenced this pull request Mar 15, 2019
dupondje pushed a commit to dupondje/telegraf that referenced this pull request Apr 22, 2019
bitcharmer pushed a commit to bitcharmer/telegraf that referenced this pull request Oct 18, 2019
athoune pushed a commit to bearstech/telegraf that referenced this pull request Apr 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants