You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am trying to send log messages through telegraf via UDP in the influx line format. It appears the telegraf influx data parser does not accept newlines inside field values - not even inside quoted field values. I believe this should be allowed. In fact, using the same steps to reproduce the issues directly with an influxdb udp listener setup works as expected - newlines are stored in the database.
# Sending to telegraf. Note the -e
$ echo -e 'test_logs message="Hey!\n"' | nc -u localhost 9089
# telegraf log:
2018-10-31T12:10:10Z E! Error in plugin [inputs.socket_listener]: unable to parse incoming packet: metric parse error: expected field at offset 23: "test_logs message=\"Hey!\n\"\n"
Sending to influxdb directly works as expected:
$ echo -e 'test_logs message="Hey!\nhow are you?"' | nc -u localhost 8089
> select * from test_logs;
name: test_logs
time message
---- -------
1540988837240091217 Hey!
how are you?
Expected behavior:
The influx parser should accept newlines inside quoted field values.
Actual behavior:
The parser rejects newlines inside quoted field values.
Additional info:
From what I could gather from the code: Telgraf simply scans for a newline character in the received data to determine where a influx line starts and terminates. The influxdb code seems more advanced in that it handles quoted fields: https://github.com/influxdata/influxdb/blob/master/models/points.go#L1081
Background:
I'm attempting to send log messages (and particularly tracebacks with newlines) using a Python logging handler [1]. Previously I had only tested against influxdb directly, but in the meantime have a use case to send logs through telegraf which should then get forwarded to influxdb.
2018-10-31T19:31:01Z E! Error in plugin [inputs.socket_listener]: unable to parse incoming packet: metric parse error: expected field at offset 24: "test_logs message=\"Hey!\\nhow are you?\"\n"
Further note, sending \n to an influxdb listener causes \n to show up in the field value - there is no CR. Seems influxdb does not support the \n escape inside field values.
Hey hey,
I am trying to send log messages through telegraf via UDP in the influx line format. It appears the telegraf influx data parser does not accept newlines inside field values - not even inside quoted field values. I believe this should be allowed. In fact, using the same steps to reproduce the issues directly with an influxdb udp listener setup works as expected - newlines are stored in the database.
Relevant telegraf.conf:
influxdb.conf for comparison
System info:
Telegraf: master (d2717f7)
Debian buster/sid
Steps to reproduce:
Sending to influxdb directly works as expected:
Expected behavior:
The influx parser should accept newlines inside quoted field values.
Actual behavior:
The parser rejects newlines inside quoted field values.
Additional info:
From what I could gather from the code: Telgraf simply scans for a newline character in the received data to determine where a influx line starts and terminates. The influxdb code seems more advanced in that it handles quoted fields: https://github.com/influxdata/influxdb/blob/master/models/points.go#L1081
Background:
I'm attempting to send log messages (and particularly tracebacks with newlines) using a Python logging handler [1]. Previously I had only tested against influxdb directly, but in the meantime have a use case to send logs through telegraf which should then get forwarded to influxdb.
[1] https://github.com/awelzel/influxpy
The text was updated successfully, but these errors were encountered: