Sumo exporter should not send empty log body attributes

[rupeshnemade]

When we removes body from the logs with filelog receiver and tries to forward it with Sumo exporter then Sumo renders the key value as Nil.
It's expected that if the value is Nil then key should not appear at all.

This code logic needs to be updated to not send blank values.

[swiatekm]

Yeah, that looks like a bug. You can work around it by setting the Body to the empty string instead of deleting it.

[rupeshnemade]

@swiatekm-sumo Is it possible that because of Nil we are seeing tons of below errors?
2023-06-21T14:41:29.458Z warn [email protected]/parser.go:198 failed to execute statement {"kind": "processor", "name": "transform/flatten", "pipeline": "logs/otlp/containers", "error": "expected pcommon.Map but got string", "statement": "merge_maps(attributes, body, \"insert\")"}

Shouldn't 'transform/flatten' set the body to empty? or do we need to set it explicitly? If yes, how to do it?
Right now we are removing body as below from filelog processor
filelog/auditlog: - field: body type: remove

[swiatekm]

That sounds like an issue in https://github.com/SumoLogic/sumologic-kubernetes-collection. Can you file it there with the configuration you're using?

[rupeshnemade]

@swiatekm-sumo I still faces the same issue-
I have used public.ecr.aws/sumologic/sumologic-otel-collector:0.80.0-sumo-0-fips image with following rule in 'filelog' receiver-
- field: body type: remove

[swiatekm]

@rupeshnemade can you add a logging processor to your configuration with verbosity: detailed and post the otel logs showing your data?

[rupeshnemade]

@swiatekm-sumo Attaching the log file
otel_nil.log

[swiatekm]

This is actually a bug in filelog receiver. The body in your record is actually a nonempty string with the value "<nil>", and this is what the exporter sends to Sumo. The offending logic is here: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/edad71083ed070e89b46bd569bb9d8d3abdc4ebd/pkg/stanza/adapter/converter.go#L305C13-L305C13, where the receiver should return a pdata Empty value instead of the string.

Can you file this bug upstream?

[rupeshnemade]

I have opened a bug open-telemetry/opentelemetry-collector-contrib#24017