Stack trace for "Can not execute the operation on ended Span" log

[neilfordyce]

I'm looking into multiple services producing Can not execute the operation on ended Span logs from this line:

opentelemetry-js/packages/opentelemetry-sdk-trace-base/src/Span.ts

Line 367 in eb3ca4f

`Can not execute the operation on ended Span {traceId: ${this._spanContext.traceId}, spanId: ${this._spanContext.spanId}}`

We have some theories about libraries and bits of instrumentation that might be broken. However, given the information available it's difficult to confidently pinpoint the source. Would it be reasonable to add a stack trace to that log? That way we'll be able to tell what instrumentation is trying to add to the ended span.

• This only affects the JavaScript OpenTelemetry library
• This may affect other libraries, but I would like to get opinions here first

[pichlermarc]

Hi @neilfordyce thanks for reaching out. I think attaching a stack trace to that log is a great idea. 👍

I wonder though: would a verbose log right after that message suffice? One would have to turn down the log-level to verbose manually but it would give the same info.

The reason I'm asking is: A stack trace dumped to the console always gives the impression to users that there's an unhandled error somewhere and I'd like to avoid the flood of bug reports that get mis-categorized as "unhandled" errors when it's actually a properly handled error.

I've written down a proposal of how that would look like at #5113, please let me know what you think 🙂

[neilfordyce]

Thanks for the feedback and raising the proposal 🏅

Happy for you to close this since all the details are captured better the proposal. I like the approach of having an additional log message at a lower priority log level. Makes a lot of sense. Would save us from a large $ logging bill too.

I'd possibly lean towards making the new log debug instead of verbose. In the specific case I'm thinking of, I will probably need to turn on this level of logging for a few containers in production to track it down. Reproducing in isolation locally might be tricky. But from DiagLogger docs, it sounds like turning on verbose would be ill-advised in prod.

A stack trace just referring to lines of code maybe fits debug. Verbose sounds like it's maybe even more verbose than that; possibly even containing any data the application is handling. Sounds like it's a bit of a grey area though and either would work, so happy with which ever you decide.

[pichlermarc]

@neilfordyce thanks for the response - let's do a debug log then 👍
I changed the description and requirements on #5113 - let's track the implementation progress there 🙂