Ensure LongTaskTimer and FunctionTimer produce consistent data

[pirgeo]

In the past, there were problems with the LongTaskTimer: In cases where only one value was exported, the max and total duration were read sequentially while the clock continued to tick in the background. Since the Dynatrace API checks this data strictly, data, where the sum and max were not exactly the same when the count was 1, were rejected. The discrepancy is caused by the non-atomicity of retrieving max and sum. As soon as there is more than one observation, this problem disappears since the Dynatrace API checks for min <= mean <= max, and that should always be the case when there is more than 1 value. If it is not the case, the underlying data collection is really broken and the data should be rejected.

We saw this issue with the metric http.server.requests.active when there was exactly one request in-flight. The retrieval of max and total are not synchronized, so the clock continues to tick and results in two different values (e.g., max=0.764418,sum=0.700539,count=1, which is invalid according to the Dynatrace specification).

Additionally, Since there is no way to retrieve (or even set) data for the FunctionTimer atomically, a similar problem might arise there. I added some guarding code to ensure only valid data should be exported.

I consider this a bugfix, since it leads to problems in various deployments. I think by targeting the 1.9.x branch, this change should be merged forward to all newer branches, right?

[sonatype-lift]

Sonatype Lift is retiring

Sonatype Lift will be retiring on Sep 12, 2023, with its analysis stopping on Aug 12, 2023. We understand that this news may come as a disappointment, and Sonatype is committed to helping you transition off it seamlessly. If you’d like to retain your data, please export your issues from the web console.
We are extremely grateful and thank you for your support over the years.

📖 Read about the impacts and timeline

[pirgeo]

@jonatan-ivanov Should I open an issue for this one or is it enough to keep the PR here?

[jonatan-ivanov]

If there is an open PR, there is usually no need for a separate issue, GitHub (and we too) handle them quite similarly.
(E.g.: if both an issue and a PR is available we usually remove the labels and the milestone from the PR and keep it only on the issue, this will also mean that in this case only the issue will be in the release notes.)

[pirgeo]

Rebased on the latest 1.9.x branch

[jonatan-ivanov]

Thank you for the PR!
I polished it a little, but was not able to update the PR (since it is not your fork but the dynatrace-oss-corntrib org's) so I merged it in directly to 1.9.x. Please check my polishing changes and let us know if something needs to be modified. Otherwise this should go out on Monday.

[pirgeo]

Ah, I must have forgotten to turn allow maintainers to modify on - it's a bit tricky with organization-owned forks. Thank you so much for looking at it and polishing it as well - the changes there look great and are much less likely to get flaky in the future.