Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updating OT Exporter to align with latest mappings spec #17392

Merged
merged 20 commits into from
Sep 9, 2021
Merged

Updating OT Exporter to align with latest mappings spec #17392

merged 20 commits into from
Sep 9, 2021

Conversation

hectorhdzg
Copy link
Member

@ghost ghost added the Monitor Monitor, Monitor Ingestion, Monitor Query label Aug 31, 2021
@check-enforcer
Copy link

check-enforcer bot commented Sep 1, 2021

This pull request is protected by Check Enforcer.

What is Check Enforcer?

Check Enforcer helps ensure all pull requests are covered by at least one check-run (typically an Azure Pipeline). When all check-runs associated with this pull request pass then Check Enforcer itself will pass.

Why am I getting this message?

You are getting this message because Check Enforcer did not detect any check-runs being associated with this pull request within five minutes. This may indicate that your pull request is not covered by any pipelines and so Check Enforcer is correctly blocking the pull request being merged.

What should I do now?

If the check-enforcer check-run is not passing and all other check-runs associated with this PR are passing (excluding license-cla) then you could try telling Check Enforcer to evaluate your pull request again. You can do this by adding a comment to this pull request as follows:
/check-enforcer evaluate
Typically evaulation only takes a few seconds. If you know that your pull request is not covered by a pipeline and this is expected you can override Check Enforcer using the following command:
/check-enforcer override
Note that using the override command triggers alerts so that follow-up investigations can occur (PRs still need to be approved as normal).

What if I am onboarding a new service?

Often, new services do not have validation pipelines associated with them, in order to bootstrap pipelines for a new service, you can issue the following command as a pull request comment:
/azp run prepare-pipelines
This will run a pipeline that analyzes the source tree and creates the pipelines necessary to build and validate your pull request. Once the pipeline has been created you can trigger the pipeline using the following comment:
/azp run js - [service] - ci

lzchen
lzchen reviewed Sep 2, 2021
View reviewed changes
sdk/monitor/monitor-opentelemetry-exporter/src/utils/spanUtils.ts
key.startsWith("peer.") ||
key.startsWith("net.")
)
) {
properties[key] = span.attributes[key] as string;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at max lengths, key and value should be truncated.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not adding truncation in this PR, will address that later, created a task for it

lzchen
lzchen reviewed Sep 2, 2021
View reviewed changes
sdk/monitor/monitor-opentelemetry-exporter/src/utils/spanUtils.ts
key.startsWith("peer.") ||
key.startsWith("net.")
)
) {
properties[key] = span.attributes[key] as string;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Truncation rules for custom properties need to be applied. _MS.links needs to be a valid JSON after truncation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you need to parse the JSON and remove links, stringify it again and keep trying until size is allowed, will address in different PR

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since links are a fixed length, we can do some math and allow only up to a certain amount of links :D

lzchen
lzchen reviewed Sep 2, 2021
View reviewed changes
sdk/monitor/monitor-opentelemetry-exporter/src/utils/spanUtils.ts Outdated
if (span.kind === SpanKind.PRODUCER) {
baseData.type = DependencyTypes.QueueMessage;
}
if (span.kind === SpanKind.INTERNAL && span.parentSpanId) {
Copy link
Member

@lzchen lzchen Sep 2, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious as to why span.parentSpanId needs to be populated as well for this to be INPROC?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is mentioned in the spec here https://github.com/microsoft/common-schema/blob/main/Mappings/AzureMonitor-AI.md#remotedependencytype

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm what about INTERNAL spans without parents? Those are quite common and type won't be populated? I will bring this up in specs.

Copy link
Member

@lzchen lzchen Sep 3, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: This topic is still up for discussion. Leaving as it is for now.

lzchen
lzchen reviewed Sep 2, 2021
View reviewed changes
sdk/monitor/monitor-opentelemetry-exporter/src/utils/spanUtils.ts Outdated
if (target) {
let url = new URL(target);
// Ignore port if it is a default port
if (url.port === "80" || url.port === "443") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't url.port be Null if target was populated by httpUrl , netPeerName or netPeerIp?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is supposed to, users can add whatever in these fields so new Regex logic will take care of wrong values as well

lzchen
lzchen reviewed Sep 2, 2021
View reviewed changes
sdk/monitor/monitor-opentelemetry-exporter/src/utils/spanUtils.ts
remoteDependencyData.resultCode = String(grpcStatusCode);
}
let target = getTarget(span);
if (target) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this logic for target specified for rpc yet?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case of rpc:

Derive target from peerService or net* attributes as above.
Default: If none of the above are available, assign rpcSystem to target.

https://github.com/microsoft/common-schema/blob/main/Mappings/AzureMonitor-AI.md#remotedependencytarget

lzchen
lzchen reviewed Sep 2, 2021
View reviewed changes
sdk/monitor/monitor-opentelemetry-exporter/src/utils/spanUtils.ts Outdated
if (span.kind === SpanKind.PRODUCER) {
baseData.type = DependencyTypes.QueueMessage;
}
if (span.kind === SpanKind.INTERNAL && span.parentSpanId) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I think data.success=True always for Internal

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if explicitly set to error?, that sounds weird

@ramya-rao-a
Copy link
Contributor

@hectorhdzg I have pushed a commit to your branch to resolve a merge conflict caused here by a PR of mine that I had merged. Please pull in the latest in case you plan to add more changes

@hectorhdzg hectorhdzg mentioned this pull request Sep 3, 2021
Merged
@hectorhdzg hectorhdzg merged commit b49ffb1 into Azure:main Sep 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ramya-rao-a ramya-rao-a ramya-rao-a left review comments

@lzchen lzchen lzchen approved these changes

@MSNev MSNev MSNev approved these changes

@richardpark-msft richardpark-msft Awaiting requested review from richardpark-msft

@maorleger maorleger Awaiting requested review from maorleger

@deyaaeldeen deyaaeldeen Awaiting requested review from deyaaeldeen

@mikeharder mikeharder Awaiting requested review from mikeharder

@praveenkuttappan praveenkuttappan Awaiting requested review from praveenkuttappan

@witemple-msft witemple-msft Awaiting requested review from witemple-msft

Assignees
No one assigned
Labels
Monitor Monitor, Monitor Ingestion, Monitor Query
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@hectorhdzg @ramya-rao-a @lzchen @MSNev @southpolesteve @omziv