Allow to opt out certain domains from setting the correlation header

[SergeyKanzhelev]

Correlation headers should not be set for the list of domains that customer can control.

[SergeyKanzhelev]

Also need a way to disable both - setting the headers on response and setting them on outgoing http connections.

[SergeyKanzhelev]

Investigate this: microsoft/ApplicationInsights-node.js#143 for .NET

[nizarq]

The 2.3 beta 1 sdk release includes cross component correlation. The feature works as follows:

Dependency Module

• Adds IKey in source header for outgoing requests.
• Intercepts target IKey in incoming responses and logs it along with dependency telemetry.

Web Module

• Adds IKey in target header of outgoing responses.
• Intercepts source IKey in incoming requests and logs it along with request telemetry.

Going with the current philosophy of application insights config. Here is a proposed sample.

Note above that each module (ExceptionTracking, DependencyTracking and ExceptionTracking) has it's own enable / disable setting, where as, the DependencyTracking module has the domain exclusion list, which can be used to override header behavior on specific domains (read hosts, see limitation below).

Alternative approach
Although the above conforms to the config philosophy. I feel it is very user unfriendly, especially, given that we will auto magically enable cross component correlation. If they then want to disable it, they will need to update the config in two modules - this assumes a deep understanding of the different modules play their part in the feature. My premise is that, that is asking a lot of a user. On the other hand we can justify the above approach by saying the default settings would work for almost everyone and it is fine to place the extra burden on the developer if they want to tinker it.
Here's an alternative settings that makes it super dumb for the customer. This config is feature first (as opposed to module / implementation first).

The above implies - when CollectionEnabled is false, it will disable the feature across both modules (i.e., it will stop sending and processing headers). As for the excluded domains list, that would apply to outgoing requests being made (the user doesn't have to be explicitly aware).

Limitations of planned implementation
Current implementation aims to start simplistic. That is, it will take each listed domain, and only do a host level comparison. That is if you add bbc.com or www.bbc.com or http://www.bbc.com or www.bbc.com/news in the exclude domain list, it will stop adding headers at the host level, i.e., any request going to www.bbc.com (not matter how deep or shallow the url) will not have the additional correlation headers. We can refine this in future to allow wildcards and other complex sub-path specific logic.

[nizarq]

should we also have a exclusion list on web / exceptions module that avoids sending response headers?

[SergeyKanzhelev]

I like the first config more. Why do you need correlation on Exception module? Exception module doesn't generate requests so nothing to correlate.

And you do not need to have exclusion list on web module.

BTW, you can insert listings into GitHub as text like this:

<xml>
  <text>This text is searchable now<text>
</xml>

[nizarq]

you are right - no need for exceptions. For a moment I mixed exceptions and failed requests there.

[qubitron]

Is it implied that "bbc.com" -> "*.bbc.com", or do we need explicit wildcards?

With the first proposal if we want to exclude these domains from incoming and outgoing requests, there should be a duplicated section but with the web module name, correct?

[nizarq]

Not implied and wild cards won't work for now. So if you wanted www.bbc.com and www.sports.bbc.com to both be blocked, you will need to list both.
I see this as something we will eventually have to fix (by allowing wild cards), but I'm doing the minimum required easier thing here. We can have a lower priority item to fix this -or- even better, hopefully some savvy user who needs it will send a pull request :)

[SergeyKanzhelev]

@nizarq, @jasongin have you had a chance to look at microsoft/ApplicationInsights-node.js#143?

Will we need to disable azure storage domains by default? Will disabling by "contains" work here and no disable too much/too little?

[AlexBulankou]

Discussed during planning. For 2.3 Beta1 the minimum requirement is to have a flag that disables cross-component correlation, and the list of domains we can add later.

[nizarq]

The configuration we ended up with is as follows:
For DependencyTrackingModule:

    <Add Type="Microsoft.ApplicationInsights.DependencyCollector.DependencyTrackingTelemetryModule, Microsoft.AI.DependencyCollector">
      <SetComponentCorrelationHttpHeaders>true</SetComponentCorrelationHttpHeaders>
      <ExcludeComponentCorrelationHttpHeadersOnDomains>
        <Add>www.bbc.com</Add>
        <Add>http://www.msn.com.cn/news</Add>
      </ExcludeComponentCorrelationHttpHeadersOnDomains>
    </Add>

For RequestTrackingModule

    <Add Type="Microsoft.ApplicationInsights.Web.RequestTrackingTelemetryModule, Microsoft.AI.Web">
      <SetComponentCorrelationHttpHeaders>false</SetComponentCorrelationHttpHeaders>
      <Handlers>...</Handlers>
    </Add>