Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for setting trace analytics via environment variables. #104

Merged
merged 3 commits into from
Jul 18, 2019

Conversation

cgilmour
Copy link
Contributor

This PR makes it possible to set the trace analytics metric on traces via an environment variable.
Users of this tracer are generally not directly writing C++ code, but using it indirectly via another application (eg: nginx, envoy, istio).
Under those circumstances, the most practical way of controlling a feature like this is via environment variables.

The variables used are:

  • DD_TRACE_ANALYTICS_ENABLED: can be true (rate=1.0), false(rate=0.0) or unset (no rate)
  • DD_GLOBAL_ANALYTICS_SAMPLE_RATE: a valid value between 0.0 and 1.0 inclusive.

@cgilmour cgilmour added this to the 1.0.0 milestone Jul 12, 2019
@ellieayla
Copy link

I'd like to get this working for the Ingress Nginx controller.

@cgilmour cgilmour force-pushed the cgilmour/analytics-env-var branch from 3211ec6 to b40c145 Compare July 17, 2019 01:27
@cgilmour
Copy link
Contributor Author

Made some changes to this implementation so that the metric is added to only the local-root span, after a trace is completed and had not been set directly.

This required some refactoring (cleanup) of the hostname tag, which also gets applied on local-root spans when finishing.

Tests have been added that execute the behavior via mocks (unit) and by setting the environment variable, creating new tracers and generating spans (integration).
The tests needed an update to the framework (catch2) to support a new macro that was added recently and needed for these tests.

@cgilmour cgilmour merged commit 2c888d6 into master Jul 18, 2019
@cgilmour cgilmour deleted the cgilmour/analytics-env-var branch July 18, 2019 02:14
@cgilmour
Copy link
Contributor Author

@alanjcastonguay from testing this before shipping our release, it's not as easy as hoped.

I'd expected it to be simple - just add the variables to the ingress-controller deployment.
However, nginx "clears" the environment before launching things, and needs to be told to pass through certain environment variables.
http://nginx.org/en/docs/ngx_core_module.html#env

A short term hack would be users providing a custom nginx template.
But it'd be nice to not have that burden, and have the right settings in the default nginx template.

@cgilmour
Copy link
Contributor Author

@alanjcastonguay I've submitted kubernetes/ingress-nginx#4371 to update to 1.0.1 so this should be in their next release. To make it work, it requires two things:

  • setting env vars with values in the nginx-ingress-controller deployment, eg:
            - name: HOST_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.hostIP
            - name: DD_ENV
              value: "test"
            - name: DD_TRACE_ANALYTICS_ENABLED
              value: "true"
  • setting the main-snippet in the configmap to pass those values through to nginx, eg:
  enable-opentracing: "true"
  datadog-collector-host: $HOST_IP
  main-snippet: |
    env DD_ENV;
    env DD_TRACE_ANALYTICS_ENABLED;

I suppose they could be set directly in main-snippet instead of needing two changes.
Not sure which is better.

@ellieayla
Copy link

ellieayla commented Jul 29, 2019

The template is already tweakable based on the tracer being enabled. Maybe go code in the Ingress-nginx controller should always set env DD_ENV and env DD_TRACE_ANALYTICS_ENABLED when the datadog tracer is enabled. Does having those env directives present without the corresponding environment variables set in the deployment's pod's container spec have any downsides? Doesn't seem to. Making a draft PR...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants