Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Cloud Metadata #1003

Merged
merged 6 commits into from
Nov 10, 2020
Merged

Add Cloud Metadata #1003

merged 6 commits into from
Nov 10, 2020

Conversation

russcam
Copy link
Contributor

@russcam russcam commented Nov 3, 2020

Relates: elastic/apm#256

This commit adds the fetching of cloud metadata to
the agent, to send as part of the agent metadata
when an agent is running in a Cloud provider.

On starting the payload sender's work loop,
the first iteration will asynchronously fetch
the metadata for the cloud provider specified with
the ELASTIC_APM_CLOUD_PROVIDER env var, or the
ElasticApm:CloudProvider app setting. Values
are aws, gcp, azure and false. When false is
specified, no cloud metadata providers are
registered, so there are no attempts to fetch
metadata. The default value of null or ""
will attempt to fetch metadata using all registered
providers.

Closes #918

This commit adds the fetching of cloud metadata to
the agent, to send as part of the agent metadata
when an agent is running in a Cloud provider.

On starting the payload sender's work loop,
the first iteration will asynchronously fetch
the metadata for the cloud provider specified with
the ELASTIC_APM_CLOUD_PROVIDER env var, or the
ElasticApm:CloudProvider app setting. Values
are aws, gcp, azure and false. When false is
specified, no cloud metadata providers are
registered, so there are no attempts to fetch
metadata. The default value of null or ""
will attempt to fetch metadata using all registered
providers.

Closes elastic#918
@apmmachine
Copy link
Contributor

apmmachine commented Nov 3, 2020

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [russcam commented: jenkins run the tests please]

  • Start Time: 2020-11-10T03:53:02.644+0000

  • Duration: 65 min 25 sec

Test stats 🧪

Test Results
Failed 2
Passed 14648
Skipped 0
Total 14650

Test errors 2

Expand to view the tests failures

  • Name: Initializing / Parallel / Linux / Test / LabelTests.Elastic.Apm.Tests.LabelTests.LabelsOnErrorTest – Elastic.Apm.Tests.LabelTests

    • Age: 1 (took 0.074 sec)
    • Error Details: System.NullReferenceException : Object reference not set to an instance of an object.
    • Error Stacktrace:
System.NullReferenceException : Object reference not set to an instance of an object.
Stack Trace:
  at Elastic.Apm.Tests.LabelTests.<>c__DisplayClass7_0.<LabelsOnErrorTest>b__0(ITransaction t) in /var/lib/jenkins/workspace/net_apm-agent-dotnet-mbp_PR-1003/apm-agent-dotnet/test/Elastic.Apm.Tests/LabelTests.cs:line 141
  at Elastic.Apm.Api.Tracer.CaptureTransaction(String name, String type, Action`1 action, DistributedTracingData distributedTracingData) in /var/lib/jenkins/workspace/net_apm-agent-dotnet-mbp_PR-1003/apm-agent-dotnet/src/Elastic.Apm/Api/Tracer.cs:line 68
  at Elastic.Apm.Tests.LabelTests.LabelsOnErrorTest() in /var/lib/jenkins/workspace/net_apm-agent-dotnet-mbp_PR-1003/apm-agent-dotnet/test/Elastic.Apm.Tests/LabelTests.cs:line 132

  • Name: Initializing / Parallel / Linux / Test / LabelTests.Elastic.Apm.Tests.LabelTests.SingleLabelOnTransactionTests(labelValue: "StrValue") – Elastic.Apm.Tests.LabelTests

    • Age: 1 (took 0.2074822 sec)
    • Error Details: Expected jsonString "{"id":"e480c74e16cbae41","sampled":false,"name":"test","outcome":"unknown","span_count":{"dropped":0,"started":0},"timestamp":1604981516246314,"trace_id":"271e9a9c0d7e55478733b3d9277f6b80","type":"test"}" to contain ""tags":{"myLabel":"StrValue"}".
    • Error Stacktrace:
Expected jsonString "{"id":"e480c74e16cbae41","sampled":false,"name":"test","outcome":"unknown","span_count":{"dropped":0,"started":0},"timestamp":1604981516246314,"trace_id":"271e9a9c0d7e55478733b3d9277f6b80","type":"test"}" to contain ""tags":{"myLabel":"StrValue"}".
Stack Trace:
  at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message)
  at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
  at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message)
  at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
  at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args)
  at FluentAssertions.Primitives.StringAssertions.Contain(String expected, String because, Object[] becauseArgs)
  at Elastic.Apm.Tests.LabelTests.<>c__DisplayClass2_0.<SingleLabelOnTransactionTests>b__0(ITransaction t) in /var/lib/jenkins/workspace/net_apm-agent-dotnet-mbp_PR-1003/apm-agent-dotnet/test/Elastic.Apm.Tests/LabelTests.cs:line 45
  at Elastic.Apm.Api.Tracer.CaptureTransaction(String name, String type, Action`1 action, DistributedTracingData distributedTracingData) in /var/lib/jenkins/workspace/net_apm-agent-dotnet-mbp_PR-1003/apm-agent-dotnet/src/Elastic.Apm/Api/Tracer.cs:line 68
  at Elastic.Apm.Tests.LabelTests.SingleLabelOnTransactionTests(Object labelValue) in /var/lib/jenkins/workspace/net_apm-agent-dotnet-mbp_PR-1003/apm-agent-dotnet/test/Elastic.Apm.Tests/LabelTests.cs:line 37

Log output

Expand to view the last 100 lines of log output

[2020-11-10T04:58:16.963Z] info: Microsoft.AspNetCore.Mvc.StatusCodeResult[1]
[2020-11-10T04:58:16.963Z]       Executing HttpStatusCodeResult, setting HTTP status code 304
[2020-11-10T04:58:16.963Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
[2020-11-10T04:58:16.963Z]       Executed action Elastic.Apm.Tests.MockApmServer.Controllers.AgentsConfigController.Get (Elastic.Apm.Tests.MockApmServer) in 0.2466ms
[2020-11-10T04:58:16.963Z] info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
[2020-11-10T04:58:16.963Z]       Request finished in 0.6457ms 304 
[2020-11-10T04:58:16.963Z] info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
[2020-11-10T04:58:16.963Z]       Request starting HTTP/1.1 POST http://localhost:23634/intake/v2/events application/x-ndjson; charset=utf-8 3139
[2020-11-10T04:58:16.963Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[3]
[2020-11-10T04:58:16.964Z]       Route matched with {action = "Post", controller = "IntakeV2Events"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Post() on controller Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController (Elastic.Apm.Tests.MockApmServer).
[2020-11-10T04:58:16.964Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
[2020-11-10T04:58:16.964Z]       Executing action method Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController.Post (Elastic.Apm.Tests.MockApmServer) - Validation state: Valid
[2020-11-10T04:58:16.964Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
[2020-11-10T04:58:16.964Z]       Executed action method Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController.Post (Elastic.Apm.Tests.MockApmServer), returned result Microsoft.AspNetCore.Mvc.OkObjectResult in 2.2187ms.
[2020-11-10T04:58:16.964Z] info: Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor[1]
[2020-11-10T04:58:16.964Z]       Executing ObjectResult, writing value of type 'System.String'.
[2020-11-10T04:58:16.964Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
[2020-11-10T04:58:16.964Z]       Executed action Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController.Post (Elastic.Apm.Tests.MockApmServer) in 2.4943ms
[2020-11-10T04:58:16.964Z] info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
[2020-11-10T04:58:16.964Z]       Request finished in 2.8111ms 200 text/plain; charset=utf-8
[2020-11-10T04:58:16.964Z] info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
[2020-11-10T04:58:16.964Z]       Request starting HTTP/1.1 POST http://localhost:23634/intake/v2/events application/x-ndjson; charset=utf-8 11161
[2020-11-10T04:58:16.964Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[3]
[2020-11-10T04:58:16.964Z]       Route matched with {action = "Post", controller = "IntakeV2Events"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Post() on controller Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController (Elastic.Apm.Tests.MockApmServer).
[2020-11-10T04:58:16.964Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
[2020-11-10T04:58:16.964Z]       Executing action method Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController.Post (Elastic.Apm.Tests.MockApmServer) - Validation state: Valid
[2020-11-10T04:58:16.964Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
[2020-11-10T04:58:16.964Z]       Executed action method Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController.Post (Elastic.Apm.Tests.MockApmServer), returned result Microsoft.AspNetCore.Mvc.OkObjectResult in 2.3293ms.
[2020-11-10T04:58:16.964Z] info: Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor[1]
[2020-11-10T04:58:16.964Z]       Executing ObjectResult, writing value of type 'System.String'.
[2020-11-10T04:58:16.964Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
[2020-11-10T04:58:16.964Z]       Executed action Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController.Post (Elastic.Apm.Tests.MockApmServer) in 2.5763ms
[2020-11-10T04:58:16.964Z] info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
[2020-11-10T04:58:16.964Z]       Request finished in 2.8467ms 200 text/plain; charset=utf-8
[2020-11-10T04:58:17.223Z] info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
[2020-11-10T04:58:17.223Z]       Request starting HTTP/1.1 POST http://localhost:23634/intake/v2/events application/x-ndjson; charset=utf-8 1026
[2020-11-10T04:58:17.223Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[3]
[2020-11-10T04:58:17.223Z]       Route matched with {action = "Post", controller = "IntakeV2Events"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Post() on controller Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController (Elastic.Apm.Tests.MockApmServer).
[2020-11-10T04:58:17.223Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
[2020-11-10T04:58:17.223Z]       Executing action method Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController.Post (Elastic.Apm.Tests.MockApmServer) - Validation state: Valid
[2020-11-10T04:58:17.223Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
[2020-11-10T04:58:17.223Z]       Executed action method Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController.Post (Elastic.Apm.Tests.MockApmServer), returned result Microsoft.AspNetCore.Mvc.OkObjectResult in 1.519ms.
[2020-11-10T04:58:17.223Z] info: Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor[1]
[2020-11-10T04:58:17.223Z]       Executing ObjectResult, writing value of type 'System.String'.
[2020-11-10T04:58:17.223Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
[2020-11-10T04:58:17.223Z]       Executed action Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController.Post (Elastic.Apm.Tests.MockApmServer) in 1.7868ms
[2020-11-10T04:58:17.223Z] info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
[2020-11-10T04:58:17.223Z]       Request finished in 2.0622ms 200 text/plain; charset=utf-8
[2020-11-10T04:58:17.223Z] info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
[2020-11-10T04:58:17.223Z]       Request starting HTTP/1.1 POST http://localhost:23634/intake/v2/events application/x-ndjson; charset=utf-8 1465
[2020-11-10T04:58:17.223Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[3]
[2020-11-10T04:58:17.223Z]       Route matched with {action = "Post", controller = "IntakeV2Events"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Post() on controller Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController (Elastic.Apm.Tests.MockApmServer).
[2020-11-10T04:58:17.223Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
[2020-11-10T04:58:17.223Z]       Executing action method Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController.Post (Elastic.Apm.Tests.MockApmServer) - Validation state: Valid
[2020-11-10T04:58:17.223Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
[2020-11-10T04:58:17.223Z]       Executed action method Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController.Post (Elastic.Apm.Tests.MockApmServer), returned result Microsoft.AspNetCore.Mvc.OkObjectResult in 2.1281ms.
[2020-11-10T04:58:17.224Z] info: Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor[1]
[2020-11-10T04:58:17.224Z]       Executing ObjectResult, writing value of type 'System.String'.
[2020-11-10T04:58:17.224Z] info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
[2020-11-10T04:58:17.224Z]       Executed action Elastic.Apm.Tests.MockApmServer.Controllers.IntakeV2EventsController.Post (Elastic.Apm.Tests.MockApmServer) in 2.3757ms
[2020-11-10T04:58:17.224Z] info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
[2020-11-10T04:58:17.224Z]       Request finished in 2.7041ms 200 text/plain; charset=utf-8
[2020-11-10T04:58:18.597Z] Elastic APM .NET Tests> 96> Console> [2020-11-10 04:58:18.464 +00:00][Info] - {LoggingTestBase} Finished test: Elastic.Apm.AspNetFullFramework.Tests.CentralConfigTests+MaxSpansAndSampleRateTests.MaxSpans_invalid_value
[2020-11-10T04:58:18.597Z] [xUnit.net 00:17:09.9326709]   Finished:    Elastic.Apm.AspNetFullFramework.Tests
[2020-11-10T04:58:18.854Z]   û Elastic.Apm.AspNetFullFramework.Tests.CentralConfigTests+MaxSpansAndSampleRateTests.MaxSpans_invalid_value [10s 362ms]
[2020-11-10T04:58:18.854Z] 
[2020-11-10T04:58:18.854Z] JunitXML Logger - Results File: C:\Users\jenkins\workspace\net_apm-agent-dotnet-mbp_PR-1003\apm-agent-dotnet\test\Elastic.Apm.AspNetFullFramework.Tests\junit-NETCoreApp21-Elastic.Apm.AspNetFullFramework.Tests.xml
[2020-11-10T04:58:18.854Z] 
[2020-11-10T04:58:18.854Z] Test Run Successful.
[2020-11-10T04:58:18.854Z] Total tests: 111
[2020-11-10T04:58:18.854Z]      Passed: 111
[2020-11-10T04:58:18.854Z]  Total time: 17.2148 Minutes
[2020-11-10T04:58:18.854Z] 
[2020-11-10T04:58:18.854Z] Build succeeded.
[2020-11-10T04:58:18.854Z]     0 Warning(s)
[2020-11-10T04:58:18.854Z]     0 Error(s)
[2020-11-10T04:58:18.854Z] 
[2020-11-10T04:58:18.854Z] Time Elapsed 00:17:14.04
[2020-11-10T04:58:19.816Z] Post stage
[2020-11-10T04:58:19.825Z] Running in C:\Users\jenkins\workspace\net_apm-agent-dotnet-mbp_PR-1003\apm-agent-dotnet
[2020-11-10T04:58:19.845Z] Archiving artifacts
[2020-11-10T04:58:20.211Z] Recording test results
[2020-11-10T04:58:21.614Z] Post stage
[2020-11-10T04:58:21.629Z] [WS-CLEANUP] Deleting project workspace...
[2020-11-10T04:58:21.629Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2020-11-10T04:58:25.715Z] [WS-CLEANUP] done
[2020-11-10T04:58:25.931Z] Stage "Release to feedz.io" skipped due to when conditional
[2020-11-10T04:58:25.989Z] Stage "Release" skipped due to when conditional
[2020-11-10T04:58:26.008Z] Stage "Release" skipped due to when conditional
[2020-11-10T04:58:26.057Z] Stage "Release" skipped due to when conditional
[2020-11-10T04:58:26.154Z] Stage "AfterRelease" skipped due to when conditional
[2020-11-10T04:58:26.173Z] Stage "AfterRelease" skipped due to when conditional
[2020-11-10T04:58:26.409Z] Running on worker-1225339 in /var/lib/jenkins/workspace/net_apm-agent-dotnet-mbp_PR-1003
[2020-11-10T04:58:26.460Z] [INFO] getVaultSecret: Getting secrets
[2020-11-10T04:58:26.595Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-11-10T04:58:28.622Z] + chmod 755 generate-build-data.sh
[2020-11-10T04:58:28.623Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-dotnet/apm-agent-dotnet-mbp/PR-1003/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-dotnet/apm-agent-dotnet-mbp/PR-1003/runs/9 UNSTABLE 3924591
[2020-11-10T04:58:28.623Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-dotnet/apm-agent-dotnet-mbp/PR-1003/runs/9/steps/?limit=10000 -o steps-info.json
[2020-11-10T04:58:29.324Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-dotnet/apm-agent-dotnet-mbp/PR-1003/runs/9/tests/?status=FAILED -o tests-errors.json
[2020-11-10T04:58:30.027Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-dotnet/apm-agent-dotnet-mbp/PR-1003/runs/9/log/ -o pipeline-log.txt

@russcam
Copy link
Contributor Author

russcam commented Nov 4, 2020

jenkins run the tests please

@russcam russcam marked this pull request as ready for review November 5, 2020 05:47
@russcam russcam requested a review from gregkalapos November 5, 2020 05:48
@russcam
Copy link
Contributor Author

russcam commented Nov 5, 2020

Will follow up in a separate PR for retrieving metadata for Azure App Services

Copy link
Contributor

@gregkalapos gregkalapos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, LGTM!

I left 1 comment regarding printing warnings - definitely not a blocker, it's for consideration.

docs/configuration.asciidoc Outdated Show resolved Hide resolved
}
catch (Exception e)
{
_logger.Warning()?.LogException(e, "Unable to get {Provider} cloud metadata", Provider);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure about the Warning level? Given the default for this setting is to try all ICloudMetadataProvider implementations, the agent will start with 3 warnings in non-cloud and with 2 if it runs in one of the cloud providers.

I feel we should use info here.

We have a similar situation with the central config fetcher and I see people are very confused about those warnings as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will change to info - agree that warning seems way too strong

Copy link
Member

@bmorelli25 bmorelli25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as Greg, otherwise docs LG

@russcam
Copy link
Contributor Author

russcam commented Nov 10, 2020

jenkins run the tests please

@russcam russcam merged commit ee6d5a3 into elastic:master Nov 10, 2020
@russcam russcam deleted the feature/cloud-metadata branch November 10, 2020 05:01
@russcam russcam added v1.7.0 enhancement New feature or request labels Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request v1.7.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Collect cloud metadata
4 participants