model: Align labels field with ECS

[marclop]

Motivation/summary

Updates all data streams to use the ecs defined labels for values that
are string or bool, and adds a new field for numeric_labels which
stores all numeric labels as scaled_float, using the scaling_factor
we were using for indexes.

There's a single exception for the metrics-apm.internal data stream,
due to the fact that dynamic is set to strict, which disables the
externally defined ecs labels dynamic ability. For that data stream, the
field is defined as a standard field.

Checklist

• Update CHANGELOG.asciidoc
  - [ ] Documentation has been updated

How to test these changes

1. Install the integration.
2. Ingest some data with both numeric, string and bool labels.
3. Verify that the string and bool labels are under the labels field
4. Verify that numeric (floats and ints) are stored under numeric_labels.

Related issues

Closes #3873

[apmmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2021-11-18T03:07:12.441+0000

• Duration: 49 min 22 sec

• Commit: 453007c

Test stats 🧪

Test	Results
Failed	0
Passed	5638
Skipped	19
Total	5657

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /hey-apm : Run the hey-apm benchmark.

• /package : Generate and publish the docker images.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request is now in conflicts. Could you fix it @marclop? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b f/labels-split-into-string-and-numeric-ecs-compatible upstream/f/labels-split-into-string-and-numeric-ecs-compatible
git merge upstream/master
git push upstream f/labels-split-into-string-and-numeric-ecs-compatible

[axw]

I think we have to define the field everywhere, and not use references. This data stream doesn't use strict mapping, but it does set dynamic: false at the top level meaning labels won't be indexed.

[axw]

Maybe it's worth opening an issue on package-spec to allow specifying the dynamic property for external fields? Seems to me it should be reasonable to specify this:

- external: ecs
  name: labels
  dynamic: true

[marclop]

you're right, I missed that they won't be indexed if we just reference the ecs fields like that.

[marclop]

Opened elastic/package-spec#251

[axw]

Just a few minor things, othewise LGTM.

I'd like to follow up on this by changing the Labels and NumericLabels fields to be more type-safe. Either

type APMEvent struct {
    Labels map[string][]string
    NumericLabels map[string][]float64
}

Or

// Labels/NumericLabels values are either always stored as arrays,
// or store as an array iff len > 1
type APMEvent struct {
    Labels map[string]LabelValue
    NumericLabels map[string]NumericLabelValue
}

type LabelValue struct {
    Value string
    Values []string // If non-nil, use this; otherwise use Value
}

type NumericLabelValue struct {
    Value float64
    Values []float64 // If non-nil, use this; otherwise use Value
}

But to be clear, let's do that in a followup.

[axw]

LGTM! 🎉

[stuartnelson3]

confirmed with 50edba1

package main

import (
	"fmt"
	"os"

	"go.elastic.co/apm"
)

func main() {
	version := "undefined"
	if len(os.Args) > 1 {
		version = os.Args[1]
	}
	name := fmt.Sprintf("apm-server-%s", version)
	tx := apm.DefaultTracer.StartTransaction(name, "type")
	tx.Context.SetLabel("enabled", true)
	tx.Context.SetLabel("string", "string")
	tx.Context.SetLabel("numbers", 1234.56)
	tx.End()
	apm.DefaultTracer.Flush(nil)
	fmt.Printf("%s: %+v\n", name, apm.DefaultTracer.Stats())
}