Allow setting numbers as tags

[felixbarny]

Having the ability to set custom number values which are indexed in Elasticsearch allows to create aggregated graphs based on transactions.

For example, you could track the number of search results for a search transaction, along with the search query to get a list of the top searches without results.

[simitt]

Do you mean something similar as marks, which can be nested objects with number values?

This wouldn't be a big issue from a technical point of view (except for the one open issue that we also have with marks #708).

[felixbarny]

Yes, similar to marks, but mark values should always be a timestamp in ms, right? Shouldn't we map it to a date then?

For this, we should pre-define a mapping so that we don't have the issue mentioned in #708. Double would be the most flexible datatype for this I guess.

[simitt]

Are you considering nested objects with double values, or a flat hierarchy as for tags?

[felixbarny]

Just flat key value pairs with double values

[alvarolobato]

@felixbarny is this related to Open Tracing?

[felixbarny]

OpenTracing allows to set string, boolean and number tags. Converting the numbers to strings does not really break the contract, but it means the users can't create graphs based on the number tags.

Example: the user could add a tag to a checkout cart span which represents the total value of the cart. If we don't support number tags, the user can't create a Kibana visualization with graphs the revenue.

[roncohen]

@felixbarny and I discussed offline and while OT prescribes both:

• Span.setTag(String key, Number value)
• Span.setTag(String key, String value)

it doesn't prescribe how the values should be stored or how users should be able to query them. I understand this would be useful, but in order to avoid holding up OT support, let's consider this to be unrelated to OT.

[watson]

Side-note: If it's easy to do while we're changing this anyway, I would like to propose that we also support boolean values.

[toughrogrammer]

Is it on working?

[simitt]

We are investigating options to implement this, but it hasn't been implemented yet.

[toughrogrammer]

Is there any roadmap about it? Or any other alternative solutions? And can I help anythings about it? :)

[simitt]

@felixbarny I suggest to use a scaled_float with relatively high precision instead of doubles, to reduce storage usage.

Possible solution in ES:

PUT apm-7.0.0-2018.12.22
{
  "mappings": {
    "_doc": {
      "dynamic_templates": [
        {
          "context.tags.keywords": {
            "path_match":   "context.tags.*",
            "match_mapping_type": "string",
            "mapping": {
              "type": "keyword"
            }
          }
        },
        {
          "context.tags.bool": {
            "path_match":   "context.tags.*",
            "match_mapping_type": "boolean",
            "mapping": {
              "type": "boolean"
            }
          }
        },
        {
          "context.tags.scaled_float": {
            "path_match":   "context.tags.*",
            "mapping" : {
              "scaling_factor" : 1000000,
              "type" : "scaled_float"
            }
          }
        }
      ]
    }
  }
}

Insert example doc

POST apm-7.0.0-2018.12.22/_doc
{
  "context": {
    "tags": {
      "ab": "sfmdskmfmsf",
      "cd": "1234",
      "ef": 1234,
      "fg": 234.980,
      "gh": true,
      "hj": "false"
    }
  }
}

Fetching mapping with GET apm*/_mapping returns:

{
  "apm-7.0.0-2018.12.22" : {
    "mappings" : {
      "_doc" : {
        "dynamic_templates" : [
          {
            "context.tags.keywords" : {
              "path_match" : "context.tags.*",
              "match_mapping_type" : "string",
              "mapping" : {
                "type" : "keyword"
              }
            }
          },
          {
            "context.tags.bool" : {
              "path_match" : "context.tags.*",
              "match_mapping_type" : "boolean",
              "mapping" : {
                "type" : "boolean"
              }
            }
          },
          {
            "context.tags.scaled_float" : {
              "path_match" : "context.tags.*",
              "mapping" : {
                "scaling_factor" : 1000000,
                "type" : "scaled_float"
              }
            }
          }
        ],
        "properties" : {
          "context" : {
            "properties" : {
              "tags" : {
                "properties" : {
                  "ab" : {
                    "type" : "keyword"
                  },
                  "cd" : {
                    "type" : "keyword"
                  },
                  "ef" : {
                    "type" : "scaled_float",
                    "scaling_factor" : 1000000.0
                  },
                  "fg" : {
                    "type" : "scaled_float",
                    "scaling_factor" : 1000000.0
                  },
                  "gh" : {
                    "type" : "boolean"
                  },
                  "hj" : {
                    "type" : "keyword"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

[simitt]

@toughrogrammer there is WIP PR #1712 which you can follow.