Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix]: Use scaled float for all numeric transaction.marks.* values. #1704

Merged
merged 5 commits into from
Dec 21, 2018

Conversation

simitt
Copy link
Contributor

@simitt simitt commented Dec 21, 2018

Follow up on #708.

This PR intends to ensure that all values stored under transaction.marks.*.* are stored as scaled float.

Tested with:

POST apm-7.0.0-2018.12.21/_doc
{
  "transaction": {
    "marks": {
      "navigationTiming": { 
        "ab": 1,
        "bc": 1.345
      },
      "else": {
        "cd": 1.90,
        "de": 1
      }
    }
  }
}

leads to following mapping:

"marks" : {
                "dynamic" : "true",
                "properties" : {
                  "*" : {
                    "properties" : {
                      "*" : {
                        "type" : "object",
                        "dynamic" : "true"
                      }
                    }
                  },
                  "else" : {
                    "properties" : {
                      "cd" : {
                        "type" : "scaled_float",
                        "scaling_factor" : 1000000.0
                      },
                      "de" : {
                        "type" : "scaled_float",
                        "scaling_factor" : 1000000.0
                      }
                    }
                  },
                  "navigationTiming" : {
                    "properties" : {
                      "ab" : {
                        "type" : "scaled_float",
                        "scaling_factor" : 1000000.0
                      },
                      "bc" : {
                        "type" : "scaled_float",
                        "scaling_factor" : 1000000.0
                      }
                    }
                  }
                }
              },

Invalid payload:

POST apm-7.0.0-2018.12.21/_doc
{
  "transaction": {
    "marks": {
      "navigationTiming": { 
        "ab": 1,
        "bc": 1.345
      },
      "else": {
        "cd": 1.90,
        "de": 1,
        "ef": {
          "gh": 1.2
        }
      }
    }
  }
}

leads to:

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "failed to parse field [transaction.marks.else.ef] of type [scaled_float]"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "failed to parse field [transaction.marks.else.ef] of type [scaled_float]",
    "caused_by": {
      "type": "json_parse_exception",
      "reason": "Current token (START_OBJECT) not numeric, can not use numeric value accessors\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@1811506a; line: 11, column: 16]"
    }
  },
  "status": 400
}

cc @jahtalab and @vigneshshanmugam

@graphaelli maybe I am overlooking something here, but it seems to me that setting scaled floats on a wildcard works as expected now.

@simitt simitt requested a review from graphaelli December 21, 2018 13:22
@simitt simitt added the bug label Dec 21, 2018
@graphaelli
Copy link
Member

I'd be really glad to hear that someone added support for that. Looking back at #708 (comment) it does seem like it didn't used to work.

@simitt
Copy link
Contributor Author

simitt commented Dec 21, 2018

It still only ensures all number types are stored as scaled_floats, but as we don't allow anything else than numbers on the intake level, that should be fine.

@hmdhk
Copy link
Contributor

hmdhk commented Dec 21, 2018

Thanks @simitt! The structure looks correct, on the agent side we always set numbers for marks, so as you said it should not be a problem!

@simitt
Copy link
Contributor Author

simitt commented Dec 21, 2018

Also checked how this behaves when upgrading from an older version, where other marks have been indexed as long or float and couldn't see any issues with that.

@simitt simitt merged commit a482a3e into elastic:master Dec 21, 2018
simitt added a commit to simitt/apm-server that referenced this pull request Dec 21, 2018
…lastic#1704)

* Use scaled float for all transaction,marks.* values.
* Exclude `transaction.marks.*.*` from Kibana index pattern.
simitt added a commit to simitt/apm-server that referenced this pull request Dec 21, 2018
…lastic#1704)

* Use scaled float for all transaction,marks.* values.
* Exclude `transaction.marks.*.*` from Kibana index pattern.
simitt added a commit that referenced this pull request Dec 22, 2018
… values. (#1709)

* [fix]: Use scaled float for all numeric transaction.marks.* values. (#1704)
* Exclude `transaction.marks.*.*` from Kibana index pattern.
simitt added a commit that referenced this pull request Dec 22, 2018
… values. (#1708)

* [fix]: Use scaled float for all numeric transaction.marks.* values. (#1704)
* Exclude `transaction.marks.*.*` from Kibana index pattern.
@simitt simitt deleted the marks-scaled-floats branch January 31, 2019 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants