feat(rum-core): limit the length of keyword/non-keyword fields

[vigneshshanmugam]

• fixes Remove null and undefined values from apm-server payload #61

• We now have the ability to truncate both keyword and non keywords fields to APM Server at a single place. This applies to Metadata, Transactions, Spans and Errors - Future: Metricsets as well

• Inspired heavily from Node.js agent truncation module. Instead of truncating the string at creation time, We are now moving the truncation when the payload is sent to the server. This enables us to measure the performance of payload creation and filtering that gets applied.

• All the undefined and null values are removed from the server payload to reduce the payload size. This is done using required field in truncation.

[hmdhk]

@vigneshshanmugam , This one seems to have commits from another PR!

[vigneshshanmugam]

@jahtalab Yes, This should be merged after merging the other PR. I have added the note in the description

[vigneshshanmugam]

The test breaks because of the increase in bundle size due to changes in Task API as well the current PR. The increase in 0.35kb in the budget is fine since one is a feature and other is a consolidation of payload that gets sent to the APM server. I am going to increase the bundle size budget value by 0.5 kB to unblock.

We will improve the bundle size as part of #159

[vigneshshanmugam]

@jahtalab Can you please review

[vigneshshanmugam]

Waiting for prototype from Hamid on an alternative solution. Should be merged after agreeing on a decision.

[hmdhk]

@vigneshshanmugam , Since I might not have the time to make a prototype let's try to reach a conclusion here:

My main concern is that maintaining a switch-case is harder compared to having an object that specifies specification for each field. Furthermore with a spec object we can write a generic function that iterates over the required fields instead of spreading and potentially duplicating the logic across multiple functions, as it is done here with truncateSpan, truncateTransaction etc. Lastly, this approach has the advantage that we only iterate over a predefined and smaller spec object rather than iterating on an unknown object, i.e the payload fields which can have arbitrary fields (for example consider the error stack trace). In other words, the complexity of the function depends on the predefined (by us and known during development) number of fields rather than an unknown and potentially deeply nested object.

[vigneshshanmugam]

Notes:

• We decided to go with Spec level instead of going over the . data payload to make the traversal time smaller

var fieldSpec = {
 'service': {
   'version': { required: false, limit:1024 }
 }
}

• Non-Keyword fields can be removed from the spec for now.

[hmdhk]

Final Implementation have been merged with #241, closing this one.