feat(common): Make ParseResult use generic Record type

[Cobalt0s]

Description

Currently, marshaling records that require extra processing—such as flattening—is not straightforward. Working with records of type map[string]any can be cumbersome in such cases, whereas using *ajson.Node simplifies the process.

Solution

common.ParseResult relies on a generic record type:

• map[string]any – Convenient when raw and field values are similar.
• *ajson.Node – Requires less code for JSON manipulation and should be used when extracting specific fields.

Utilities

Additionaly added a common.RecordFlattener method, which moves fields from nested object to the root.
Outreach and Klaviyo connectors produce response with similiar structure and would benefit from this method.
The algorithm is the same.

[Cobalt0s]

• feat(common): Make ParseResult use generic Record type #1321 : 6 dependent PRs (#1320 , #1324 , #1325 and 3 others) 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[RajatPawar]

Let's hold off on this until we get around to refactoring some of our HTTP clients and have a good understanding of where parsing lives there - maybe it can be a part of that. Also, type constraints come with certain limitations. Until then feel free to add functions to common to flatten records, etc

[Cobalt0s]

Let's hold off on this until we get around to refactoring some of our HTTP clients and have a good understanding of where parsing lives there - maybe it can be a part of that. Also, type constraints come with certain limitations. Until then feel free to add functions to common to flatten records, etc

@RajatPawar common.ParseResult had its limitation that marshalFunc only operates on map[string]any. For those connectors where raw differed from fields it made marshalFunc unnecessary complex, where it either had to operate on map[string]any by undoing work done by recordsFunc or create higher order function.

Instead I am suggesting to match output of recordsFunc with input of marshalFunc. By having this the connector implementation becomes a one liner: common.MakeMarshaledDataFunc(common.RecordFlattener("attributes")).

Could we have ParseResult live in parallel 🙏 to the refactored version, in whenever place it will be or level?

The generic could be changed to type any if you dont like type constraints. The implementation goal is to preserve the matching chain of types from recordsFunc to marshalFunc. Not having this kind of seperation produced a bug where raw and fields were processed in recordsFunc affecting both.