Add support for dataset discovery from event

[rkpandey]

What are the schemas that are affected by the issue

context/ExperienceEvent

What are examples of products that are impacted by the issue

ACP

Requirement is to discover the dataset id to disambiguate un-normalized xdm attributes.

[fmeschbe]

Can you share some background why you want to encode the target dataset (which is specific to Adobe Cloud Platform) in the Experience Event ?

Encoding such an information in the event creates a pretty tight coupling.

[lrosenthol]

Exactly my concern, @fmeschbe . If this is XC specific, then it needs to go into an extension schema and then merged/combined with experienceEvent at runtime.

[cdegroot-adobe]

@rkpandey @fmeschbe @lrosenthol
To step back a bit and use non-adobe terms.
This property is being used to associate a storage instance with each record in that storage instance. The storage management system maintains additional metadata that needs to be looked up based on this identifier.
A common case we see is multiple storage instances being merged together (when they are of a compatible schema). When that happens we loose the link to the metadata specific to each row.

We have a related bit of metadata that is the datasource, which is used to identify the system that mastered the data originally. I think "datasource" is a good name for that case. In this case "dataset" has a good general meaning in that it describes a set of records that have common schemas and consistent meaning.

So I think the meaning is good generically(Adobe is not the only potential implementor that will want to use this property) as is the naming. The description is something that needs to reflect this better. I will propose a new naming in the PR.

[fmeschbe]

Thanks @cdegroot-adobe. Let me try to disect to see, whether I understand this. And I am going to use less generic terminology from traditional databases: database and table and row and column and field.

This property is being used to associate a storage instance with each record in that storage instance. The storage management system maintains additional metadata that needs to be looked up based on this identifier.
A common case we see is multiple storage instances being merged together (when they are of a compatible schema). When that happens we loose the link to the metadata specific to each row.

Is it correct to say, that this property comes into play when tables are merged into a new table to indicate the name of the original table before the merge. Correct ?

What if that resulting table is further merged ?

Is that property to be filled by the original Event sender ? Or is it auto-filled by the process which stores the event in the first table ?

The first option would create the coupling I was mentioning above, the second option would be a by-product of storing the event in the table.

Neither is good; the former definitely worse, of course. I think it should actually be a third option: As tables are merged, this field should be added into the merged table.

[cdegroot-adobe]

@fmeschbe The intent is the second option "auto-filled by the process which stores the event in the first table". It may do this on read rather than write, but that is an implementation detail.

[cdegroot-adobe]

@fmeschbe - Any further thoughts on this? We need to get this into the July 16th milestone, so I would like to make a decision asap on a standard XDM property or an Adobe Extension. I prefer the former and see the general utility of this in non-adobe cases. But given time constraints I can go with the extension if others are not of the same mind set.

[cdegroot-adobe]

Changed to an extension. Should be ready for merge now.

[fmeschbe]

Thanks @cdegroot-adobe, that helps me. And I am now +1 in general.

I see you changed it to be an extension in PR #426. Fine for me. Thanks.

[fmeschbe]

PR #426 is merged, so let's close this issue as well.