Formalize a "grouped lineage" concept

[sdboyer]

For Thema's versioning rules to be meaningfully enforceable, we have to retain the basic rule that lineages are self-contained structures - no external references. However, it is clear that there are cases where grouping of multiple "distinct" objects is desirable.

The particular thing that makes the grouped objects "distinct", and yet still desirably a member of the same group is probably important. Maybe even crucial to the right design. I don't have a good general model of it yet, though, so i'll give some examples in order to help work by induction.

One obvious example is the notion of a Kubernetes group-version, where multiple distinct objects are grouped and indeed versioned together. AIUI, this is essentially because

• The API generated and exposed for interacting with these objects is viewed as the irreducible unit
• It is undesirable that any objects within that GV should ever have to deal with version skew relative to the other objects within the GV.

In Grafana, we have another kind of case that's come up. Here's some WIP code:

// The slots named and specified in this file are meta-schemas that act as a
// shared contract between Grafana plugins (producers) and coremodel types
// (consumers).
//
// On the consumer side, any coremodel Thema lineage can choose to define a
// standard Thema composition slot that specifies one of these named slots as
// its meta-schema. Such a specification entails that all schemas in any lineage
// placed into that composition slot must adhere to the meta-schema.
//
// On the producer side, Grafana's plugin system enforces that certain plugin
// types are expected to provide Thema lineages for these named slots which
// adhere to the slot meta-schema.
//
// For example, the Panel slot is consumed by the dashboard coremodel, and is
// expected to be produced by panel plugins.
//
// The name given to each slot in this file must be used as the name of the
// slot in the coremodel, and the name of the field under which the lineage
// is provided in a plugin's models.cue file.
//
// Conformance to meta-schema is achieved by Thema's native lineage joinSchema,
// which Thema internals automatically enforce across all schemas in a lineage.

slots: Panel: {
  // Defines plugin-specific options for a panel that should be persisted. Required,
  // though a panel without any options may specify an empty struct.
  PanelOptions: {...}
  // Plugin-specific custom field properties. Optional.
  PanelFieldConfig?: {...}
}

// Meta-schema for the DSOptions slot, as implemented in Grafana datasource plugins.
// DSOptions slot joinSchema. This provides space for both the normal and
// encrypted configuration portions of a datasource plugin's options.
slots: DSOptions: {
  // Normal datasource configuration options.
  Options: {...}
  // Sensitive datasource configuration options that require encryption.
  SecureOptions: {...}
}

// Meta-schema for the Query slot, as implemented in Grafana datasource plugins.
slots: Query: {...}

This is specifying one conventional lineage (Query), and two grouped lineages (Panel, DSOptions). Crucially, it is not expected that there ever exists a literal object instance of the schema, as-written. Rather, the definitions are grouped together because:

• There is no use case in which it is valuable for them to be versioned independently
• They are expected to be consumed as a group by the object composing them

These two reasons look a lot like the k8s ones. I don't think that's an accident. Rather, I have a strong feeling that this is a case where finding and applying a few math-y formalisms will lead to an elegant solution, which ultimately might be called a "grouped lineage."

Almost certainly relates to #8, though i'm not sure how yet.