TUF repo control plane artifact should be split into individual zone artifacts

[davepacheco]

I don't have a lot of the background on this but I gathered from discussions with @iliana that this is something we want to do.

[iliana]

In the past I think we wanted to do this so that it'd be easier to avoid needing to update individual control plane zones if nothing has changed between releases. At our current release cadence this seems unlikely.

But I still think we need to split these out, at least when we extract artifacts from the repository, in order for sled-agent to be able to directly use the artifacts on the M.2 when creating control plane zones. Because the extraction code is in the update-common crate, this also involves changing installinator, I think.

[iliana]

Here's my proposal for doing this work; it's split into a few phases. Would like @sunshowers and @jgallagher's thoughts before I dig into implementation.

The first phase is to change the update-common crate to extract the individual control plane zones out of the composite control plane artifact. This would mean Nexus and Wicket would perceive the TUF repo as having several zone artifacts. Wicket and Installinator need to be changed to support this (or perhaps Wicket could just re-tar up the individual zone artifacts on the fly). At the same time, we can make Nexus and Wicket handle repos that contain several zone artifacts instead of a composite artifact.

At least one release later, we can start producing TUF repos that no longer use the composite control plane artifact.

I might be missing some context here as I type this up; please ask questions if this plan is missing something.

[sunshowers]

Wicket and Installinator need to be changed to support this (or perhaps Wicket could just re-tar up the individual zone artifacts on the fly).

Ah I'm a little concerned about this step, because currently installinator is hardcoded to obtain artifacts via two hashes (and that's what the old version of wicket would write out as the installinator image ID.)

Ideally we'd change the semantics of the installinator image ID to write out a single ID, which fetches a manifest with a list of artifacts -- but this may have to be done across more than one release -- because current wicket doesn't understand this format.

In general, how are you thinking about retaining compatibility between old versions of wicket and newer installinator? This has always been a challenge -- our model involves wicket not just being a dumb pipe but actually parsing the contents of known artifact IDs. Unknown artifact IDs are just passed through.

[iliana]

Oh right, there is a backwards compatibility thing there, because installinator comes from the repo uploaded to wicket...

[jgallagher]

To add some flavor and details on the backwards compatibility issue:

• wicket on release N has to be able to read the TUF repo of release N+1
• installinator in release N+1 has to fetch its data from wicket that is running release N but serving release N+1
• The two hashes thing @sunshowers mentioned is serialized as an InstallinatorImageId and sent from wicket to installinator via the SP and IPCC. That could expand to grow more stuff, but has some size limits that we'd bump into if doing so (a single MGS -> SP UDP packet is currently limited to about 1 KiB, although we could make it span multiple packets without too much work, and a single SP -> host IPCC packet is limited to 4 KiB; making that one span multiple packets is doable but more work). The two hashes are currently interpreted as "the hash of the OS image" and "the hash of the control plane combo tarball", and I think Rain is right in that if we want to send multiple hashes to installinator, it's probably easier to have the "control plane hash" actually fetch a manifest of some kind instead of trying to expand InstallinatorImageId itself.

Given all of that I think I'd lean towards "wicketd repackages the control plane artifacts into a tarball for installinator"? That doesn't seem like it would be too bad, since the repackaged thing is just a plain tarball. ("installinator fetches a manifest" seems fine too but more work?) Even that requires staging across a couple of releases, I think; e.g., release N teaches wicket how to handle TUF repos with split-up artifacts but doesn't itself use that format; release N+1 uses the split-up format now that wicket N knows how to repackage it.

[iliana]

As I was going to bed last night I realized I could also make it so that update-common has two paths, one which always splits the control plane artifacts, and one which always combines them (if needed), and Nexus and wicket could use those paths respectively.