repo.RootKeys returns wrong metadata

[zwass]

Now that #354 is merged, the only way to initialize a client is using the full root metadata (please correct if I'm wrong here).

The repo.RootKeys implementation, which is documented to provide the metadata needed to initialize clients returns an array of public keys, which can no longer be used to initialize the client.

1. Is it correct to now distribute the full root.json to clients?
2. Should repo.RootKeys be updated to return the full root metadata? (found docs: update the CLI usage documentation #322 mentioning this)
3. What should be done for clients that have already been distributed with the old style of root keys (just the keys, not the full metadata)?

[zwass]

cc @znewman01 and @haydentherapper who were involved in these changes.

[znewman01]

We appreciate you bearing with us as we tweak the API; I think it's rough in the short term, but in the long term will lead to a more-usable and secure-by-default implementation.

Is it correct to now distribute the full root.json to clients?

Yes.

Should repo.RootKeys be updated to return the full root metadata? (found #322 mentioning this)

That's a good idea.

What should be done for clients that have already been distributed with the old style of root keys (just the keys, not the full metadata)?

I don't know if I see the issue here—those clients would be on an old version of go-tuf, which supports old-style initialization. Brand new clients will be on the new go-tuf version, but that's okay because we'll be shipping a new root.json with them.

Are you worried about a client that has local TUF state and then updates? IIUC that should just work.

[zwass]

Previously, clients could have been distributed with the root key metadata (not the full root.json) and could then bootstrap all trust from that. If those clients update to the new API and try using the old metadata, they will essentially have their updates bricked (because the new API cannot initialize/update with the old metadata and it would be impossible to push an update to use the locally stored full root.json).

I tried to take this into account in https://github.com/fleetdm/fleet/pull/7666/files#diff-c92239951d5af44bbf2ae6686800530d93668c48d38718d3f5289f721b506344R109-R126 by always trying to use the full root.json first and only if that fails falling back to the bootstrap metadata. Does that seem like an appropriate approach to you? We will need to to some substantial testing to ensure that we aren't bricking updates for any of our clients.

[znewman01]

I think I see what you're saying.

I tried to take this into account in https://github.com/fleetdm/fleet/pull/7666/files#diff-c92239951d5af44bbf2ae6686800530d93668c48d38718d3f5289f721b506344R109-R126 by always trying to use the full root.json first and only if that fails falling back to the bootstrap metadata. Does that seem like an appropriate approach to you?

Yes, that seems appropriate to me. There are four cases I can think of:

1. New client, never had the old-style root key metadata. No problem.
2. Old client, shipped with the old-style root key metadata but has synced with the repo and now has a full root.json. The new client will work seamlessly.
3. Old client, never updated, has the old-style root key metadata at version n-1. Upgrade to new client, which ships with full root.json at version n. Falling back here does the right thing.
4. Old client, never updated, has the old-style root key metadata at version n. Upgrade to new client, which ships with full root.json at version n-1. Falling back is not the optimal thing, but I assume the embedded root metadata is pretty easy to bump every time the root keys change. And it will still work, which is the more important aspect.

This seems okay to me in all cases.

We will need to to some substantial testing to ensure that we aren't bricking updates for any of our clients.

Yes, that's important 😄 Let us know if we can help. We're happy to backport security fixes to the 0.3.X branch (which has both Init() and InitLocal()) if needed and maintain that for a little while.

[trishankatdatadog]

@zwass sorry for the late reply, but also, I can't say I understand all the details. Could you please explain a bit concretely why clients upgraded to the new library with the new API might fail? Is it because they are given the root keys/metadata by something else (e.g., a configuration service)?

[znewman01]

Ping, @zwass: anything we need to do on the go-tuf end here, or can I close out?

[asraa]

Hey @zwass : is there anything we can do to help?

We're a little confused how it's possible to update client without updating them to be able to ingest the whole repo root keys (or even extract the root keys from the root metadata if they're relying on the old API that takes the root keys)

[znewman01]

Closing for now; comment if there's anything to do here and we can reopen.

[zwass]

Sorry folks. @lucasmrod pointed out to me that we do actually initialize the whole repository metadata before shipping packages, so the change was really just needed to use the different metadata at initialization.

As long as the documentation and metadata returned by the commands are consistent I think we are good.