specification has new version

[github-actions]

It seems specification (https://github.com/theupdateframework/specification/blob/master/tuf-spec.md) has new version.
Please review the version.

[jku]

There's two changes that are relevant https://github.com/theupdateframework/specification/compare/v1.0.29..v1.0.30:

1. Spec now says WRT client root update:

In case they [versions] are equal, again discard the new root metadata, but
proceed the update cycle with the already trusted root metadata.

I'm not sure if we use or discard the new metadata in this case: the spec advice seems reasonable. The only problem I can imagine with it is that now two clients might have two different versions even if they are updated at the same time (one had an earlier published root of this version already, the other one did not have this version yet). Still seems to make sense, we should check the implementation.

2. Spec now says WRT client timestamp update:

In case they [versions] are equal, discard the new
timestamp metadata and abort the update cycle.
This is normal and it shouldn't raise any error

We likely do this already as it's the logical thing to do but should check.

[jku]

Actually I'm also pretty sure we only accept specific root versions, anything else would be silly (since we download specific versions, surely the version in json must match or it's an error). We never try to download a version equal to current version so there's no non-error way to end up with root metadata with equal versions...

So likely nothing needs to be done

[lukpueh]

Actually I'm also pretty sure we only accept specific root versions, anything else would be silly (since we download specific versions, surely the version in json must match or it's an error). We never try to download a version equal to current version so there's no non-error way to end up with root metadata with equal versions...

I confirm this. The initial root updating routine from Updater.refresh starts with trusted version + 1 (see first line)...

python-tuf/tuf/ngclient/updater.py

Lines 306 to 317 in 46979bb

	# Update the root role
	lower_bound = self._trusted_set.root.signed.version + 1
	upper_bound = lower_bound + self.config.max_root_rotations
	for next_version in range(lower_bound, upper_bound):
	try:
	data = self._download_metadata(
	Root.type,
	self.config.root_max_length,
	next_version,
	)
	self._trusted_set.update_root(data)

... and in TrustedMetadataSet.update_root (see last line above), we fail if new version is not trusted version + 1:

python-tuf/tuf/ngclient/_internal/trusted_metadata_set.py

Lines 166 to 167 in 46979bb

	if new_root.signed.version != self.root.signed.version + 1:
	raise exceptions.BadVersionNumberError(

[lukpueh]

2. Spec now says WRT client timestamp update:

In case they [versions] are equal, discard the new
timestamp metadata and abort the update cycle.
This is normal and it shouldn't raise any error

We likely do this already as it's the logical thing to do but should check.

This is not what ngclient does (Updater.refresh->Updater._load_timestamp->TrustedMetadataSet.update_timestamp):

python-tuf/tuf/ngclient/_internal/trusted_metadata_set.py

Lines 222 to 242 in 46979bb

	if self.timestamp is not None:
	# Prevent rolling back timestamp version
	if new_timestamp.signed.version < self.timestamp.signed.version:
	raise exceptions.BadVersionNumberError(
	f"New timestamp version {new_timestamp.signed.version} must"
	f" be >= {self.timestamp.signed.version}"
	)
	# Prevent rolling back snapshot version
	snapshot_meta = self.timestamp.signed.snapshot_meta
	new_snapshot_meta = new_timestamp.signed.snapshot_meta
	if new_snapshot_meta.version < snapshot_meta.version:
	raise exceptions.BadVersionNumberError(
	f"New snapshot version must be >= {snapshot_meta.version}"
	f", got version {new_snapshot_meta.version}"
	)
	# expiry not checked to allow old timestamp to be used for rollback
	# protection of new timestamp: expiry is checked in update_snapshot()
	self._trusted_set[Timestamp.type] = new_timestamp
	logger.info("Updated timestamp v%d", new_timestamp.signed.version)

We only fail if new timestamp version is less than trusted timestamp version (L224). Otherwise, e.g. if versions are equal, we continue the update cycle and the new timestamp (with equal version) becomes the trusted timestamp (L241)

[jku]

oh interesting!

• I think the "continue the update cycle" is the correct thing to do, maybe the new spec is a bit badly worded: we likely have e.g. snapshot.json available locally already but we should still check, aka continue the update cycle.
• but using the local (old) timestamp seems fine to me and we should do that -- it's good the spec makes it clear that repositories can't publish the same version twice and clients should not accept it

[lukpueh]

Re-reading the detailed client workflow, I'm somehow missing clear instructions for the case where no new (valid) metadata exists on the repo, but trusted metadata is available on the client. In my understanding the client will proceed until the point where it downloads the target, but that does not seem to be spelled out anywhere.

[lukpueh]

To me the main action items from this discussion here are:

• Update ngclient to "continue update cycle" with old trusted timestamp if versions of old trusted and newly downloaded timestamp are equal.
• Clarify in spec what "continuing or aborting the update cycle" means for the updater's ability to download a target. Most notably, reword the seemingly contradictory line: "[...] abort the update cycle. This is normal and it shouldn't raise any error."

[lukpueh]

Created two tickets for above items:

• ngclient: used trusted timestamp if new has equal version  #2023 (blocks this issue)
• Clarify client workflow when no new metadata is available  specification#235

[lukpueh]

#2023 was merged. Shall we bump ...

python-tuf/tuf/api/metadata.py

Line 78 in 46979bb

SPECIFICATION_VERSION = ["1", "0", "29"]

?

[MVrachev]

It seems there is no other immediate blocker.
We confirmed that for root we are doing what the spec says.
I will propose a pr bumping this now.