Need for a level/requirement option

[laurentsimon]

Do we need to start thinking of a SLSA level flag during verification?

This could encourage users to use our tool for verification, even when the provenance has lower levels, for example.

Possible caveat here is that we'd have to add all sorts of self-made systems... maybe there are minimum requirements we could ask for...?

We also have an idea of supporting cosign OPA policies, which may be an easier way forward to support various builders

Thoughts welcome

[asraa]

Dupe of #21, I'll close that one.

We also have an idea of supporting cosign OPA policies, which may be an easier way forward to support various builders

Yeah, for v2 let's support policies, and check in some policies that can verify SLSA 3 with our builders.

For a generic --level, I'm trying to figure out:

• Is it easy to figure out if a build is isolated, except with specific cases we know are true? e.g. with a trusted re-useable workflow (is it true that ANY reuseable workflow satisfies isolation? sort of, dependent on whether it's maintainer controlled, maybe)
• For ephemerality, I think we can check for a few cases, e.g. GCB, github hosted runners, etc, so that might be easy

[laurentsimon]

Here are additional thoughts. Since we're going to release the generic provenance generator soon, we're going to need to be able to differentiate between the types of builders and the requirements they meet:

Things I've been thinking about:

--requirements flag

This would be a new flag that mirrors the https://slsa.dev/spec/v0.1/requirements requirements. It would need to be flexible enough. There are 4 classes of requirements: Source, Build, Provenance, Common.

Below is one option to expose this to users:
*:3 -> all requirements should satisfy the level 3
Build:3 -> Build requirement must satisfy level 3
*:3, Source:0 -> everything is at least level 3 except Source which is disabled
Build,Source:3 -> Build and Source >= level 3
*:3,Build:AsCode:2,Source:0 -> everything is at least level 3 except Source which is disabled and Build.AsCode set to level 2
*:3,Build:AsCode,Reproducible:2,Common:0 -> everything is at least level 3 except Common which is disabled and Build.AsCode/Reproducible set to level 2
*:3,Build:2,Source:0 ...

Versioning

The current spec is at v0.1, so we're implicitly assuming the requirements are matched against their definition. What o we do when the spec is updated? If some of these requirement change buckets, it would break users. Do we want to support older specs over time?
My initial though on this is that we should only support the latest specs, since it's going to supersede the previous one. If we ever change our mind, we could add a -specs-version for users. Another option would be to have multiple --v0.1--requirements, etc but this clobbers the CLI

Defaults

Should there be a default for the requirements (and if so what should this default be)? The majority of users won't know about SLSA levels and requirements, and the Provenance requirement is likely going to be confusing for users. So shall we have a default requirement of Provenance:3? This would allow the majority of users to run the CLI without deep knowledge of SLSA, yet let more advanced users make use of the --requirements option if they wish.

If we decide to support SLSA2 provenance, it would have to be explicit for users to lower the requirement, which is a good thing.

In cases where we the provenance only attest to a single category like Source, we would expect the users to add the Source to its --requirements list.

The last question around default is: would the --requirements option always assume the Provenance:3 (if that's the default), or should users be required to add it anyway? I'm temped to say it can be assumed, unless overwritten by users.

Dynamic builder requirement

There may be cases where the builder can generate SLSA N provenance, but the final provenance only satisfy N-1. One example is the Complete field: although a builder may be able to set it to true, in many cases it will be set to false. So we need to dynamically adjust the results based on provenance content, not just the builder's identity. I don't think this is a problem in practice.

@ianlewis @joshuagl @asraa thoughts?

[asraa]

This would be a new flag that mirrors the https://slsa.dev/spec/v0.1/requirements requirements. It would need to be flexible enough.

Having a single flag with sub-options is pretty cool, it leaves the UX clean.

I'm curious about the feasibility of achieving this, if we can. Are all of the options possible to check with a verifier based on the builder identity and extra information we can check at verification? I went through a thought exercise, and I think most are determined by the builder and the source information. If we cannot determine something, e.g. 2P review, how do we expose that particular options or levels are NOT supported? Logging?

My initial though on this is that we should only support the latest specs, since it's going to supersede the previous one

I think I agree. Assume they have fixed long-lived provenance, update their verifier, and now don't meet SLSA 3 anymore because requirements changed. This should be their behavior.

I strongly dislike --v0.1--requirements, FWIW.

The last question around default is: would the --requirements option always assume the Provenance:3 (if that's the default), or should users be required to add it anyway? I'm temped to say it can be assumed, unless overwritten by users.

Should there be any default at all? These are all additional "policy" checks on the attestation. If there's nothing specified, perhaps all this does is verify the signature and basic options. Some users may not require or need level verification. The options you have are able to express policies that aren't even on a strict level (e.g. pick and choose properties from each level), but I think the defaults should be 0.

[laurentsimon]

This would be a new flag that mirrors the https://slsa.dev/spec/v0.1/requirements requirements. It would need to be flexible enough.

Having a single flag with sub-options is pretty cool, it leaves the UX clean.

I'm curious about the feasibility of achieving this, if we can. Are all of the options possible to check with a verifier based on the builder identity and extra information we can check at verification? I went through a thought exercise, and I think most are determined by the builder and the source information. If we cannot determine something, e.g. 2P review, how do we expose that particular options or levels are NOT supported? Logging?

I would say documentation (.md and CLI --help) + error at verification time: if we detect that your requirements use Common, we'll say it's not supported and exit.

Wdut?

My initial though on this is that we should only support the latest specs, since it's going to supersede the previous one

I think I agree. Assume they have fixed long-lived provenance, update their verifier, and now don't meet SLSA 3 anymore because requirements changed. This should be their behavior.

I strongly dislike --v0.1--requirements, FWIW.

+1

The last question around default is: would the --requirements option always assume the Provenance:3 (if that's the default), or should users be required to add it anyway? I'm temped to say it can be assumed, unless overwritten by users.

Should there be any default at all? These are all additional "policy" checks on the attestation. If there's nothing specified, perhaps all this does is verify the signature and basic options. Some users may not require or need level verification. The options you have are able to express policies that aren't even on a strict level (e.g. pick and choose properties from each level), but I think the defaults should be 0.

The default would mostly be to avoid complicating users's journey by always typing --requirements Provenance:3 which is odd since... the tool is supposed to be verifying provenance. On the other hand, no default makes everything explicit, which is a good thing.

But again, I don't know if we want a default or not and I wanted to hear from everyone. You seem to be voting against. Thanks!

[laurentsimon]

/cc @MarkLodato

[laurentsimon]

One possible complication is that some builders may improve over time: e.g., GCB is SLSA2, then a year later it's SLSA3.
We may need to use time of build in this case. Ideally each builder should have a different ID for each level they comply with. (Probably should push for this)

[laurentsimon]

@var-sdk @hepwori any thoughts?

[MarkLodato]

The newly formed SLSA "tooling" workstream should answer this. One of the workstream's major goals is to decide on a policy model for SLSA. In @wietse-gmail and @var-sdk's draft proposal, there could be multiple components:

• The "integrity" policy describing what sources it is expected to come from, set by the producer.
• The "trust" policy describing what builders are trusted to be at what levels, set by the consumer/verifier.
• The "minimum" policy describing the minimum level that is acceptable (TBD who sets this?)

[laurentsimon]

Thanks. @wietse-gmail and @var-sdk, lets have a chat. Where's the draft proposal?

[wietse-gmail]

The proposal has been shared internally with laurentsimon.

By my understanding, SLSA defines a single ladder. That is, an artifact can satisfy SLSA level N only if it satisfies (at least) all requirements for that level with respect to source, build, and so on. An artifact that scores as (3, 3, 2, 3) would still be a level-2 artirfact.

[laurentsimon]

Thanks for sharing, @wietse-gmail . It'd be great to discuss more granular options. A lot of projects will only satisfy the "provenance" requirement to start with (because they don't want to change their build), and we'd like to be able to capture this. This is just one example. Another example would be a super sensitive project that wants to satisfy 2LGTM, but does not care about "Retained indefinitely".

[wietse-gmail]

I will not deny that custom ladders can have a place in certain contexts, but SLSA is targeting cooperation on a global scale. In the 10 years that the predecessor to SLSA evolved internally, we have learned that a single ladder with N levels helps more to make progress, than 4 ladders with N**4 (or even more) combinations.

[inferno-chromium]

I will not deny that custom ladders can have a place in certain contexts, but SLSA is targeting cooperation on a global scale. In the 10 years that the predecessor to SLSA evolved internally, we have learned that a single ladder with N levels helps more to make progress, than 4 ladders with N**4 (or even more) combinations.

I think this needs broader discussion with the community (lets add that in next community meeting). Internal usecases can be different from what the open source community needs. I agree that simplicity is better, but lets get concensus with community and steering group on what usecases needs seperate ladders. Also, @laurentsimon - did anyone in the community asking for this seperation or is this coming from usecases you are thinking about ?

[laurentsimon]

/cc @rbehjati @tiziano88

[laurentsimon]

Also, @laurentsimon - did anyone in the community asking for this seperation or is this coming from usecases you are thinking about ?

Some examples:

• In this repo, we have a generic generator that satisfies "Provenance" at level 3, but not "Build" at level 3. We also have a Go builder that verifies both. We would like to be able to distinguish between the two, i.e. give a user the ability to do so. "Provenance" requirement alone tells you the "origin" of a project, without asking users to change their build pipeline, which helps with adoption, even though it makes no claim about how the build was made. But given "Provenance" at level 3, a human can read the code; and it's a it's a great security improvement.
• Shopify says hermetic build is too much of an ask and would not consider SLSA level 4 for this reason. Yet, they may want to have 2LGTM or some other level 4 requirements. It's fine no to satisfy level 4, but it'd be great for them to be able to express these requirements.
• [I will let Oak team (cc'ed) comment for their use case]

In the 10 years that the predecessor to SLSA evolved internally, we have learned that a single ladder with N levels helps more to make progress, than 4 ladders with N**4 (or even more) combinations.

Maybe there's some confusion. We're not asking for N**4 combination of ladders / levels. 4 Levels is enough. What we're pointing out is that there are reasons for users to want to verify provenance with more granularity than the levels, and we would like to discuss how to express these. All that's really needed to do this (which goes beyond the scope of this issue) would be the ability for a builder at level N to invalidate claims at level <= N. Something like a requirements field mirroring the requirements from the slsa.dev would work just fine. Below is an example of this field generated by a level 4 builder for an artifact that does not satisfy all of the level 4 requirements:

"requirements":{
  "Source": {
    "TwoPersonReviewed": false,
    "RetainedIndefinitely": false,
  },
  "Build": {
    "Hermetic": false,
    "Reproducible": false,
  },
  "Provenance": {
    "DependenciesComplete": false,
  },
  
}

You could then have ready-made templates with this list of requirement to express the official SLSA levels, and someone may create their own template if they wish too (possibly with a base template as starting point, like a base image in Dockerfiles).

In this issue, we were discussing how to express these via a CLI.

Hope this helps.

[laurentsimon]

/cc @TomHennen

[wietse-gmail]

I think that you're describing a policy for SLSA level N, with selected SLSA requirements turned off.

For the purpose of global interoperability, would it be possible to say that your policy requires SLSA level M, where M = N-1, plus additional requirements that go beyond level M?

If an artifact is shared externally, as in the case of a public package repository, then with "SLSA level N minus..." it will be harder for a receiver to determine what an artifact's assurances are (i.e. whether the policy disables a requirement that the receiver should care about), whereas the approach with "SLSA level M plus..." provides a known assurance threshold M.

If an artifact is shared only internally, then either approach would work, because the receiver has inside knowledge.

[wietse-gmail]

We appear to agree that each SLSA level implies a number of baseline requirements that won't need to be enumerated in a policy (such as individual builder or source repo requirements). This is something that I care about for usability.

Then, the idea is to enumerate in the policy specific (builder, repo, etc.) requirements that differ from the implied baseline requirements. Whether those differences should be additive, subtractive, or both (I hope not), that's something to consider carefully. Is this the direction being proposed?

[laurentsimon]

I think that you're describing a policy for SLSA level N, with selected SLSA requirements turned off.

I did not intend to describe a policy, just the content of the provenance itself. Since there is no way for the builder to express the current level of the final provenance, I just removed the requirements that were not met. If there was a field to say "Level X + requirements", that'd work too. At first sight, I don't think expressing the requirements as "Builder level - requirements" in the provenance is incompatible with expressing a policy "Level X + requirements" for a consumer. Another option is to simply explicitly list all the requirements in the provenance, and let the verifier map them to levels internally.

For the purpose of global interoperability, would it be possible to say that your policy requires SLSA level M, where M = N-1, plus additional requirements that go beyond level M?

That should work. Our generic generator would have a policy (assuming "Common" is 0): "LEVEL 0 + Source:2 + Provenance 3"; our Go builder would have "LEVEL 0 + Source 2 + Build 3 + Build.Hermeticity + Build.Parameterless + Provenance 3". Using the suggestion in #84 (comment), we'd express this in a CLI via --requirements "Source:2, Provenance:3" and --requirements "Source:2, Build:3, Build:Hermeticity,Parameterless:4, Provenance:3" respectively.

If an artifact is shared only internally, then either approach would work, because the receiver has inside knowledge.

So long as there is a builder profile, a verifier can inspect the highest level possible and inspect the provenance. An external party needs to trust the builder to trust the provenance, so they must always have this knowledge.

What do you mean inside knowledge?

Btw, shall we create an issue on the main SLSA repository? Seems like a more appropriate place to discuss this.

[laurentsimon]

We appear to agree that each SLSA level implies a number of baseline requirements that won't need to be enumerated in a policy (such as individual builder or source repo requirements). This is something that I care about for usability.

+1, the levels provide a shortcut to a list of requirements.

Then, the idea is to enumerate in the policy specific (builder, repo, etc.) requirements that differ from the implied baseline requirements. Whether those differences should be additive, subtractive, or both (I hope not), that's something to consider carefully. Is this the direction being proposed?

SGTM. And maybe also think about whether the provenance content and the policy should be treated as two different entities or not. I treated them as separate entities, you seemed to be treating the content as a policy provided by the builder. There may be pros and cons to each approach.

Thanks again for the discussion and your patience

[wietse-gmail]

SGTM. And maybe also think about whether the provenance content and the policy should be treated as two different entities or not. I treated them as separate entities, you seemed to be treating the content as a policy provided by the builder. There may be pros and cons to each approach.

They serve different purposes.

• Provenance describes how an instance of package FOO was built (what source repository, baseline differences, etc.).

• Policy for project FOO describes how instances of package FOO must be built (expected source repository, expected baseline differences).

[laurentsimon]

Sorry, I meant in terms of "Level N - requirement" vs "Level N + requirements". They are distinct entities for sure.

[rbehjati]

[I will let Oak team (cc'ed) comment for their use case]

Sorry for the late reply. I am not sure if this is still relevant! We have considered the possibility of verifying provenances by rebuilding the binaries. In particular, if the builds are reproducible, and the provenance gives a complete description of the build materials, in theory it should be possible to verify the provenance by rebuilding the binary. It is not always desirable, as running the build might be resource- or compute-expensive or take too long. I believe a verifiable provenance like this should be considered conformant with SLSA Level 4 for provenances, but I am not entirely sure, as the description of "Provenance Authenticated" states that "...This SHOULD be through a digital signature from a private key accessible only to the service generating the provenance". In any case, we are replacing verification through rebuilding the binary with having signed provenances. This would satisfy the "Provenance Authenticated" requirement.

The main consumer of the provenances in our case is an internal tool for the transparent release of Oak binaries. The tool verifies provenances and based on that generates a new type of in-toto statement that we call an endorsement statement. But so far, conformance to SLSA levels has not been a focus of this verification. Is the goal of the slsa-verifer to also verify conformance to SLSA levels for build and source? That would be interesting.

[wietse-gmail]

Is the goal of the slsa-verifer to also verify conformance to SLSA levels for build and source? That would be interesting.

That matches what I had in mind. The policy states SLSA requirements, either explicitly (dependencies complete, reproducible build), or implied by a policy SLSA level (SLSA-1 requires that a provenance attestation exists).

An example of this is may be found in a recent presentation.