feat: add validation for predicates via cue and rego policy files support #641
Conversation
|
We thought there are two ways to implement it on the cli’s UX First is users runs cosign for each attestation type of their image's existing attestations. This means if an image has both For example: $ cosign verify-attestation -key cosign-keys/cosign.pub -policy=demos/spdx.cue -type spdx devopps/alpine:3.8
$ cosign verify-attestation -key cosign-keys/cosign.pub -policy=demos/slsaprovenance.cue -type slsaprovenance devopps/alpine:3.8Seconds is users run cosign just one time. Here is the example to describe it better. cosign verify-attestation -key cosign-keys/cosign.pub -policy=slsaprovenance=demos/slsaprovenance.cue -policy=spdx=demos/spdx.cue devopps/alpine:3.8We implemented the first way now. WDYT @dlorenc |
developer-guy
force-pushed
the
main
branch
2 times, most recently
from
ac6cf93 to
f1b9414
Compare
|
Just updated to 3a738f1 commit, added some sanity and error checks. |
developer-guy
force-pushed
the
main
branch
3 times, most recently
from
f9e5de3 to
fcf8be9
Compare
developer-guy
changed the title
|
We think we are ready. WDYT @dlorenc |
|
This is so cool and the code looks great! Do you mind if I leave the PR open for a bit to play with the cli, cue and the overall experience? This is really new and exciting for all of the in-toto space! |
| cuejson "cuelang.org/go/encoding/json" | ||
| ) | ||
|
|
||
| func ValidateJSON(jsonBody []byte, entrypoints []string) error { |
There was a problem hiding this comment.
nit: ValidateCueJSON as function name.
There was a problem hiding this comment.
Thanks for the suggestion. The method is already in the cue package and it receives a normal JSON. It would be better to use it like this. WDYT?
cmd/cosign/cli/verify_attestation.go
Outdated
| if len(validationErrors) > 0 { | ||
| fmt.Println("Some errors occurred during the validation:") | ||
| for _, v := range validationErrors { | ||
| _, _ = fmt.Fprintf(os.Stderr, "- %v\n", v) |
There was a problem hiding this comment.
suggestion: use fmt.Printf( instead of Fprintf
There was a problem hiding this comment.
Do you mean should we print out the errors to the Stdout instead of Stderr?
cmd/cosign/cli/verify_attestation.go
Outdated
| @@ -49,6 +75,8 @@ func applyVerifyAttestationFlags(cmd *VerifyAttestationCommand, flagset *flag.Fl | |||
| flagset.BoolVar(&cmd.CheckClaims, "check-claims", true, "whether to check the claims found") | |||
| flagset.StringVar(&cmd.FulcioURL, "fulcio-url", "https://fulcio.sigstore.dev", "[EXPERIMENTAL] address of sigstore PKI server") | |||
| flagset.StringVar(&cmd.RekorURL, "rekor-url", "https://rekor.sigstore.dev", "[EXPERIMENTAL] address of rekor STL server") | |||
| flagset.StringVar(&cmd.PredicateType, "type", "custom", "specify predicate type (default: custom) (slsaprovenance|link|spdx)") | |||
| flagset.Var(&cmd.Policies, "policy", "specify CUE files will be using for validation") | |||
There was a problem hiding this comment.
i suggest to include in the description the statement "comma separeted list of CUE files".
There was a problem hiding this comment.
Right now it's not working like that. You should specify all your Cue files with separate -policy flags.
For example:
cosign verify-attestation -key cosign-keys/cosign.pub -policy=demos/spdx1.cue -policy=demos/spdx2.cue -type spdx devopps/alpine:3.8Is this command should work as you said?
|
Hello @dlorenc, we thought that we could support verifying in-toto attestations by using Rego policies as well like the following:
As you already know, we can use OPA as a go library. So the only thing that we have to do is detecting the extension of whether it is Cue or Rego, then calling the correct implementation. But there is a small workaround here to make the flags consistent with the cue implementation. We don't want to accept queries for Rego; instead, what we expect from users is that they have to write Rego policies in package "cosign" and they have to give "allow" names to rules like the above picture. |
|
@developer-guy I was thinking that it would be nice if I could specify all my policies in a single CUE file and then select the ones I'd like to use for a given attestation. The Both CUE and Rego look to have the concept of modules and packages. It seems that both could be handled similarly, in that Generally, it looks like the code is intended to be that each policy flag holds a separate policy. I think it would be helpful to clarify the ideas of entrypoints from how policies are written. The CUE validation passes all policies together during validation, for each iteration of the loop. I'm not familiar enough with the details yet to know if this is implementing the desired outcomes. How do the policy flags relate to the PredicateType here: |
|
Another thing to consider is required fields. There is an open proposal related to this in CUE: cue-lang/cue#822 However, consider the following example modified from the one in #512
package main
import (
"cuelang.org/go/cue"
"cuelang.org/go/cue/cuecontext"
cuejson "cuelang.org/go/encoding/json"
"io/ioutil"
)
func main() {
ctx := cuecontext.New()
cueBs, _ := ioutil.ReadFile("policy.cue")
v := ctx.CompileString(string(cueBs), []cue.BuildOption{}...)
input := `{
"foo": {
"bar": {
"baz": 10
}
},
"user": {
"email": "[email protected]"
}
}`
err := cuejson.Validate([]byte(input), v)
if err != nil {
panic(err)
}
}
foo: bar: {
baz: <11
taz: int
}
user: role: "dev"This has no output, which is passing the policy checks as I would assume as a user. Should this fail because the user lacks a role at all? |
Actually, this one requires one additional flag to get the expression like "-e" and it is also helpful for the Rego implementation if we decide to support Rego because we need a query like an expression in Cue, so yeah this implementation would be helpful for us, WDYT @dlorenc?
Yeah, definitely we should support loading CUE or Rego policies from directories, thanks for the idea @verdverm. 🤝 If we decide to support Rego files we might use separate flags for these, like "-cue" or "-rego" to avoid this kind of problem like you said extension detection for traversing within the directory
There are various types of attestations provided by the in-toto community and cosign uses them internally. So, we thought that users might want to make a one-to-one relationship between the CUE policy file and the attestation's type to verify them like "-policy slsaprovenance.cue -type slsaprovenance" because we need to know which policy is associated with which type of attestation. Additionally, you can store more than one attestation in the OCI registry for an image, so all of them will be validated with the same CUE policy file |
I think it would be better if @dlorenc takes this because IMHO it is ok, if you specify any field within the CUE file that is not in the attestation body, we should pass the validation but if we can support required fields, that would be good for the security perspective. |
|
Sorry I lost track of this one last week! I'll get some time to play with it this week and get it merged! |
It totally ok, we know you are busy 😋 but actually we are waiting for your thoughts about supporting to validate predicates by using Rego policies also, if you want us to do this, we are so excited to implement it 🥳🙋🏻♂️ |
Sure! |
rego validation would be really useful as well, especially with its high adoption rate at present. |
|
We're not sure if it's good UX to allow to validate both CUE and Rego policies at the same time; we couldn't distinguish which UX is better: # using specific flags; `foo.rego`, takes the precedence and will be executing first
$ cosign verify-attestation -cue foo.cue -rego bar.rego
# passing engine flag
$ cosign verify-attestation -p foo.cue # default `-engine` is cue?
$ cosign verify-attestation -p foo.cue -engine cue # valid
$ cosign verify-attestation -p bar.rego -engine rego # valid
$ cosign verify-attestation -p bar.cue -engine rego # invalid
# directory example
$ cosign verify-attestation -p foo.rego -p bar.cue -p ./dir/to/rego/files/
# multi dir example (Regos and CUEs)
$ cosign verify-attestation -p foo.rego -p bar.cue -p ./dir/to/rego/files/ -p ./dir/to/cue/files/Since we're traversing the |
|
Our changes are lost while migrating from Maybe, we can talk a bit about how to do that again. |
No problem, I can help, and sorry it got lost. Do you have the snip of code that it use to integrate into or is the code lost lost? |
No worries @n3wscott, thank you for such a quick response and your kind words. Here is the code I found on my local copy. Sample Code// Copyright 2021 The Sigstore Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package cli
import (
"context"
"encoding/base64"
"encoding/json"
"flag"
"github.com/in-toto/in-toto-golang/in_toto"
"github.com/sigstore/cosign/pkg/cosign/cue"
"io"
"strings"
"github.com/google/go-containerregistry/pkg/name"
"github.com/peterbourgon/ff/v3/ffcli"
"github.com/pkg/errors"
"github.com/sigstore/cosign/cmd/cosign/cli/fulcio"
"github.com/sigstore/cosign/pkg/cosign"
"github.com/sigstore/cosign/pkg/cosign/pivkey"
"github.com/sigstore/sigstore/pkg/signature"
"github.com/sigstore/sigstore/pkg/signature/dsse"
)
type policies struct {
entrypoints []string
}
func (p *policies) Set(s string) error {
if p.entrypoints == nil {
p.entrypoints = []string{}
}
p.entrypoints = append(p.entrypoints, s)
return nil
}
func (p *policies) String() string {
return strings.Join(p.entrypoints, ",")
}
// VerifyAttestationCommand verifies a signature on a supplied container image
type VerifyAttestationCommand struct {
CheckClaims bool
KeyRef string
Sk bool
Slot string
Output string
FulcioURL string
RekorURL string
PredicateType string
Policies policies
}
func applyVerifyAttestationFlags(cmd *VerifyAttestationCommand, flagset *flag.FlagSet) {
flagset.StringVar(&cmd.KeyRef, "key", "", "path to the public key file, URL, KMS URI or Kubernetes Secret")
flagset.BoolVar(&cmd.Sk, "sk", false, "whether to use a hardware security key")
flagset.StringVar(&cmd.Slot, "slot", "", "security key slot to use for generated key (default: signature) (authentication|signature|card-authentication|key-management)")
flagset.BoolVar(&cmd.CheckClaims, "check-claims", true, "whether to check the claims found")
flagset.StringVar(&cmd.FulcioURL, "fulcio-url", "https://fulcio.sigstore.dev", "[EXPERIMENTAL] address of sigstore PKI server")
flagset.StringVar(&cmd.RekorURL, "rekor-url", "https://rekor.sigstore.dev", "[EXPERIMENTAL] address of rekor STL server")
flagset.StringVar(&cmd.PredicateType, "type", "custom", "specify predicate type (default: custom) (slsaprovenance|link|spdx)")
flagset.Var(&cmd.Policies, "policy", "specify CUE files will be using for validation")
}
// Verify builds and returns an ffcli command
func VerifyAttestation() *ffcli.Command {
cmd := VerifyAttestationCommand{}
flagset := flag.NewFlagSet("cosign verify-attestation", flag.ExitOnError)
applyVerifyAttestationFlags(&cmd, flagset)
return &ffcli.Command{
Name: "verify-attestation",
ShortUsage: "cosign verify-attestation -key <key path>|<key url>|<kms uri> <image uri> [<image uri> ...]",
ShortHelp: "Verify an attestation on the supplied container image",
LongHelp: `Verify an attestation on an image by checking the claims
against the transparency log.
EXAMPLES
# verify cosign attestations on the image
cosign verify-attestation <IMAGE>
# verify multiple images
cosign verify-attestation <IMAGE_1> <IMAGE_2> ...
# additionally verify specified annotations
cosign verify-attestation -a key1=val1 -a key2=val2 <IMAGE>
# (experimental) additionally, verify with the transparency log
COSIGN_EXPERIMENTAL=1 cosign verify-attestation <IMAGE>
# verify image with public key
cosign verify-attestation -key cosign.pub <IMAGE>
# verify image with public key provided by URL
cosign verify-attestation -key https://host.for/<FILE> <IMAGE>
# verify image with public key stored in Google Cloud KMS
cosign verify-attestation -key gcpkms://projects/<PROJECT>/locations/global/keyRings/<KEYRING>/cryptoKeys/<KEY> <IMAGE>
# verify image with public key stored in Hashicorp Vault
cosign verify-attestation -key hashivault:///<KEY> <IMAGE>`,
FlagSet: flagset,
Exec: cmd.Exec,
}
}
// DSSE messages contain the signature and payload in one object, but our interface expects a signature and payload
// This means we need to use one field and ignore the other. The DSSE verifier upstream uses the signature field and ignores
// The message field, but we want the reverse here.
type reverseDSSEVerifier struct {
signature.Verifier
}
func (w *reverseDSSEVerifier) VerifySignature(s io.Reader, m io.Reader, opts ...signature.VerifyOption) error {
return w.Verifier.VerifySignature(m, nil, opts...)
}
// Exec runs the verification command
func (c *VerifyAttestationCommand) Exec(ctx context.Context, args []string) (err error) {
if len(args) == 0 {
return flag.ErrHelp
}
if !oneOf(c.KeyRef, c.Sk) && !EnableExperimental() {
return &KeyParseError{}
}
co := &cosign.CheckOpts{
RegistryClientOpts: DefaultRegistryClientOpts(ctx),
SigTagSuffixOverride: cosign.AttestationTagSuffix,
}
if c.CheckClaims {
co.ClaimVerifier = cosign.IntotoSubjectClaimVerifier
}
if EnableExperimental() {
co.RekorURL = c.RekorURL
co.RootCerts = fulcio.GetRoots()
}
keyRef := c.KeyRef
// Keys are optional!
var pubKey signature.Verifier
if keyRef != "" {
pubKey, err = publicKeyFromKeyRef(ctx, keyRef)
if err != nil {
return errors.Wrap(err, "loading public key")
}
} else if c.Sk {
sk, err := pivkey.GetKeyWithSlot(c.Slot)
if err != nil {
return errors.Wrap(err, "opening piv token")
}
defer sk.Close()
pubKey, err = sk.Verifier()
if err != nil {
return errors.Wrap(err, "initializing piv token verifier")
}
}
co.SigVerifier = &reverseDSSEVerifier{
Verifier: dsse.WrapVerifier(pubKey),
}
for _, imageRef := range args {
ref, err := name.ParseReference(imageRef)
if err != nil {
return err
}
sigRepo, err := TargetRepositoryForImage(ref)
if err != nil {
return err
}
co.SignatureRepo = sigRepo
//TODO: this is really confusing, it's actually a return value for the printed verification below
co.VerifyBundle = false
verified, err := cosign.Verify(ctx, ref, co)
for _, vp := range verified {
var payloadData map[string]interface{}
err := json.Unmarshal(vp.Payload, &payloadData)
if err != nil {
return err
}
if predicateTypeMap[c.PredicateType] != payloadData["payloadType"] {
continue
}
decodedPayload, err := base64.StdEncoding.DecodeString(payloadData["payload"].(string))
if err != nil {
return err
}
switch c.PredicateType {
case predicateCustom:
var cosignStatement in_toto.Statement
if err := json.Unmarshal(decodedPayload, &cosignStatement); err != nil {
return err
}
payload, _ := json.Marshal(cosignStatement.Predicate)
if err := cue.ValidateJSON(payload, c.Policies.entrypoints); err != nil {
return err
}
case predicateLink:
var linkStatement in_toto.LinkStatement
if err := json.Unmarshal(decodedPayload, &linkStatement); err != nil {
return err
}
payload, _ := json.Marshal(linkStatement.Predicate)
if err := cue.ValidateJSON(payload, c.Policies.entrypoints); err != nil {
return err
}
case predicateSlsa:
var slsaProvenanceStatement in_toto.ProvenanceStatement
if err := json.Unmarshal(decodedPayload, &slsaProvenanceStatement); err != nil {
return err
}
payload, _ := json.Marshal(slsaProvenanceStatement.Predicate)
if err := cue.ValidateJSON(payload, c.Policies.entrypoints); err != nil {
return err
}
case predicateSpdx:
var spdxStatement in_toto.SPDXStatement
if err := json.Unmarshal(decodedPayload, &spdxStatement); err != nil {
return err
}
payload, _ := json.Marshal(spdxStatement.Predicate)
if err := cue.ValidateJSON(payload, c.Policies.entrypoints); err != nil {
return err
}
default:
continue
}
}
// The attestations are always JSON, so use the raw "text" mode for outputting them instead of conversion
PrintVerification(imageRef, verified, co, "text")
}
return nil
} |
|
@developer-guy please take a look at this PR developer-guy#1 |
developer-guy
force-pushed
the
main
branch
2 times, most recently
from
d3048ad to
6c526ca
Compare
developer-guy
changed the title
| "github.com/google/go-containerregistry/pkg/name" | ||
| "github.com/in-toto/in-toto-golang/in_toto" | ||
| "github.com/pkg/errors" | ||
| "github.com/sigstore/cosign/pkg/cosign/rego" | ||
| "io" |
There was a problem hiding this comment.
looks like you need to sort the imports
|
I screwed up the rebase, sorry about that! Should be good now :) |
developer-guy
force-pushed
the
main
branch
3 times, most recently
from
1a3ba99 to
9838869
Compare
Signed-off-by: Batuhan Apaydın <[email protected]> Co-authored-by: Erkan Zileli <[email protected]> Co-authored-by: Scott Nichols <[email protected]> Co-authored-by: Furkan Türkal <[email protected]> Co-authored-by: Dan Lorenc <[email protected]> Signed-off-by: Batuhan Apaydın <[email protected]>
Fixes #512