Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat bundle matcher languages harmonization #110

Merged
merged 2 commits into from
Feb 7, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,28 @@

FEATURES:

* template/engine:
* `isodate` time formatter to RFC3389 date format.
* bundle/pipeline:
* Support custom input reader and output writer. [#105](https://github.com/elastic/harp/pull/105)
* bundle/selector:
* support `glob` for package path and secret key matcher. [#110](https://github.com/elastic/harp/pull/110)
* support `rego` policy for `bunde filter` command and `BundlePatch` selector. [#106](https://github.com/elastic/harp/pull/106)
* support `cel` expressions used in `BundleRuleSet` for package matchers in `bundle filter` command and `BundlePatch` selector. [#109](https://github.com/elastic/harp/pull/109)
* sdk/value:
* support `age` encryption as value transformer. [#102](https://github.com/elastic/harp/pull/102)
* support deterministic authenticated encryption value transformers. [#103](https://github.com/elastic/harp/pull/103)
* support additional data for AEAD/DAE transformers. [#104](https://github.com/elastic/harp/pull/104)
* DAE transformers can be initialized using an optional salt to derive different keys from the transformer key. [#104](https://github.com/elastic/harp/pull/104)

DIST

* go: Build with Golang 1.17.6.
* build/ci
* Add SLSA Level 1 - Provenance generation step for binaries.
* Add Snyk as code / dependencies scanner via SARIF.
* Add Trivy dependencies scanner via SARIF.

## 0.2.5

### 2022-01-28
Expand Down
178 changes: 100 additions & 78 deletions api/gen/go/harp/bundle/v1/patch.pb.go

Large diffs are not rendered by default.

6 changes: 6 additions & 0 deletions api/proto/harp/bundle/v1/patch.proto
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,9 @@ message PatchSelectorMatchPath {
// Regex path matching.
// Value can be templatized.
string regex = 2;
// Glob path matching. - https://github.com/gobwas/glob
// Value can be templatized.
string glob = 3;
}

// PatchSelectorMatchPath represents package path matching strategies.
Expand All @@ -105,6 +108,9 @@ message PatchSelectorMatchSecret {
// Regex secret matching.
// Value can be templatized.
string regex = 2;
// Glob path matching. - https://github.com/gobwas/glob
// Value can be templatized.
string glob = 3;
}

// PatchPackagePath represents package path operations.
Expand Down
44 changes: 44 additions & 0 deletions docs/onboarding/3-secret-bundle/4-patch.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ bundle source without altering the source bundle.
- [Match by regex path](#match-by-regex-path)
- [Match by JMES filter](#match-by-jmes-filter)
- [Match by Rego policy](#match-by-rego-policy)
- [Match by CEL expression](#match-by-cel-expression)
- [Match by secret key](#match-by-secret-key)
- [PatchSelectorMatchPath](#patchselectormatchpath)
- [PatchPackage](#patchpackage)
Expand Down Expand Up @@ -187,6 +188,8 @@ message PatchSelector {
string rego = 3;
// Match a package by secret.
PatchSelectorMatchSecret matchSecret = 4;
// Match a package using CEL expressions.
repeated string cel = 5;
}
```

Expand Down Expand Up @@ -278,6 +281,43 @@ spec:

```

#### Match by CEL expression

```yaml
selector:
cel:
- "p.is_cso_compliant()"
```

Sample use case

```yaml
apiVersion: harp.elastic.co/v1
kind: BundlePatch
meta:
name: "cso-compliance-flagger"
owner: [email protected]
description: "Flag non CSO complaint packages"
spec:
rules:
- selector:
cel:
# No CSO compliant path as package name
- "!p.is_cso_compliant()"
package:
labels:
add:
cso-compliant: false
- selector:
cel:
# CSO compliant path as package name
- "p.is_cso_compliant()"
package:
labels:
add:
cso-compliant: true
```

#### Match by secret key

Strict matcher
Expand Down Expand Up @@ -323,6 +363,7 @@ spec:

* `strict` is used to filter package path when strictly equal to the given value
* `regex` is used to match the package path with the given regular expression
* `glob` is used to match the package path with the given glob expression

```cpp
// PatchSelectorMatchPath represents package path matching strategies.
Expand All @@ -333,6 +374,9 @@ message PatchSelectorMatchPath {
// Regex path matching.
// Value can be templatized.
string regex = 2;
// Glob path matching. - https://github.com/gobwas/glob
// Value can be templatized.
string glob = 3;
}
```

Expand Down
39 changes: 27 additions & 12 deletions docs/onboarding/3-secret-bundle/5-ruleset.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,33 @@
<!--
Copyright 2022 Thibault NORMAND
# RuleSet

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
- [RuleSet](#ruleset)
- [Query language](#query-language)
- [CEL Expressions](#cel-expressions)
- [Package matchers](#package-matchers)
- [Secret context](#secret-context)

http://www.apache.org/licenses/LICENSE-2.0
## Query language
### CEL Expressions

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
#### Package matchers

* `p.match_path(globstring) bool` - Returns true if the package name match the given Glob pattern.
* `p.match_label(globstring) bool` - Returns true if one of the package labels match the given Glob pattern.
* `p.match_annotation(globstring) bool` - Returns true if one of the package annotations match the given Glob pattern.
* `p.match_secret(globstring) bool` - Returns true if the package has a secret with given pattern.
* `p.has_secret(string) bool` - Returns true if the package has a secret with given key.
* `p.has_all_secrets(...string) bool` - Returns true if the package has all secrets with given keys.
* `p.is_cso_compliant() bool` - Returns true is the package name is CSO compliant.

#### Secret context

* `p.secret(string) Secret` - Returns the secret context matching the secret key of the package.
* `p.secret(string).is_required()` - Flag the given secret key as required.
* `p.secret(string).is_base64()` - Flag the given secret value has a valid base64 encoded string.
* `p.secret(string).is_url()` - Flag the given secret value as a valid URL.
* `p.secret(string).is_uuid()` - Flag the given secret value as a valid UUID.
* `p.secret(string).is_email()` - Flag the given secret value as a valid email.
* `p.secret(string).is_json()` - Flag the given secret value as a valid JSON.

---

Expand Down
45 changes: 33 additions & 12 deletions pkg/bundle/patch/executor.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ package patch
import (
"context"
"encoding/json"
"errors"
"fmt"
"regexp"

Expand Down Expand Up @@ -80,7 +81,7 @@ func executeRule(r *bundlev1.PatchRule, p *bundlev1.Package, values map[string]i
return packageUnchanged, nil
}

//nolint:gocyclo // to refactor
//nolint:gocyclo,funlen // to refactor
func compileSelector(s *bundlev1.PatchSelector, values map[string]interface{}) (selector.Specification, error) {
// Check parameters
if s == nil {
Expand All @@ -89,7 +90,8 @@ func compileSelector(s *bundlev1.PatchSelector, values map[string]interface{}) (

// Has matchPath selector
if s.MatchPath != nil {
if s.MatchPath.Strict != "" {
switch {
case s.MatchPath.Strict != "":
// Evaluation with template engine first
value, err := engine.Render(s.MatchPath.Strict, map[string]interface{}{
"Values": values,
Expand All @@ -100,24 +102,30 @@ func compileSelector(s *bundlev1.PatchSelector, values map[string]interface{}) (

// Return specification
return selector.MatchPathStrict(value), nil
}
if s.MatchPath.Regex != "" {
case s.MatchPath.Glob != "":
// Evaluation with template engine first
value, err := engine.Render(s.MatchPath.Regex, map[string]interface{}{
value, err := engine.Render(s.MatchPath.Glob, map[string]interface{}{
"Values": values,
})
if err != nil {
return nil, fmt.Errorf("unable to evaluate template before matchPath build: %w", err)
}

// Compile regexp
re, err := regexp.Compile(value)
// Return specification
return selector.MatchPathGlob(value)
case s.MatchPath.Regex != "":
// Evaluation with template engine first
value, err := engine.Render(s.MatchPath.Regex, map[string]interface{}{
"Values": values,
})
if err != nil {
return nil, fmt.Errorf("unable to compile macthPath regexp `%s`: %w", s.MatchPath.Regex, err)
return nil, fmt.Errorf("unable to evaluate template before matchPath build: %w", err)
}

// Return specification
return selector.MatchPathRegex(re), nil
return selector.MatchPathRegex(value)
default:
return nil, errors.New("no strict, glob or regexp defined for path matcher")
}
}

Expand All @@ -135,7 +143,8 @@ func compileSelector(s *bundlev1.PatchSelector, values map[string]interface{}) (

// Has matchSecret selector
if s.MatchSecret != nil {
if s.MatchSecret.Strict != "" {
switch {
case s.MatchSecret.Strict != "":
// Evaluation with template engine first
value, err := engine.Render(s.MatchSecret.Strict, map[string]interface{}{
"Values": values,
Expand All @@ -146,8 +155,18 @@ func compileSelector(s *bundlev1.PatchSelector, values map[string]interface{}) (

// Return specification
return selector.MatchSecretStrict(value), nil
}
if s.MatchSecret.Regex != "" {
case s.MatchSecret.Glob != "":
// Evaluation with template engine first
value, err := engine.Render(s.MatchSecret.Glob, map[string]interface{}{
"Values": values,
})
if err != nil {
return nil, fmt.Errorf("unable to evaluate template before matchSecret build: %w", err)
}

// Return specification
return selector.MatchSecretGlob(value), nil
case s.MatchSecret.Regex != "":
// Evaluation with template engine first
value, err := engine.Render(s.MatchSecret.Regex, map[string]interface{}{
"Values": values,
Expand All @@ -164,6 +183,8 @@ func compileSelector(s *bundlev1.PatchSelector, values map[string]interface{}) (

// Return specification
return selector.MatchSecretRegex(re), nil
default:
return nil, errors.New("no strict, glob or regexp defined for secret matcher")
}
}

Expand Down
Loading