Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Support results output as in-toto statement #4491

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

puerco
Copy link
Member

@puerco puerco commented Jan 16, 2025

What kind of change does this PR introduce?

Feature

What is the current behavior?

Currently scorecard does not support generating in-toto statements.

What is the new behavior (if this is a feature change)?**

This PR adds support to generate the scorecard output in an in-toto statement. It introduces a new possible value for the format flag: --format=statement.

The predicate format is open to discussion, right now is just a reformated version of JSONScorecardResultV2 with the repo removed (as it is now in the statement's subject section):

scorecard --format=statement --checks=Maintained --repo=github.com/protobom/protobom   | jq
{
  "type": "https://in-toto.io/Statement/v1",
  "subject": [
    {
      "name": "github.com/protobom/protobom",
      "uri": "git+https://github.com/protobom/protobom@314008b063c88a2ac684bbda95c454bf0f55bdf0",
      "digest": {
        "gitCommit": "314008b063c88a2ac684bbda95c454bf0f55bdf0"
      }
    }
  ],
  "predicate_type": "https://scorecard.dev/result/v0.1",
  "predicate": {
    "date": "2025-01-17T13:27:13-06:00",
    "scorecard": {
      "version": "devel",
      "commit": "unknown"
    },
    "score": 10.0,
    "checks": [
      {
        "details": null,
        "score": 10,
        "reason": "30 commit(s) and 3 issue activity found in the last 90 days -- score normalized to 10",
        "name": "Maintained",
        "documentation": {
          "url": "https://github.com/ossf/scorecard/blob/main/docs/checks.md#maintained",
          "short": "Determines if the project is \"actively maintained\"."
        }
      }
    ],
    "metadata": null
  }
}
  • Tests for the changes have been added (for bug fixes/features)

Which issue(s) this PR fixes

Fixes #3352

Special notes for your reviewer

Let me know if the predicate format should change (or if fields should be added or removed).

/cc @adityasaky as the original author of #3352
/cc @marcelamelara @mlieberman85

Does this PR introduce a user-facing change?

Scorecard can now generate its output as an in-toto statement by specifying --format=statement 

@puerco puerco requested a review from a team as a code owner January 16, 2025 06:37
@puerco puerco requested review from justaugustus and raghavkaul and removed request for a team January 16, 2025 06:37
@puerco puerco temporarily deployed to integration-test January 16, 2025 06:37 — with GitHub Actions Inactive
Copy link

codecov bot commented Jan 16, 2025

Codecov Report

Attention: Patch coverage is 73.43750% with 17 lines in your changes missing coverage. Please review.

Project coverage is 68.47%. Comparing base (353ed60) to head (05f6ed4).
Report is 100 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4491      +/-   ##
==========================================
+ Coverage   66.80%   68.47%   +1.67%     
==========================================
  Files         230      247      +17     
  Lines       16602    18502    +1900     
==========================================
+ Hits        11091    12670    +1579     
- Misses       4808     5004     +196     
- Partials      703      828     +125     

Copy link

@adityasaky adityasaky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is neat, @puerco!

pkg/scorecard/statement.go Outdated Show resolved Hide resolved
func (r *Result) AsStatement(writer io.Writer, checkDocs docs.Doc, opt *AsJSON2ResultOption) error {
// Attestatio subject
subject := intoto.ResourceDescriptor{
Name: r.Repo.Name,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this set the URI field by any chance?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, it now populates the URi with an spdx-style downlaod locator (check the updated example above)

pkg/scorecard/statement.go Outdated Show resolved Hide resolved
@puerco
Copy link
Member Author

puerco commented Jan 16, 2025

OK, I've updated the patch with changes addressing the statement suggestions (and updated the example in the PR body)

pkg/scorecard/json.go Show resolved Hide resolved
Comment on lines +91 to +92
// FormatStatement specifies that results should be output in an in-toto statement.
FormatStatement = "statement"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discussion: any preference on statement vs intoto ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also considered attestation.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think intoto would make it crystal clear what the output format would be.

pkg/scorecard/statement.go Outdated Show resolved Hide resolved
Comment on lines 48 to 49
func (r *Result) AsStatement(writer io.Writer, checkDocs docs.Doc, opt *AsJSON2ResultOption) error {
// Attestatio subject
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we envision any configuration we'd want to do independent of the JSON where we should we define an AsStatementOption (or AsIntotoOption depending on other comment)?

(also: typo in Attestation)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. Right now, there is nothing to change, but I think we'll iterate the predicate or add other types, so specifying the predicate type will probably be the first option. I've wrapped the JSON2 options in a new AsStatement options set (which is empty for now).

pkg/scorecard/statement.go Outdated Show resolved Hide resolved
pkg/scorecard/statement.go Outdated Show resolved Hide resolved
Signed-off-by: Adolfo Garcia Veytia (puerco) <[email protected]>
Signed-off-by: Adolfo Garcia Veytia (puerco) <[email protected]>
Signed-off-by: Adolfo Garcia Veytia (puerco) <[email protected]>
Signed-off-by: Adolfo Garcia Veytia (puerco) <[email protected]>
Signed-off-by: Adolfo Garcia Veytia (puerco) <[email protected]>
This adds an options set for the intoto statement output.
It just wraps the json2 options for now.

Signed-off-by: Adolfo García Veytia (Puerco) <[email protected]>
@puerco puerco force-pushed the in-toto-statement branch from 60aac30 to 05f6ed4 Compare January 17, 2025 19:25
@puerco puerco deployed to integration-test January 17, 2025 19:25 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Output Scorecard results as in-toto attestation
4 participants