Feature: Give projects extra credits for "going the extra mile"

[pnacht]

Is your feature request related to a problem? Please describe.
There are some existing checks and feature requests which are a huge lift for many/most projects.

For example:

• Existing check: Requiring 2 reviewers for 10/10 in Branch Protection, which is very rare in open-source and simply impossible for nebraska-dev projects.
• Feature request: Requiring signed commits (New Check: signed commits #379, New check: signed commits #779, New Check: Signed Commits #1938), which is effectively equivalent to "we don't accept external contributions"

Likewise, other checks are more controversial:

• CII-Best Practices (Feature: Remove CII-Best-Practices check #2904)
• Pinned-Dependencies ([Feedback] Unclear value proposition for hash-pinning dependencies and keeping them up-to-date with dependency update tools #3549), especially in unprivileged testing workflows (Feature: Allow unpinned in non-privileged workflows #2018)

This is part of the reason for the "disconnect" between what a Scorecard score means and what it feels like (#2466) – a 7.0/10 feels like a C-, but is actually an A/A+ (top 7% of relevant projects, top 0.2% of all projects).

Describe the solution you'd like
Projects can earn "extra credit" for taking these "security-paranoid" steps.

This could happen in two ways:

• These settings can be separated into their own checks (i.e. Branch-Protection-Extra), which score either -1 (ignored) or 10. Therefore, projects that don't do these things can still get a 10/10 score if they do everything else, but they can also take these extra steps to dilute the impact of other checks they haven't done.
• Projects could get an 11/10 score for Branch-Protection, for example. However, this could mean projects get a final score above 10. I personally don't mind that scenario, even if just for the PR aspect of it: "this project is so secure, its score goes to 11." Alternatively, we could just clamp the score to 10.

Either solution would lead to an overall increase in projects' Scorecard scores, which would help to close the gap between a score's meaning and sensation.

[spencerschrock]

I'm going to start of by saying scoring is the most opinionated part of Scorecard, and it's impossible to score things in a way that satisfies everyone. Which is part of the reason for structured/results/probes, so we can frame things differently for different audiences. I know we've talked about having different "policies" for Single-Maintainer, , Security Conscious, and I think that also helps at what you're getting at. But Scorecard isn't there yet.

• Existing check: Requiring 2 reviewers for 10/10 in Branch Protection, which is very rare in open-source and simply impossible for nebraska-dev projects.

If the difference is between a 9 and a 10, I think that difference is small enough it already counts as "extra credit".

Projects can earn "extra credit" for taking these "security-paranoid" steps.

I'm not sure I'd classify pinned dependencies as security-paranoid.

This is part of the reason for the "disconnect" between what a Scorecard score means and what it feels like (#2466) – a 7.0/10 feels like a C-, but is actually an A/A+ (top 7% of relevant projects, top 0.2% of all projects).
Either solution would lead to an overall increase in projects' Scorecard scores, which would help to close the gap between a score's meaning and sensation.

I think this is combining two distinct things. Just because a repo is in a high percentile, doesn't mean there aren't improvements to be made.

[github-actions]

This issue has been marked stale because it has been open for 60 days with no activity.

[github-actions]

This issue has been marked stale because it has been open for 60 days with no activity.