Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch from test-ref-configs.pl to separate components #9565

Conversation

gilles-peskine-arm
Copy link
Contributor

@gilles-peskine-arm gilles-peskine-arm commented Sep 14, 2024

test-ref-configs.pl doesn't do much. It just has a list of things to build and test. It's poor at reporting: if there's any failure, it just stops. This pull requests moves all the testing of reference configurations to separate components in all.sh.

Also, test-ref-configs.pl was doing the testing twice: once the configuration as given and once with PSA enabled, but in the development branch the configurations all have PSA enabled already. Remove this duplication.

This is an alternative to #9564, which improves test-ref-configs.pl. Differences:

  • Splitting into separate components gives us not only better reporting through the list of failed components, but also upload of failure logs from ssl-opt.sh and compat.sh.
  • There's more code to review here, but not much more.
  • Separate components use more CI resources. In the long term this is best handled by having CI scripts do grouping. In the short term, we don't have a performance problem right now, so I think this doesn't matter.

PR checklist

Signed-off-by: Gilles Peskine <[email protected]>
This also suffices for compat.sh.

Include the sample programs in this build. They aren't tested by ssl-opt.sh
yet, but they soon will be.

Signed-off-by: Gilles Peskine <[email protected]>
This also suffices for compat.sh.

Include the sample programs in this build. They aren't tested by ssl-opt.sh
yet, but they soon will be.

Signed-off-by: Gilles Peskine <[email protected]>
@gilles-peskine-arm gilles-peskine-arm added needs-ci Needs to pass CI tests component-test Test framework and CI scripts priority-medium Medium priority - this can be reviewed as time permits size-xs Estimated task size: extra small (a few hours at most) labels Sep 14, 2024
Rename the existing component_test_tfm_config which tests a modified version
of config-tfm.h for the sake of driver-vs-reference comparison.

Signed-off-by: Gilles Peskine <[email protected]>
@gilles-peskine-arm gilles-peskine-arm added needs-review Every commit must be reviewed by at least two team members, needs-reviewer This PR needs someone to pick it up for review and removed needs-ci Needs to pass CI tests labels Sep 14, 2024
mpg
mpg previously approved these changes Sep 17, 2024
Copy link
Contributor

@mpg mpg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@gilles-peskine-arm gilles-peskine-arm added priority-high High priority - will be reviewed soon and removed priority-medium Medium priority - this can be reviewed as time permits labels Sep 19, 2024
@gilles-peskine-arm
Copy link
Contributor Author

Changing to priority-high after a discussion with Ronald because this is convenient for the repository split: it's one less script to adapt.

Remove the components migrated from test-ref-configs.pl that use legacy
crypto (no enabling of MBEDTLS_USE_PSA_CRYPTO). In the 4.0 preparation
branch, we are no longer interested in such configurations.

Signed-off-by: Gilles Peskine <[email protected]>
In the components migrated from test-ref-configs.pl, we don't need to
activate PSA: it's always on. Also, since there is no "_legacy" component to
contrast with, drop "_psa" from the component names.

Signed-off-by: Gilles Peskine <[email protected]>
@gilles-peskine-arm
Copy link
Contributor Author

In the initial version of this pull request (up to "Remove test-ref-configs.pl, which no longer does anything"), I had made the new components replicate exactly what the Perl script was doing (except for component names). But the script was in fact causing us to do useless work: we were running both legacy and PSA variants of the configurations, but the "legacy" variant actually has MBEDTLS_PSA_CRYPTO_C and MBEDTLS_USE_PSA_CRYPTO enabled nowadays, so we were testing the same configuration twice. So I've added commits to drop the duplicate testing: now there is only one component for each configuration.

set(programs_target "${MBEDTLS_TARGET_PREFIX}programs")
add_custom_target(${programs_target})

set(ssl_opt_target "${MBEDTLS_TARGET_PREFIX}ssl-opt")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ssl-opt should also generate tls13-compat.sh which is added by #9563. This can only be done once both parts of the code are in — reminder to make a follow-up PR.

Copy link
Contributor

@mpg mpg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Copy link
Contributor

@eleuzi01 eleuzi01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gilles-peskine-arm gilles-peskine-arm added approved Design and code approved - may be waiting for CI or backports needs-backports Backports are missing or are pending review and approval. and removed needs-review Every commit must be reviewed by at least two team members, needs-reviewer This PR needs someone to pick it up for review labels Sep 23, 2024
@gilles-peskine-arm gilles-peskine-arm added this pull request to the merge queue Sep 24, 2024
Merged via the queue into Mbed-TLS:development with commit 26650f5 Sep 24, 2024
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Design and code approved - may be waiting for CI or backports component-test Test framework and CI scripts needs-backports Backports are missing or are pending review and approval. priority-high High priority - will be reviewed soon size-xs Estimated task size: extra small (a few hours at most)
Development

Successfully merging this pull request may close these issues.

3 participants