-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: Test performance updates #466
Merged
Merged
+346
−100
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
consolidate configs in tox.ini update test markers and tox selection docker/workflow int test update
moved arc connection check into action, and updated int test workflows to use the new action
vilit1
reviewed
Dec 20, 2024
vilit1
reviewed
Dec 20, 2024
vilit1
reviewed
Dec 20, 2024
vilit1
approved these changes
Dec 20, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
update https://github.com/Azure/azure-iot-ops-cli-extension/blob/dev/docs/integration-tests.md if needed and also post some example runs
otherwise looks good enough to me
digimaun
approved these changes
Dec 20, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Major changes:
rpsaas
ande2e
to further define test groupings for pytestpytest-xdist
High-level summary of workflow file changes:
int_test.yml
AIO_SP_*
andCA_*
)container_int_test.yml
addedpython-e2e-int
tox envcreate-k3s-cluster/action.yml
action added - can be used in other workflows to quickly start a k3s cluster and setup kubeconfigpublish_test_container_image.yml
modified to usecontainer_int_test.yml
instead of fullint_test.yml
Dockerfile
updated to runpython-e2e-int
as well as add parallelization to run tests more quickly.Code changes:
tox.ini
[testenv:python-{init,e2e,rpsaas,edge,all}-int]
setenv
you can see the pytest marker inputs for each scenario:init: SCENARIO="init_scenario_test"
e2e: SCENARIO="e2e"
rpsaas: SCENARIO="rpsaas"
edge: SCENARIO="not rpsaas and not init_scenario_test"
all: SCENARIO="not init_scenario_test"
pytest-randomly
dependency to integration tests to fix test distribution / randomization issue withpytest-xdist
azext_edge/tests/generators.py
generate_names
to use insecure random strings (no need for cryptographically "strong" resource names), allowingpytest-randomly
to assist in seeding randomization per test run (allowing us to parallelize pytest withpytest-xdist
)check
andsupport
int tests withpytest.mark.e2e
to break them out to a separate tox env to run in our containerized test environment.pytest.mark.rpsaas
to break them out to a separate tox envThis project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.
Thank you for contributing to Azure IoT Operations tooling!
This checklist is used to make sure that common guidelines for a pull request are followed.
General Guidelines
Intent for Production
dev
ormain
are of production grade. Corollary to this, any merged contributions to these branches may be deployed in a public release at any given time. By checking this box, you agree and commit to the expected production quality of code.Basic expectations
pytest <project root> -vv
. Please provide evidence in the form of a screenshot showing a succesful run of tests locally OR a link to a test pipeline that has been run against the change-set..pylintrc
and.flake8
rules? Look at the CI scripts for example usage.Azure IoT Operations CLI maintainers reserve the right to enforce any of the outlined expectations.
A PR is considered ready for review when all basic expectations have been met (or do not apply).