Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sanitycheck --test-only --device-testing --hardware-map shouldn't run tests on all boards from --build-only #29166

Closed
erwango opened this issue Oct 13, 2020 · 0 comments · Fixed by #29724
Closed

sanitycheck --test-only --device-testing --hardware-map shouldn't run tests on all boards from --build-only #29166

erwango opened this issue Oct 13, 2020 · 0 comments · Fixed by #29724
Assignees
@nashif
Labels
area: Sanitycheck Sanitycheck has been renamed to Twister bug The issue is a bug, or the PR is fixing a bug priority: medium Medium impact/importance bug

Comments

@erwango
Copy link
Member

erwango commented Oct 13, 2020 •
edited
Loading

Describe the bug
When using sanitycheck --build-only followed by sanitycheck --test-only --device-testing --hardware-map, sanitycheck tries to run test on all targets given to sanitycheck --build-only.
Though, it could be possible that hardware-map only provides a subset

To Reproduce
Steps to reproduce the behavior:

  1. sanitycheck --build-only -p x -p y -p z
  2. sanitycheck --test-only --device-testing --hardware-map map.yml (which contains only x)
  3. See error: sanitycheck tries to run tests on y and z

Expected behavior
sanitycheck --test-only --device-testing --hardware-map map.yml should only run test on the common subset between --build-only targets and map.yaml

Impact
Prevents use of --test-only combined with --device-testing --hardware-map map.yml in such conditions.
@erwango erwango added bug The issue is a bug, or the PR is fixing a bug area: Sanitycheck Sanitycheck has been renamed to Twister labels Oct 13, 2020
@nashif nashif added priority: low Low impact/importance bug priority: medium Medium impact/importance bug and removed priority: low Low impact/importance bug labels Oct 13, 2020
@nashif nashif mentioned this issue Nov 2, 2020
Merged
nashif added a commit to nashif/zephyr that referenced this issue Nov 18, 2020
@nashif
sanitycheck: do not try platforms not in hardware map
f3e0fd2 
Get list of connected hardware and filter tests to only be run on
connected hardware in cases where no platform was specified when runn
the tests.  If the platform does not exist in the hardware map, just
skip it.

Fixes zephyrproject-rtos#29166

Signed-off-by: Anas Nashif <[email protected]>
nashif added a commit that referenced this issue Nov 27, 2020
@nashif
sanitycheck: do not try platforms not in hardware map
82a6a46 
Get list of connected hardware and filter tests to only be run on
connected hardware in cases where no platform was specified when runn
the tests.  If the platform does not exist in the hardware map, just
skip it.

Fixes #29166

Signed-off-by: Anas Nashif <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nashif nashif

Labels
area: Sanitycheck Sanitycheck has been renamed to Twister bug The issue is a bug, or the PR is fixing a bug priority: medium Medium impact/importance bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

sanitycheck: do not try platforms not in hardware map nashif/zephyr
2 participants
@nashif @erwango