Sanitycheck block after passing some test

[amergnat]

Describe the bug
Sanitycheck on a RISC V board seems to stop working after while, more precisely at step 23/155 for this example. After some retry, it seems it isn't bug at a specific test (I've been blocked at 17/155 or 21/155). This sanity is done from master branch (updated October 14 2020)

To Reproduce
Steps to reproduce the behavior:

1. Plug the board hifive1_revb to your computer
2. Setup Zephyr build environment
3. Type the command: "sanitycheck --device-testing --device-serial /dev/ttyACM0 -p hifive1_revb -v"

Expected behavior
Execute all sanity test planned (155)

Impact
That impact the validation of this pull request: #27063
This PR can't use sanity on QEMU instead due to lack of support.

Logs and console output

Full log: sanitycheck.log

Console:

sanitycheck --device-testing --device-serial /dev/ttyACM0 -p hifive1_revb -v
INFO    - JOBS: 8
INFO    - Building initial testcase list...
INFO    - 934 test configurations selected, 779 configurations discarded due to filters.

Device testing on:

| Platform     | ID   | Serial device   |
|--------------|------|-----------------|
| hifive1_revb |      | /dev/ttyACM0    |

INFO    - Adding tasks to the queue...
INFO    -   1/155 hifive1_revb              samples/drivers/entropy/sample.drivers.entropy     SKIPPED (filter)
INFO    -   2/155 hifive1_revb              samples/drivers/led_apa102c_bitbang/sample.drivers.led.apa102c_bitbang SKIPPED (filter)
INFO    -   3/155 hifive1_revb              samples/drivers/led_lp503x/sample.drivers.led.lp503x SKIPPED (filter)
INFO    -   4/155 hifive1_revb              samples/smp/pi/sample.smp.pi                       SKIPPED (filter)
INFO    -   5/155 hifive1_revb              samples/posix/eventfd/sample.posix.eventfd         SKIPPED (RAM overflow)
INFO    -   6/155 hifive1_revb              samples/userspace/prod_consumer/sample.userspace.prod_consumer SKIPPED (filter)
INFO    -   7/155 hifive1_revb              samples/synchronization/sample.kernel.synchronization PASSED (device 6.446s)
INFO    -   8/155 hifive1_revb              samples/application_development/out_of_tree_driver/sample.drivers.out_of_tree PASSED (device 7.965s)
INFO    -   9/155 hifive1_revb              samples/scheduler/metairq_dispatch/sample.kernel.metairq_dispatch SKIPPED (RAM overflow)
INFO    -  10/155 hifive1_revb              samples/philosophers/sample.kernel.philosopher.coop_only PASSED (device 7.817s)
INFO    -  11/155 hifive1_revb              samples/philosophers/sample.kernel.philosopher.static PASSED (device 5.300s)
INFO    -  12/155 hifive1_revb              samples/userspace/shared_mem/sample.kernel.memory_protection.shared_mem SKIPPED (filter)
INFO    -  13/155 hifive1_revb              samples/application_development/external_lib/sample.app_dev.external_lib PASSED (device 7.157s)
INFO    -  14/155 hifive1_revb              samples/philosophers/sample.kernel.philosopher.fifos PASSED (device 7.539s)
INFO    -  15/155 hifive1_revb              samples/hello_world/sample.basic.helloworld        PASSED (device 7.014s)
INFO    -  16/155 hifive1_revb              samples/philosophers/sample.kernel.philosopher.preempt_only PASSED (device 7.356s)
INFO    -  17/155 hifive1_revb              samples/philosophers/sample.kernel.philosopher.semaphores PASSED (device 5.298s)
INFO    -  18/155 hifive1_revb              tests/kernel/fatal/exception/kernel.common.stack_protection_arm_fpu_sharing SKIPPED (filter)
INFO    -  19/155 hifive1_revb              samples/philosophers/sample.kernel.philosopher     PASSED (device 7.206s)
INFO    -  20/155 hifive1_revb              tests/kernel/fatal/exception/kernel.common.stack_protection SKIPPED (filter)
INFO    -  21/155 hifive1_revb              samples/philosophers/sample.kernel.philosopher.lifos PASSED (device 4.926s)
INFO    -  22/155 hifive1_revb              samples/philosophers/sample.kernel.philosopher.stacks PASSED (device 5.235s)
INFO    -  23/155 hifive1_revb              tests/kernel/spinlock/kernel.multiprocessing.spinlock SKIPPED (filter)

Environment (please complete the following information):

• OS: Linux
• Toolchain: Zephyr SDK 0.11.1
• Commit SHA: 2d247cc (HEAD -> master, origin/master)

Additionnal information

I probably do something wrong, let me know.

[abrodkin]

@IRISZZW @evgeniy-paltsev this one might be of interest to you.

[nashif]

I am not able to reproduce on the same board and other boards.