coverage: When enable coverage, some testcases need more time

[wangnuannuan]

Describe the bug
I am testting coverage on qemu_arc_em and nsim_em with sanitycheck. I found that when enable coverage, some testcases will timeout.

• nsim_em:

tests/lib/heap_align/lib.heap_align FAILED: Timeout
tests/kernel/timer/timer_monotonic/kernel.timer.monotonic FAILED: Timeout
tests/crypto/mbedtls/crypto.mbedtls FAILED: Timeout
tests/kernel/lifo/lifo_usage/kernel.lifo.usage FAILED: Failed

The above 3 timeout failures will pass when use west,. But accoding to the console output, these testcases need more time to finish
For tests/kernel/lifo/lifo_usage/kernel.lifo.usage, if increase timeout (K_MSEC(timeout + 10)) , it will pass. For examples, K_MSEC(100).

zephyr/tests/kernel/lifo/lifo_usage/src/main.c

Line 328 in 3dba8cc

packet = k_lifo_get(&lifo_timeout[0], K_MSEC(timeout + 10));

• qemu_arc_em:

ERROR   - qemu_arc_em               samples/portability/cmsis_rtos_v2/timer_synchronization/sample.portability.cmsis_rtos_v2.timer_synchronization FAILED: Exited with 1
ERROR   - qemu_arc_em               samples/scheduler/metairq_dispatch/sample.kernel.metairq_dispatch FAILED: Exited with 1
ERROR   - qemu_arc_em               samples/portability/cmsis_rtos_v1/timer_synchronization/sample.portability.cmsis_rtos_v1.timer_synchronization FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/portability/cmsis_rtos_v2/portability.cmsis_rtos_v2 FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/subsys/logging/log_immediate/logging.log_immediate FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/subsys/logging/log_immediate/logging.log_immediate.clean_output FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/posix/common/portability.posix.common        FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/posix/common/portability.posix.common.newlib FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/portability/cmsis_rtos_v1/portability.cmsis_rtos_v1 FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/workq/critical/kernel.workqueue       FAILED: Failed
ERROR   - qemu_arc_em               tests/kernel/workq/work_queue/kernel.workqueue     FAILED: Failed
ERROR   - qemu_arc_em               tests/kernel/workq/work_queue_api/kernel.workqueue.api FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/sched/schedule_api/kernel.scheduler.multiq FAILED: Failed
ERROR   - qemu_arc_em               tests/kernel/sleep/kernel.common.timing            FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/sched/schedule_api/kernel.scheduler   FAILED: Failed
ERROR   - qemu_arc_em               tests/kernel/sched/schedule_api/kernel.scheduler.multiq_no_timeslicing FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/sched/schedule_api/kernel.scheduler.no_timeslicing FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/sched/preempt/kernel.scheduler.preempt FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/queue/kernel.queue.poll               FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/queue/kernel.queue                    FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/pipe/pipe/kernel.pipe                 FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/pipe/pipe_api/kernel.pipe.api         FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/mem_pool/mem_pool_threadsafe/kernel.memory_pool.threadsafe FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/mem_pool/mem_pool_concept/kernel.memory_pool.concept.legacy FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/common/kernel.common                  FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/common/kernel.common.misra            FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/context/kernel.common                 FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/fifo/fifo_timeout/kernel.fifo.timeout FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/fifo/fifo_timeout/kernel.fifo.timeout.poll FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/kernel/fifo/fifo_api/kernel.fifo             FAILED: Failed
ERROR   - qemu_arc_em               tests/kernel/fifo/fifo_api/kernel.fifo.poll        FAILED: Failed
ERROR   - qemu_arc_em               tests/kernel/mbox/mbox_usage/kernel.mailbox.usage  FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/crypto/mbedtls/crypto.mbedtls                FAILED: Exited with 1
ERROR   - qemu_arc_em               tests/benchmarks/timing_info/benchmark.kernel.timing.default FAILED: Exited with 1

I tested samples/portability/cmsis_rtos_v2/timer_synchronization with west , if I increase the TIMER_TICKS to 250, it will pass:

zephyr/samples/portability/cmsis_rtos_v2/timer_synchronization/src/main.c

Line 15 in 3dba8cc

#define TIMER_TICKS 50

[nashif]

not sure how the timeout in the code relates to the timeout of running the test and failing due to timeouts in sanitycheck.

[andrewboie]

are any of these tests CPU bound? coverage accounting will add nontrivial overhead.

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.

[nashif]

I think this one was already fixed with recent move to multiprocessing.