Cannot use alternative simulation runner with sanitycheck

[abrodkin]

So now when we have finally mdb runner implemented (see #24806) it's possible to use it for instantiation of nSIM-driven simulation instead of previously used arc-nsim runner. It's not that it solves any problem or is really required but given we will use it by default with MetaWare toolchain (once it gets merged) IMHO it's good to have is as generic as possible and if so have a way to use it with nsim_* plaforms.

And while it works perfectly fine for a single app or test this way:

west build -b nsim_hs samples/hello_world/
west flash -r mdb

The seem to be no simple way to run the sanitycheck script with mdb runner.

The first thing that comes to my mind is:

./scripts/sanitycheck --west-runner=mdb --platform nsim_em --test samples/hello_world/sample.basic.helloworld

But it says ERROR - west-runner requires west-flash to be enabled. Ok let's add --west-flash:

./scripts/sanitycheck --west-runner=mdb --west-flash=mdb --platform nsim_em --test samples/hello_world/sample.basic.helloworld

now ERROR - west-flash requires device-testing to be enabled. And only when we add --device-testing it starts to work:

 ./scripts/sanitycheck --west-runner=mdb --west-flash=mdb --device-testing --platform nsim_em --test samples/hello_world/sample.basic.helloworld
INFO    - JOBS: 48
INFO    - Building initial testcase list...
INFO    - 1 test configurations selected, 0 configurations discarded due to filters.

Device testing on:

| Platform   | ID   | Serial device   |
|------------|------|-----------------|

INFO    - Adding tasks to the queue...
INFO    - Total complete:    1/   1  100%  skipped:    0, failed:    0
INFO    - 1 of 1 tests passed (100.00%), 0 failed, 0 skipped with 0 warnings in 22.13 seconds
INFO    - In total 1 test cases were executed on 1 out of total 253 platforms (0.40%)
INFO    - 1 tests executed on platforms, 0 tests were only built.

Hardware distribution summary:

| Board   | ID   | Counter   |
|---------|------|-----------|

But now we sometime see funny messages about target device being busy:

DEBUG   - run test: nsim_hs_smp/tests/kernel/workq/work_queue_api/kernel.workqueue.api
DEBUG   - Waiting for device nsim_hs_smp to become available
DEBUG   - Waiting for device nsim_hs_smp to become available
DEBUG   - Waiting for device nsim_hs_smp to become available

which obviously makes no sense as we're talking about simulated target which we may have as many instances as we'd like.

What's even funnier if I try the same style execution of the sanitycheck but with our default arc-nsim in both --west-runner && --west-flash it will behave in the same way asking for --device-testing.

Probably I'm doing something wrong or do I?

[abrodkin]

@galak, @nashif any hints on how to use an alternative simulation runner?
BTW I guess soon it will become even more interesting as binaries for qemu_arc_{em|hs} will soon be compatible with nsim_{em|hs} (and vice versa) and so we'll be able to use mdb, arc-nsim & qemu runners with nsim_{em|hs} as well as with qemu_arc_{em|hs} equally well.

[nashif]

The main issue here is that we use flash with nsim to run a simulator, this is not something we do for example with qemu. Try to "flash" qemu, it is not supported.
nsim uses a different handler in sanitycheck than the one used for real devices, it just call a binary and does not use west directly, instead it uses 'make run' which calls west based on what was configured for the board.

i have an idea how to improve this, will submit a PR soon.

[evgeniy-paltsev]

With all of my fixes (#29130) to MDB runner it is working now.