Run All Tests not working in VS

[mateoatr]

Running all tests from Visual Studio isn't working. NUnit stops with the following message:
An exception occurred while invoking executor 'executor://nunit3testexecutor/': Incorrect format for TestCaseFilter Error: Missing '('. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed. It seems that this is an issue within the communication between NUnit and VS (see https://github.com/nunit/nunit3-vs-adapter.issues/tree/master/Issue691/Issue691). Started failing after 0e38c61.

[marek-safar]

@sbomer thoughts on this?

[sbomer]

I don't think we can do much here other than work around it, either by using dotnet test or by disabling the problematic tests. I can look into which ones would need to be disabled.

[sbomer]

It seems like most of the signatures (any that include '(' or '#' for example) trigger the bug, so I don't think we want to work around it by disabling tests. For now I suggest using dotnet test to run all. It looks like test groups under a project (or individual tests) can still be run from VS.

If nunit/nunit3-vs-adapter#691 isn't fixed soon, I can spend some more time looking for other workarounds, but it would take some understanding of how nunit communicates with VS. It's possible that the testcase data is being filtered using its ToString(), in which case there might be a workaround like defining a different ToString on the test data types.

[marek-safar]

Can this be closed?

[sbomer]

It might be worth trying the workaround suggested in nunit/nunit3-vs-adapter#691 (comment). I'll look into it.