Enable testing with Test Explorer

[sharwell]

• Revert a previous change which disabled Test Explorer
• Update to a new version of the xUnit.net Visual Studio test runner

Should not be merged prior to confirmation from @davkean that the underlying critical performance issues in Test Explorer have shipped.

[alrz]

A code action for running/debugging tests would be highly appreciated. Is anything similar planned?

[davkean]

@alrz This is an infrastructure change for the Roslyn repo - can you please file a bug if you would like to see a new feature?

[jcouv]

I'm trying to dogfood SBD and this change will help. Are we planning to merge soon?

[sharwell]

@jcouv This PR cannot be merged until we have confirmation that the latest public release of VS won't suffer from the performance-crippling bugs that were still in place as of #22638.

[sharwell]

This will need to be redone after changes to infrastructure.

[jcouv]

@sharwell @mavasani @jaredpar I understand that a number of blocking source-based discovery issues (including some perf ones) exist in 15.5 and were fixed in 15.6 preview 1.
Would it be ok to require Roslyn contributors to use 15.6 preview 1 until 15.6 is released?
That would allow this PR to move forward and dogfooding of SBD on Roslyn without requiring local changes by each dev.

[sharwell]

Would it be ok to require Roslyn contributors to use 15.6 preview 1 until 15.6 is released?

This requirement introduces a substantial barrier to external contributors which I would prefer to avoid.

[jcouv]

@sharwell I disagree that the negative impact is "substantial" or that it outweights the upside of delivering a properly dogfooded and high-quality feature to contributors and customers at large.

I don't think the impact substantial because: (1) the requirement is already 15.5 (since the codebase uses C# 7.2), (2) the new install experience makes side-by-side fairly painless, (3) we've done such dogfooding in the past with little negative impact or feedback, (4) from daily usage of 15.6 preview 1 and informal feedback it seems very usable already, and (5) contributors are already working on 15.6 preview 1 codebase anyways (since it is in master) so 15.6 is preferable for testing in dev hive.

Going back to the importance of Roslyn dogfooding Roslyn features: Realistically, if devs are required to make extensive local changes to dogfood, then dogfooding will simply not happen (and it has not). Without proper and extensive dogfooding, we will end up in the same situation once we start working on 15.7.

I would be open to delay dogfooding until preview 2 is released, although I fear that will be late in the release cycle, as the bug bar goes up significantly past that preview. But delaying dogfooding until 15.6 RTM is not acceptable, as it leaves us in the same position again.
I'll let @jaredpar @mavasani comment on whether that is a workable trade-off.

[sharwell]

@jcouv At last check, merging this (or equivalent) will cripple 15.5. As in wholly unusable even for the most patient of contributors.

[jinujoseph]

@ManishJayaswal

[jcouv]

Thanks Jinu. I included the wrong Manish ;-)
We're having email thread to clarify options and policy.

[alrz]

Requiring preview versions is not new, once new features are integrated, for the best experience you would need to use the preview anyways. I don't think that is/was a barrier for most contributors, actually having this disabled is more of an inconvenience now IMO. Looking forward for this to be enabled. thanks.

[jcouv]

@alrz Thanks for the feedback. With people being away, I'm expecting a resolution on this the first week of January (I set up a meeting for this). Cheers

[alrz]

As a workaround I temporarily added those packages to test projects until this goes in.

https://gist.github.com/alrz/c2c5c4797b80e2128e5dc8c35961367f

[genlu]

Just FYI, I have create a separate PR that includes VS adapter and upgraded packages. Still trying to assess the perf impact with and without source based discovery turned on.
#24163