Test Platform fails to run tests if an adapter supplied does not exist

[AbhitejJohn]

Description

The platform fails to run any test if one of the adapters does not exist.

Steps to reproduce

1. Create an nunit test project with nunit 3.10.1 and nunit3testadapter 3.10.0
2. Ensure that tests are discovered.
3. Remove/rename the test adapter from "packages\NUnit3TestAdapter.3.10.0\build\netcoreapp1.0"
4. Run all the tests in the solution.

Expected behavior

The tests run (using the default net351 version of the nunit adapter.)

Actual behavior

Tests do not run and the following stack is shown:

System.IO.FileNotFoundException: C:\Users\aajohn.REDMOND\source\repos\PlaylistTests\packages\NUnit3TestAdapter.3.10.0\build\netcoreapp1.0\NUnit3.TestAdapter.dll
   at System.Diagnostics.FileVersionInfo.GetVersionInfo(String fileName)
   at Microsoft.VisualStudio.TestPlatform.Utilities.Helpers.FileHelper.GetFileVersion(String path)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DefaultTestHostManager.GetAndLogFileVersion(String path)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DefaultTestHostManager.FilterExtensionsBasedOnVersion(IEnumerable`1 extensions)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DefaultTestHostManager.GetTestPlatformExtensions(IEnumerable`1 sources, IEnumerable`1 extensions)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.InitializeExtensions(IEnumerable`1 sources)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, ITestRunEventsHandler eventHandler)
[3/30/2018 3:14:49 PM Error] System.IO.FileNotFoundException: C:\Users\aajohn.REDMOND\source\repos\PlaylistTests\packages\NUnit3TestAdapter.3.10.0\build\netcoreapp1.0\NUnit3.TestAdapter.dll
   at System.Diagnostics.FileVersionInfo.GetVersionInfo(String fileName)
   at Microsoft.VisualStudio.TestPlatform.Utilities.Helpers.FileHelper.GetFileVersion(String path)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DefaultTestHostManager.GetAndLogFileVersion(String path)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DefaultTestHostManager.FilterExtensionsBasedOnVersion(IEnumerable`1 extensions)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DefaultTestHostManager.GetTestPlatformExtensions(IEnumerable`1 sources, IEnumerable`1 extensions)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.InitializeExtensions(IEnumerable`1 sources)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, ITestRunEventsHandler eventHandler)

Environment

Latest dogfoodable previews of VS 15.7.

[AbhitejJohn]

This is something that we've run into as part of our testing. We do not yet know how the adapter files are being deleted. It also does not help that in a design mode scenario these extensions are initialized once and cached thereafter. So an intermittent issue like so, can cause issues throughout the session.

[ManishJayaswal]

@cltshivash FYI. We have seen this several times now. I also suspect some feedback where we are asking folks to delete the extensions folder might be caused by this. It would be great if this could be fixed in 15.7.

[cltshivash]

@ManishJayaswal @AbhitejJohn Haven't we fixed the code to not use the Extensions cache folder for 15.7 ? Is there some user sceanrio (apart from a manual delete) where we are hitting this ?

[ManishJayaswal]

@cltshivash - yes we have fixed the code to not use the cache in 15.7. So that set of users should not see it in 15.7. However I believe this specific issue is still happening in 15.7 and tests are not getting run.

[AbhitejJohn]

@cltshivash : Post the cache fix there still appears to be something that is causing certain adapter dependencies to disappear. I haven't figured out what yet, since this is pretty rare and is intermittent. Probably I can add selective filelogger logging to get to this, since it does happen quite often with Manish at least. However, when this does happen it would be great to continue on with the test run for other adapters that are found.

[abhishkk]

@AbhitejJohn Closing this issue as changes are checked in master with warning message in case adapter is not found. Please validate the same.
PR: #1578