[Not for merging] Repro of integration test failure

[Neme12]

For demonstration purposes only (do not merge)

This is a minimum repro to show the integration test failures encountered in #25875 (comment)

@sharwell Can you please take a quick look at this very small repro to see if there's anything I'm doing wrong? If I run both tests in the Example class this error dialog appears:

Test1 and Test2 are identical. Running only one of them does not reproduce this. Changing WellKnownProjectTemplates.CSharpNetCoreClassLibrary to WellKnownProjectTemplates.ClassLibrary fixes the error.

[Neme12]

@jasonmalinowski Can you please take a look?

[jasonmalinowski]

@davkean or @Pilchie do you recognize this or would know where to route?

[Neme12]

It's possible that if this was fixed, a lot of other tests that are currently skipped could be enabled again. When I was browsing through the integration tests to find usages of CSharpNetCoreClassLibrary (to see if the problem is on my side), the majority of tests I found were disabled.

[Neme12]

@jinujoseph Could this please get a buddy? This PR is not for merging but I would appreciate if somebody could take a look at this or at least tell me what the best path forward is, because this is blocking my other PR (#25875). Should I just skip the tests and file an issue? Is it OK to merge skipped tests in a PR?

I don't think this should be hard to investigate (for the right person) because there's a really small repro and the issue repros consistently, even locally. And as I sad earlier it's also possible this issue is affecting other currently skipped tests (because the repro is very small), so it might be worthwhile trying to find where the issue is because it could potentially allow us to unskip other tests.

[sharwell]

@Pilchie @davkean Do you know what the original bug here is? It's the same bug I encountered in my work on the new test harness, resulting in a rather elaborate (and very expensive) workaround in 6c73dfa and 9ec79b4.

[Pilchie]

This is a core CPS issue. Can someone include the file mentioned in the dialog box? @lifengl can you help route?

[Neme12]

@Pilchie @lifengl Here is the file from two different runs (they do seem to be a little different)

VsProjectFault_25556e03-5a24-4658-a745-4f0ca14160a8.failure.txt
VsProjectFault_e752c0d1-4b75-4ed4-9400-5a45a019f510.failure.txt

[Neme12]

Note: This error would be harder to find in CI because both of these tests actually pass. The error dialog appears after the second test is started, and the second test finishes running successfully while the dialog is displayed. But then the dialog prevents the creation of a new project, so the failure would actually show up in a third (potentially completely unrelated) test.

[Neme12]

retest windows_release_vs-integration_prtest please

[Neme12]

retest windows_debug_vs-integration_prtest please

[Neme12]

Hm, the tests are passing in CI. It seems like I'm only getting the dialog locally. Should I close this then?

[lifengl]

Which build of VS are you using when you saw the problem. It was reported as a race condition related to the design time build service before. + Huizhong

…

Sent from my phone On Oct 12, 2018, at 5:05 AM, Šimon Koníček <[email protected]<mailto:[email protected]>> wrote: Hm, the tests are passing in CI. It seems like I'm only getting the dialog locally. Should I close this then? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdotnet%2Froslyn%2Fpull%2F30147%23issuecomment-429302910&data=02%7C01%7C%7C8d2abe00d1734bb19a7d08d6303af2a1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636749427061471134&sdata=Ps1kpwHElnanNIiXPdARrd71RG%2F%2BRP5u5sLC9dkkwEc%3D&reserved=0>, or mute the thread<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALGWwvujWtMmQhxOucUJ9gPKXHlGYFZbks5ukIVwgaJpZM4W7Ibn&data=02%7C01%7C%7C8d2abe00d1734bb19a7d08d6303af2a1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636749427061627392&sdata=qy0HdATogWJL0ehrzWsi9RgZR5UqfM3ls%2FezGGoJS%2Fw%3D&reserved=0>.

[Neme12]

@lifengl I'm using 15.9 Preview 3.

[Neme12]

This now passed but my original PR still keeps failing in CI, I don't know what's going on... 😕 maybe it's because there's more than 2 tests?

[Neme12]

Just to be clear, it's still failing for me locally, it's just that I that noticed CI passed here.

[Neme12]

@lifengl I'm still encountering this in 16.0 preview1. Is there an existing issue or something I can point to as a reason for skipping the tests? Thanks.

[Neme12]

➡️ Workaround for this was added in 675adf0 (PR #32122)