[HIGH IMPORTANCE] dotnet test should return a non-0 exit code when any tests fail

[TheRealPiotrP]

This is important and we'd really appreciate a quick fix.

Repro:

  [Fact]
  public void AlwaysFail()
  {
       throw new Exception("Not a winner...");
  }

dotnet test

$? on OS X, similar 'get last exit code' elsewhere

Expected:
non-0 exit code returned, indicating test failure

Actual:
0 returned, indicating success.

Impact:
CLI Builds cannot be trusted right now. There is no way to detect test failure during build execution on local dev box nor in CI. @MattGertz this is likely associated with the NuGet PR @livarcocc is backing out right now.

@blackdwarf I know you're looking at UX in this space, so FYI.

dotnet --info:

.NET Command Line Tools (1.0.0-preview4-004114)

Product Information:
 Version:            1.0.0-preview4-004114
 Commit SHA-1 hash:  04c5d39c03

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.12
 OS Platform: Darwin
 RID:         osx.10.11-x64
 Base Path:   /Users/piotrp/code/github/piotrpMSFT/cli/artifacts/osx.10.11-x64/stage2/sdk/1.0.0-preview4-004114

[TheRealPiotrP]

Additional repro info: I am using zsh as my console. Will see if this repros out of plain terminal

[TheRealPiotrP]

ACK on terminal repro. Also attaching a proper repro zip.
microsoft.vstest.issue.241.repro.zip

[codito]

It looks like vstest.console is returning non-zero on test failure and vstest task is always returning true.

Will confirm shortly.

[sbaid]

@Faizan2304

[codito]

Fixed with 592a213

[TheRealPiotrP]

Wonderful! Thanks!

What build # should we pick up?

[Faizan2304]

Fix is in this PR dotnet/cli#4810