Debug.Assert cause test host process to crash #2309
Comments
|
It is possible to plugin a custom trace listener to wrap the debug failure. I don't think we can (or should) give you the pop up back. Instead we should translate the assert failure to an exception that will fail the test and will get reported the normal way. Even more, because we have access to the stack trace we can look if there is source code available and even print the actual line of code that failed, imho this is very useful when you are looking at the logs (but not sure if it will be safe to do to make it to the official solution): The below implementation will throw also on Debug.WriteLine. see #2335 for proper solution if you want to create your own listener. Test method UnitTestProject1.UnitTest1.FailedAssert threw exception:
UnitTestProject1.MyTraceListener+DebugAssertException: Debug failed at:
Debug.Assert(thisShouldBe10 == 10); <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Stack Trace:
at UnitTestProject1.MyTraceListener.WriteLine(String message) in /mnt/c/Projects/temp/ConsoleApp5/UnitTestProject1/UnitTest1.cs:line 47
at System.Diagnostics.TraceListener.Fail(String message, String detailMessage)
at System.Diagnostics.TraceInternal.Fail(String message, String detailMessage)
at System.Diagnostics.TraceInternal.TraceProvider.Fail(String message, String detailMessage)
at System.Diagnostics.Debug.Assert(Boolean condition)
at ConsoleApp5.Class1.MyMethod2() in /mnt/c/Projects/temp/ConsoleApp5/ConsoleApp5/Program.cs:line 38
at ConsoleApp5.Class1.MyMethod1() in /mnt/c/Projects/temp/ConsoleApp5/ConsoleApp5/Program.cs:line 31
... shortenedHere is quick prototype code for inspiration. There are no guarantees but it should be easy to plugin into your solution before we fix this, because it does not need any help from the test framework. 😊 using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;
using System.Diagnostics;
using System.IO;
using System.Reflection;
namespace UnitTestProject1
{
[TestClass]
public class UnitTest1
{
[ClassInitialize]
public static void SetupListeners(TestContext c)
{
Trace.Listeners.RemoveAt(0);
Trace.Listeners.Add(new MyTraceListener());
}
[TestMethod()]
public void FailedAssert()
{
var class1 = new Class1();
class1.MyMethod1();
}
[TestMethod()]
public void FailedFail()
{
var class1 = new Class1();
class1.MyMethod3();
}
}
class MyTraceListener : TraceListener
{
public override void Write(string message)
{
var stack = new StackTrace(true);
throw new DebugAssertException(GetReport(message, stack), stack);
}
public override void WriteLine(string message)
{
var stack = new StackTrace(true);
throw new DebugAssertException(GetReport(message, stack), stack);
}
private string GetReport(string message, StackTrace stack)
{
var frames = stack.GetFrames();
string report = null;
MethodBase debugMethod = null;
StackFrame frame = null;
foreach (var f in frames)
{
if (debugMethod != null)
{
if (f.HasMethod())
{
frame = f;
break;
}
}
if (f.HasMethod())
{
var m = f.GetMethod();
if (m.DeclaringType == typeof(Debug))
{
debugMethod = m;
}
}
}
if (frame != null)
{
if (frame.HasSource())
{
var fileName = frame.GetFileName();
var lineNumber = frame.GetFileLineNumber();
string line = null;
using (var sr = new StreamReader(fileName))
{
for (int i = 1; i < lineNumber; i++)
sr.ReadLine();
line = sr.ReadLine();
}
var filteredMessage = message == "Fail:" ? null : message;
report += Environment.NewLine;
report += Environment.NewLine;
if (filteredMessage != null)
{
report += filteredMessage;
}
if (line != null)
{
report = string.Join(Environment.NewLine, "Debug failed at:", line.Trim());
}
}
}
return report;
}
internal class DebugAssertException : Exception
{
public DebugAssertException(string message, StackTrace stack) : base(message ?? "Debug.Assert failed.")
{
StackTrace = stack.ToString();
}
public override string StackTrace { get; }
}
}
public class Class1
{
public void MyMethod1()
{
MyMethod2();
}
public void MyMethod2()
{
var thisShouldBe10 = 20;
Debug.Assert(thisShouldBe10 == 10);
}
public void MyMethod3()
{
Debug.Fail("this fails");
}
}
} |
|
Hi Jakub, I originally stumbled into this while working on a PR for https://github.com/dotnet/aspnetcore, I will open a ticket there to discuss adding this in the project. |
|
That was quick! Thanks @nohwnd ! |
|
It's available in the myget feed until we do next preview release. https://dotnet.myget.org/feed/vstest/package/nuget/Microsoft.NET.Test.Sdk just used it and it works. (One small gotcha: you need to add the nuget source to your configuration, installing it via console using the command in the link, won't work if you don't have the source in the configuration, because -Source parameter is not honored for core projects.) |
|
@alefranz it's on nuget in pre-release channel now. |
|
cheers! |
FWIW there's a school of thought that debug assertions should immediately terminate the test process, by-design (although ideally in a sane way that allows understanding which test failed). The main issue with translating the failure into an exception, is that it may get handled in various ways by the normal program flow. Since code typically doesn't expect exceptions to actually be raised from assertions, what actually happens when they do tends to be pretty undefined and can destabilize the program state in various ways that are hard to debug. In other cases, if some has a catch swallowing Exception, the assertion is silently ignored (but who does that, right?). This is why a simple, fail-fast mechanism that completely bypasses program flow can make a lot of sense. I mostly use Jetbrains Rider, and the default assertion failure behavior (crash) is handled quite well there (even if not perfectly). |
Description
Debug.Assert cause test host process to crash when tests target .NET Core.
Not sure if this is the right place to report this, but starting from here as there was already an issue here about it: #1022
Steps to reproduce
netcoreapp2.1ornetcoreapp3.0ornetcoreapp3.1Expected behavior
Ideally the test should just fail in my opinion.
Otherwise it should just have the previous behaviour on .NET Framework:
you get Debug.Assert window
Actual behavior
Environment
tested on dotnet SDK 3.1.101 and .NET 5
tested with Microsoft.NET.Test.Sdk between 16.2.0 and 16.5.0-preview-20200116-01
The text was updated successfully, but these errors were encountered: