Code coverage: crash when instrumenting CoreLib with Coverlet

[pjanotti]

The instrumentation added by coverlet adds a reference to "coverlet.tracker", however, this reference is causing a crash in coreclr when trying to jit the instrumented System.Private.CoreLib. At this stage I'm not sure yet about the best way to handle the issue, for now opening it on CoreFx repo where we collect coverage numbers for System.Private.CoreLib.

Basically when jitting a instrumented method from System.Private.CoreLib (coverage runs by default use the IL version of CoreLib) reference to "coverlet.tracker" causes a null reference exception because the call:

https://github.com/dotnet/coreclr/blob/1f28125ad1f9975fbe68dd6839908aa6e63fc43b/src/vm/coreassemblyspec.cpp#L135-136

return null so later the code hits the null exception at:

https://github.com/dotnet/coreclr/blob/1f28125ad1f9975fbe68dd6839908aa6e63fc43b/src/vm/coreassemblyspec.cpp#L171

In debug builds it a, corresponding, earlier assert checking for pBinder null:

  Assert failure(PID 591480 [0x00090678], Thread: 575160 [0x8c6b8]): pBinder != NULL

  CORECLR! AssemblySpec::Bind + 0x6D9 (0x00007ffb`9e7b3059)
  CORECLR! AppDomain::BindAssemblySpec + 0xCFE (0x00007ffb`9e68d9de)
  CORECLR! PEFile::LoadAssembly + 0x611 (0x00007ffb`9e514581)
  CORECLR! Module::LoadAssembly + 0x776 (0x00007ffb`9e63aa56)
  CORECLR! Assembly::FindModuleByTypeRef + 0xEE4 (0x00007ffb`9e4183a4)
  CORECLR! ClassLoader::LoadTypeDefOrRefThrowing + 0xA06 (0x00007ffb`9e554d86)
  CORECLR! MemberLoader::GetDescFromMemberRef + 0xDB3 (0x00007ffb`9e7d9d63)
  CORECLR! CEEInfo::resolveToken + 0xD24 (0x00007ffb`9e5e5b54)
  CLRJIT! Compiler::impResolveToken + 0xA4 (0x00007ffb`a177cee4)
  CLRJIT! Compiler::impImportBlockCode + 0x9E4D (0x00007ffb`a17643dd)
      File: c:\s\github\pjanotti\coreclr\src\vm\coreassemblyspec.cpp Line: 142
      Image: C:\s\github\pjanotti\corefx\bin\testhost\netcoreapp-Windows_NT-Debug-x64\dotnet.exe

[pjanotti]

Full stack just before the crash:

0:000> k
 # Child-SP          RetAddr           Call Site
00 (Inline Function) --------`-------- coreclr!BaseWrapper<ICLRPrivAssembly *,FunctionBase<ICLRPrivAssembly *,&DoNothing<ICLRPrivAssembly *>,&DoTheRelease<ICLRPrivAssembly>,2>,0,&CompareDefault<ICLRPrivAssembly *>,2>::AddressInitHolder::{ctor} [c:\s\github\pjanotti\coreclr\src\inc\holder.h @ 527] 
01 (Inline Function) --------`-------- coreclr!BaseWrapper<ICLRPrivAssembly *,FunctionBase<ICLRPrivAssembly *,&DoNothing<ICLRPrivAssembly *>,&DoTheRelease<ICLRPrivAssembly>,2>,0,&CompareDefault<ICLRPrivAssembly *>,2>::TypedAddressInitHolder::{ctor} [c:\s\github\pjanotti\coreclr\src\inc\holder.h @ 596] 
02 (Inline Function) --------`-------- coreclr!BaseWrapper<ICLRPrivAssembly *,FunctionBase<ICLRPrivAssembly *,&DoNothing<ICLRPrivAssembly *>,&DoTheRelease<ICLRPrivAssembly>,2>,0,&CompareDefault<ICLRPrivAssembly *>,2>::operator& [c:\s\github\pjanotti\coreclr\src\inc\holder.h @ 633] 
03 0000002c`8a779850 00007ffb`99cd5d1a coreclr!AssemblySpec::Bind+0x15c [c:\s\github\pjanotti\coreclr\src\vm\coreassemblyspec.cpp @ 171] 
04 0000002c`8a77a1d0 00007ffb`99d90983 coreclr!AppDomain::BindAssemblySpec+0x15a [c:\s\github\pjanotti\coreclr\src\vm\appdomain.cpp @ 6948] 
05 0000002c`8a77ac10 00007ffb`99ccc78c coreclr!PEFile::LoadAssembly+0x117 [c:\s\github\pjanotti\coreclr\src\vm\pefile.cpp @ 1640] 
06 0000002c`8a77ad40 00007ffb`99d7b06e coreclr!Module::LoadAssembly+0xa4 [c:\s\github\pjanotti\coreclr\src\vm\ceeload.cpp @ 5611] 
07 0000002c`8a77aea0 00007ffb`99cfa843 coreclr!Assembly::FindModuleByTypeRef+0x21e [c:\s\github\pjanotti\coreclr\src\vm\assembly.cpp @ 1274] 
08 0000002c`8a77af60 00007ffb`99d29dbb coreclr!ClassLoader::LoadTypeDefOrRefThrowing+0x2a3 [c:\s\github\pjanotti\coreclr\src\vm\clsload.cpp @ 3003] 
09 0000002c`8a77b0b0 00007ffb`99d26bbe coreclr!MemberLoader::GetDescFromMemberRef+0x93b [c:\s\github\pjanotti\coreclr\src\vm\memberload.cpp @ 297] 
0a 0000002c`8a77b400 00007ffb`f0045b82 coreclr!CEEInfo::resolveToken+0x31e [c:\s\github\pjanotti\coreclr\src\vm\jitinterface.cpp @ 1126] 
0b 0000002c`8a77b810 00007ffb`f004423d clrjit!Compiler::impImportBlockCode+0x572 [c:\s\github\pjanotti\coreclr\src\jit\importer.cpp @ 13373] 
0c 0000002c`8a77c9a0 00007ffb`f0043fcb clrjit!Compiler::impImportBlock+0x7d [c:\s\github\pjanotti\coreclr\src\jit\importer.cpp @ 16495] 
0d 0000002c`8a77ca90 00007ffb`f000e215 clrjit!Compiler::impImport+0x30b [c:\s\github\pjanotti\coreclr\src\jit\importer.cpp @ 17577] 
0e (Inline Function) --------`-------- clrjit!Compiler::fgImport+0xf [c:\s\github\pjanotti\coreclr\src\jit\flowgraph.cpp @ 6809] 
0f 0000002c`8a77cb00 00007ffb`f000f2d6 clrjit!Compiler::compCompile+0x95 [c:\s\github\pjanotti\coreclr\src\jit\compiler.cpp @ 4484] 
10 0000002c`8a77cc60 00007ffb`f000ef6d clrjit!Compiler::compCompileHelper+0x296 [c:\s\github\pjanotti\coreclr\src\jit\compiler.cpp @ 6085] 
11 0000002c`8a77cd10 00007ffb`effdabbf clrjit!Compiler::compCompile+0x21d [c:\s\github\pjanotti\coreclr\src\jit\compiler.cpp @ 5431] 
12 0000002c`8a77cdc0 00007ffb`f0058072 clrjit!jitNativeCode+0x24f [c:\s\github\pjanotti\coreclr\src\jit\compiler.cpp @ 6713] 
13 0000002c`8a77cf70 00007ffb`99d60d81 clrjit!CILJit::compileMethod+0x92 [c:\s\github\pjanotti\coreclr\src\jit\ee_il_dll.cpp @ 303] 
14 (Inline Function) --------`-------- coreclr!invokeCompileMethodHelper+0x63 [c:\s\github\pjanotti\coreclr\src\vm\jitinterface.cpp @ 12228] 
15 0000002c`8a77cfe0 00007ffb`99d60be1 coreclr!invokeCompileMethod+0xdd [c:\s\github\pjanotti\coreclr\src\vm\jitinterface.cpp @ 12295] 
16 0000002c`8a77d080 00007ffb`99d4ba26 coreclr!CallCompileMethodWithSEHWrapper+0x4d [c:\s\github\pjanotti\coreclr\src\vm\jitinterface.cpp @ 12347] 
17 0000002c`8a77d110 00007ffb`99d4c337 coreclr!UnsafeJitFunction+0x4e6 [c:\s\github\pjanotti\coreclr\src\vm\jitinterface.cpp @ 12844] 
18 0000002c`8a77d6d0 00007ffb`99d4b4aa coreclr!MethodDesc::JitCompileCodeLocked+0x213 [c:\s\github\pjanotti\coreclr\src\vm\prestub.cpp @ 842] 
19 0000002c`8a77d860 00007ffb`99d4b0d8 coreclr!MethodDesc::JitCompileCodeLockedEventWrapper+0xb6 [c:\s\github\pjanotti\coreclr\src\vm\prestub.cpp @ 706] 
1a 0000002c`8a77d950 00007ffb`99d79b45 coreclr!MethodDesc::JitCompileCode+0x378 [c:\s\github\pjanotti\coreclr\src\vm\prestub.cpp @ 646] 
1b (Inline Function) --------`-------- coreclr!MethodDesc::PrepareILBasedCode+0x176 [c:\s\github\pjanotti\coreclr\src\vm\prestub.cpp @ 320] 
1c (Inline Function) --------`-------- coreclr!MethodDesc::PrepareCode+0x176 [c:\s\github\pjanotti\coreclr\src\vm\prestub.cpp @ 302] 
1d (Inline Function) --------`-------- coreclr!MethodDesc::PrepareInitialCode+0x19f [c:\s\github\pjanotti\coreclr\src\vm\prestub.cpp @ 258] 
1e 0000002c`8a77da50 00007ffb`99d79412 coreclr!MethodDesc::DoPrestub+0x5c5 [c:\s\github\pjanotti\coreclr\src\vm\prestub.cpp @ 1830] 
1f 0000002c`8a77dba0 00007ffb`99e05635 coreclr!PreStubWorker+0x382 [c:\s\github\pjanotti\coreclr\src\vm\prestub.cpp @ 1566] 
20 0000002c`8a77ddf0 00007ffb`99e06093 coreclr!ThePreStub+0x55 [C:\s\github\pjanotti\coreclr\src\vm\amd64\ThePreStubAMD64.asm @ 22] 
21 0000002c`8a77dea0 00007ffb`99cf1d93 coreclr!CallDescrWorkerInternal+0x83 [C:\s\github\pjanotti\coreclr\src\vm\amd64\CallDescrWorkerAMD64.asm @ 101] 
22 (Inline Function) --------`-------- coreclr!CallDescrWorkerWithHandler+0x52 [c:\s\github\pjanotti\coreclr\src\vm\callhelpers.cpp @ 78] 
23 0000002c`8a77def0 00007ffb`99dc5825 coreclr!MethodDescCallSite::CallTargetWorker+0x273 [c:\s\github\pjanotti\coreclr\src\vm\callhelpers.cpp @ 628] 
24 (Inline Function) --------`-------- coreclr!MethodDescCallSite::Call+0x43 [c:\s\github\pjanotti\coreclr\src\vm\callhelpers.h @ 467] 
25 0000002c`8a77e040 00007ffb`99dc56d5 coreclr!AppDomain::InitializeDomainContext+0x105 [c:\s\github\pjanotti\coreclr\src\vm\appdomain.cpp @ 7626] 
26 0000002c`8a77e1d0 00007ffb`99dc5afd coreclr!SystemDomain::InitializeDefaultDomain+0x55 [c:\s\github\pjanotti\coreclr\src\vm\appdomain.cpp @ 3194] 
27 0000002c`8a77e250 00007ffb`99dfd9a5 coreclr!SystemDomain::SetupDefaultDomain+0x199 [c:\s\github\pjanotti\coreclr\src\vm\appdomain.cpp @ 3130] 
28 0000002c`8a77e3e0 00007ffb`99dfd94a coreclr!SystemDomain::SetupDefaultDomainNoThrow+0x4d [c:\s\github\pjanotti\coreclr\src\vm\appdomain.cpp @ 3153] 
29 0000002c`8a77e470 00007ffb`99dbee69 coreclr!EEStartup+0x6e [c:\s\github\pjanotti\coreclr\src\vm\ceemain.cpp @ 1230] 
2a 0000002c`8a77e4d0 00007ffb`99db2eb8 coreclr!EnsureEEStarted+0xbd [c:\s\github\pjanotti\coreclr\src\vm\ceemain.cpp @ 367] 
2b (Inline Function) --------`-------- coreclr!InitializeEE+0x1b [c:\s\github\pjanotti\coreclr\src\vm\ceemain.cpp @ 304] 
2c (Inline Function) --------`-------- coreclr!CorRuntimeHostBase::Start+0x24 [c:\s\github\pjanotti\coreclr\src\vm\corhost.cpp @ 226] 
2d 0000002c`8a77e530 00007ffb`99db2af4 coreclr!CorHost2::Start+0x98 [c:\s\github\pjanotti\coreclr\src\vm\corhost.cpp @ 185] 
2e 0000002c`8a77e580 00007ffb`d3f3fa05 coreclr!coreclr_initialize+0x164 [c:\s\github\pjanotti\coreclr\src\dlls\mscoree\unixinterface.cpp @ 216] 
2f 0000002c`8a77e640 00007ffb`d3f404ed hostpolicy!corehost_unload+0x17435
30 0000002c`8a77ebf0 00007ffb`d5e9e148 hostpolicy!corehost_main+0x7d
31 0000002c`8a77ed70 00007ffb`d5ea67aa hostfxr+0x1e148
32 0000002c`8a77ee40 00007ffb`d5ea46c2 hostfxr!hostfxr_resolve_sdk+0x7e5a
33 0000002c`8a77f910 00007ffb`d5ea3506 hostfxr!hostfxr_resolve_sdk+0x5d72
34 0000002c`8a77f9c0 00007ffb`d5e9e915 hostfxr!hostfxr_resolve_sdk+0x4bb6
35 0000002c`8a77fb10 00007ff7`42d99b65 hostfxr!hostfxr_main_startupinfo+0x95
36 0000002c`8a77fc10 00007ff7`42d9e2c9 dotnet+0x9b65
37 0000002c`8a77fda0 00007ffc`127f3034 dotnet+0xe2c9
38 0000002c`8a77fde0 00007ffc`146e1431 KERNEL32!BaseThreadInitThunk+0x14 [base\win32\client\thread.c @ 64] 
39 0000002c`8a77fe10 00000000`00000000 ntdll!RtlUserThreadStart+0x21 [minkernel\ntdll\rtlstrt.c @ 998] 

[hughbe]

Has this been fixed now support had been added to coverlet? Or do we need to update the libraries

[danmoseley]

@ViktorHofer owns code coverage now and may know. It is not obvious that coverlet-coverage/coverlet#209 would fix it.

[tonerdo]

@ViktorHofer @danmosemsft out of curiosity, what extra stuff is needed to finalize coverage support from Coverlet?

[pjanotti]

Hi everyone, what is missing is to deploy latest coverlet.console as a global tool instead of a msbuild task (see dotnet/buildtools#2184). Meanwhile if coverlet.msbuild is updated to latest you should be able to get coverage against CoreLib using coverlet (I did test it before leaving).

@danmosemsft here is the explanation on why the fix works: coverlet-coverage/coverlet#209 (comment)

[pjanotti]

So closing this one...

[danmoseley]

Thank you @pjanotti! Glad to see you again!