Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test failure Loader\\binding\\tracing\\BinderTracingTest.ResolutionFlow\\BinderTracingTest.ResolutionFlow.cmd #84813

Closed
v-wenyuxu opened this issue Apr 14, 2023 · 2 comments · Fixed by #86180
Closed

Test failure Loader\\binding\\tracing\\BinderTracingTest.ResolutionFlow\\BinderTracingTest.ResolutionFlow.cmd #84813

v-wenyuxu opened this issue Apr 14, 2023 · 2 comments · Fixed by #86180
Assignees
@davmason
Labels
arch-x64 area-Tracing-coreclr blocking-clean-ci-optional Blocking optional rolling runs JitStress CLR JIT issues involving JIT internal stress modes os-windows
Milestone
8.0.0

Comments

@v-wenyuxu
Copy link

Run: runtime-coreclr jitstress 20230412.3

Failed test:

coreclr windows x64 Checked zapdisable @ Windows.10.Amd64.Open

- Loader\\binding\\tracing\\BinderTracingTest.ResolutionFlow\\BinderTracingTest.ResolutionFlow.cmd

Error message:

Assert failure(PID 9264 [0x00002430], Thread: 9256 [0x2428]): Consistency check failed: AV in clr at this callstack:
CORECLR! dn_list_insert + 0xAB (0x00007fffe04a214b) CORECLR! buffer_manager_allocate_buffer_for_thread + 0x511 (0x00007fffe0488361)
CORECLR! ep_buffer_manager_write_event + 0x3D2 (0x00007fffe048e0e2) CORECLR! ep_session_write_event + 0x218 (0x00007fffe0496938)
CORECLR! write_event_2 + 0x1F5 (0x00007fffe049a155) CORECLR! ep_write_event + 0x92 (0x00007fffe0497da2)
CORECLR! EventPipeAdapter::WriteEvent + 0xD7 (0x00007fffe047c447) CORECLR! EventPipeWriteEventGCFinalizersBegin_V1 + 0x6D (0x00007fffe046e5cd)
CORECLR! FinalizerThread::FinalizeAllObjects + 0x35 (0x00007fffdff3c1d5) CORECLR! FinalizerThread::FinalizerThreadWorker + 0x884 (0x00007fffdff3d3d4)
CORECLR! ManagedThreadBase_DispatchInner + 0xB4 (0x00007fffdfe2d194) CORECLR! ManagedThreadBase_DispatchMiddle + 0xB6 (0x00007fffdfe2d296)
CORECLR! ``ManagedThreadBase_DispatchOuter'::11'::__Body::Run'::5'::__Body::Run + 0x7B (0x00007fffdfe2fa2b) CORECLR! ManagedThreadBase_DispatchOuter'::11'::__Body::Run + 0x93 (0x00007fffdfe2fd23)
CORECLR! ManagedThreadBase_DispatchOuter + 0xDD (0x00007fffdfe2d3ad) CORECLR! ManagedThreadBase_NoADTransition + 0xCB (0x00007fffdfe2d5db)
CORECLR! FinalizerThread::FinalizerThreadStart + 0x19C (0x00007fffdff3c87c) KERNEL32! BaseThreadInitThunk + 0x14 (0x00007ff80ff784d4)
NTDLL! RtlUserThreadStart + 0x21 (0x00007ff8`101b1791)
.AV on tid=0x2428 (9256), cxr=000000B0A167DB00, exr=000000B0A167DFF0
FAILED: false

CORECLR! CHECK::Trigger + 0x1E6 (0x00007fffe0312286) CORECLR! CLRVectoredExceptionHandlerPhase3 + 0x4B5 (0x00007fffdfc87145)
CORECLR! CLRVectoredExceptionHandlerPhase2 + 0x91 (0x00007fffdfc86a21) CORECLR! CLRVectoredExceptionHandler + 0xD36 (0x00007fffdfc86956)
CORECLR! CLRVectoredExceptionHandlerShim + 0x1B5 (0x00007fffdfc873c5) NTDLL! RtlInitializeCriticalSection + 0x190 (0x00007ff8101c6b40)
NTDLL! RtlImageNtHeaderEx + 0x17B (0x00007ff8101946cb) NTDLL! KiUserExceptionDispatcher + 0x3A (0x00007ff81020989a)
CORECLR! dn_list_insert + 0xAB (0x00007fffe04a214b) CORECLR! buffer_manager_allocate_buffer_for_thread + 0x511 (0x00007fffe0488361)
File: D:\a_work\1\s\src\coreclr\vm\excep.cpp Line: 7116
Image: C:\h\w\AE2709A7\p\corerun.exe

Return code:      1
Raw output file:      C:\h\w\AE2709A7\w\B20C0987\uploads\Reports\Loader.binding\tracing\BinderTracingTest.ResolutionFlow\BinderTracingTest.ResolutionFlow.output.txt
Raw output:
BEGIN EXECUTION
"C:\h\w\AE2709A7\p\corerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  BinderTracingTest.ResolutionFlow.dll
[5:51:53 AM] Running AssemblyLoadContextResolving_ReturnNull...
[5:51:57 AM] Running AssemblyLoadContextResolving_LoadAssembly...
[5:51:58 AM] Running AssemblyLoadContextResolving_NameMismatch...
[5:51:58 AM] Running AssemblyLoadContextResolving_MultipleHandlers...
[5:51:58 AM] Running AppDomainAssemblyResolve_ReturnNull...
[5:51:58 AM] Running AppDomainAssemblyResolve_LoadAssembly...
[5:51:58 AM] Running AppDomainAssemblyResolve_NameMismatch...
[5:51:58 AM] Running AppDomainAssemblyResolve_MultipleHandlers...
[5:51:58 AM] Launching process for AssemblyLoadFromResolveHandler_LoadDependency...
[5:52:00 AM] Running AssemblyLoadFromResolveHandler_LoadDependency...

[5:52:03 AM] Launching process for AssemblyLoadFromResolveHandler_NotTracked...
[5:52:04 AM] Running AssemblyLoadFromResolveHandler_NotTracked...

[5:52:08 AM] Launching process for FindInLoadContext_CustomALC...
[5:52:09 AM] Running FindInLoadContext_CustomALC...

[5:52:12 AM] Launching process for FindInLoadContext_DefaultALC...
[5:52:13 AM] Running FindInLoadContext_DefaultALC...

[5:52:17 AM] Launching process for FindInLoadContext_DefaultALC_IncompatibleVersion...
[5:52:18 AM] Running FindIn


Stack trace
   at Loader_binding._tracing_BinderTracingTest_ResolutionFlow_BinderTracingTest_ResolutionFlow_._tracing_BinderTracingTest_ResolutionFlow_BinderTracingTest_ResolutionFlow_cmd()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
@v-wenyuxu v-wenyuxu added os-windows JitStress CLR JIT issues involving JIT internal stress modes arch-x64 labels Apr 14, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Apr 14, 2023
@dotnet-issue-labeler dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Apr 14, 2023
@vcsjones vcsjones removed the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Apr 16, 2023
@tommcdon tommcdon removed the untriaged New issue has not been triaged by the area owner label Apr 17, 2023
@tommcdon tommcdon added this to the 8.0.0 milestone Apr 17, 2023
@BruceForstall
Copy link
Member

Another failure, in gcstress

https://dev.azure.com/dnceng-public/public/_build/results?buildId=267863&view=ms.vss-test-web.build-test-results-tab

coreclr windows x64 Checked gcstress0xc_zapdisable_heapverify1

set DOTNET_TieredCompilation=0
set DOTNET_GCStress=0xC
set DOTNET_HeapVerify=1
set DOTNET_ReadyToRun=0
set DOTNET_ZapDisable=1

    Loader\binding\tracing\BinderTracingTest.ResolutionFlow\BinderTracingTest.ResolutionFlow.cmd [FAIL]
      
      Assert failure(PID 5216 [0x00001460], Thread: 4072 [0x0fe8]): Consistency check failed: AV in clr at this callstack:
      ------
      CORECLR! dn_list_insert + 0xAB (0x00007ffa`671da10b)
      CORECLR! buffer_manager_allocate_buffer_for_thread + 0x511 (0x00007ffa`671c0341)
      CORECLR! ep_buffer_manager_write_event + 0x3D2 (0x00007ffa`671c60c2)
      CORECLR! ep_session_write_event + 0x218 (0x00007ffa`671ce918)
      CORECLR! write_event_2 + 0x1F5 (0x00007ffa`671d2135)
      CORECLR! ep_write_event + 0x92 (0x00007ffa`671cfd82)
      CORECLR! EventPipeAdapter::WriteEvent + 0xD7 (0x00007ffa`671b4427)
      CORECLR! EventPipeWriteEventGCFinalizersBegin_V1 + 0x6D (0x00007ffa`671a65ad)
      CORECLR! FinalizerThread::FinalizeAllObjects + 0x35 (0x00007ffa`66c6e9c5)
      CORECLR! FinalizerThread::FinalizerThreadWorker + 0x884 (0x00007ffa`66c6fbc4)
      CORECLR! ManagedThreadBase_DispatchInner + 0xB4 (0x00007ffa`66b60784)
      CORECLR! ManagedThreadBase_DispatchMiddle + 0xB6 (0x00007ffa`66b60886)
      CORECLR! ``ManagedThreadBase_DispatchOuter'::`11'::__Body::Run'::`5'::__Body::Run + 0x7B (0x00007ffa`66b6301b)
      CORECLR! `ManagedThreadBase_DispatchOuter'::`11'::__Body::Run + 0x93 (0x00007ffa`66b63313)
      CORECLR! ManagedThreadBase_DispatchOuter + 0xDD (0x00007ffa`66b6099d)
      CORECLR! ManagedThreadBase_NoADTransition + 0xCB (0x00007ffa`66b60bcb)
      CORECLR! FinalizerThread::FinalizerThreadStart + 0x19C (0x00007ffa`66c6f06c)
      KERNEL32! BaseThreadInitThunk + 0x14 (0x00007ffa`98a284d4)
      NTDLL! RtlUserThreadStart + 0x21 (0x00007ffa`98cd1791)
      -----
      .AV on tid=0xfe8 (4072), cxr=0000009D71F7DCC0, exr=0000009D71F7E1B0
      FAILED: false
      
      CORECLR! CHECK::Trigger + 0x1E6 (0x00007ffa`67049796)
      CORECLR! CLRVectoredExceptionHandlerPhase3 + 0x4B5 (0x00007ffa`669b93b5)
      CORECLR! CLRVectoredExceptionHandlerPhase2 + 0x91 (0x00007ffa`669b8c91)
      CORECLR! CLRVectoredExceptionHandler + 0xD36 (0x00007ffa`669b8bc6)
      CORECLR! CLRVectoredExceptionHandlerShim + 0x1B5 (0x00007ffa`669b9635)
      NTDLL! RtlInitializeCriticalSection + 0x190 (0x00007ffa`98ce6b40)
      NTDLL! RtlImageNtHeaderEx + 0x17B (0x00007ffa`98cb46cb)
      NTDLL! KiUserExceptionDispatcher + 0x3A (0x00007ffa`98d2989a)
      CORECLR! dn_list_insert + 0xAB (0x00007ffa`671da10b)
      CORECLR! buffer_manager_allocate_buffer_for_thread + 0x511 (0x00007ffa`671c0341)
          File: D:\a\_work\1\s\src\coreclr\vm\excep.cpp Line: 7115
          Image: C:\h\w\9EA008B3\p\corerun.exe
      
      
      Return code:      1
      Raw output file:      C:\h\w\9EA008B3\w\ADFD0960\uploads\Reports\Loader.binding\tracing\BinderTracingTest.ResolutionFlow\BinderTracingTest.ResolutionFlow.output.txt
      Raw output:
      BEGIN EXECUTION
       "C:\h\w\9EA008B3\p\corerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  BinderTracingTest.ResolutionFlow.dll 
      [9:54:06 PM] Running AssemblyLoadContextResolving_ReturnNull...
      [9:54:29 PM] Running AssemblyLoadContextResolving_LoadAssembly...
      [9:54:32 PM] Running AssemblyLoadContextResolving_NameMismatch...
      [9:54:36 PM] Running AssemblyLoadContextResolving_MultipleHandlers...
      [9:54:37 PM] Running AppDomainAssemblyResolve_ReturnNull...
      Expected: 100
      Actual: -1073740286
      END EXECUTION - FAILED
      FAILED
      Test failed. Trying to see if dump file was created in C:\cores since 5/9/2023 9:54:02 PM
      Processing C:\cores\corerun.exe.5216.dmp
      Invoking: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe -c "$<C:\h\w\9EA008B3\t\tmpbh5rqi.tmp" -z "C:\cores\corerun.exe.5216.dmp"
      stdout: 
      Microsoft (R) Windows Debugger Version 10.0.18362.1 AMD64
      Copyright (c) Microsoft Corporation. All rights reserved.
      
      
      Loading Dump File [C:\cores\corerun.exe.5216.dmp]
      User Mini Dump File with Full Memory: Only application data is available
      
      
      ************* Path validation summary **************
      Response                         Time (ms)     Location
      OK                                             C:\h\w\9EA008B3\p\PDB
      Symbol search path is: C:\h\w\9EA008B3\p\PDB
      Executable search path is: 
      Windows 10 Version 14393 MP (4 procs) Free x64
      Product: Server, suite: TerminalServer DataCenter SingleUserTS
      10.0.14393.5125 (rs1_release.220429-1732)
      Machine Name:
      Debug session time: Tue May  9 21:54:42.000 2023 (UTC + 0:00)
      System Uptime: 0 days 1:50:27.487
      Process Uptime: 0 days 0:00:39.000
      ..........................................
      
      ************* Symbol Loading Error Summary **************
      Module name            Error
      ntdll                  The system cannot find the file specified
      
      You can troubleshoot most symbol related issues by turning on symbol loading diagnostics (!sym noisy) and repeating the command that caused symbols to be loaded.
      You should also verify that your symbol search path (.sympath) is correct.
      This dump file has an exception of interest stored in it.
      The stored exception information can be accessed via .ecxr.
      (1460.fe8): Unknown exception - code c0000602 (first/second chance not available)
      For analysis of this file, run !analyze -v
      *** WARNING: Unable to verify checksum for coreclr.dll
      KERNELBASE!RaiseFailFastException+0xae:
      00007ffa`955b96ce 803d4d78120000  cmp     byte ptr [KERNELBASE!local_unwind+0x7fe2 (00007ffa`956e0f22)],0 ds:00007ffa`956e0f22=00
      0:004> cdb: Reading initial command '$<C:\h\w\9EA008B3\t\tmpbh5rqi.tmp'
      0:004> .load C:\Users\runner\.dotnet\sos\sos.dll
      0:004> ~*k
      
         0  Id: 1460.1274 Suspend: 0 Teb: 0000009d`716cd000 Unfrozen
      Child-SP          RetAddr           Call Site
      0000009d`7197d368 00007ffa`95516d1f ntdll!ZwWaitForSingleObject+0x14
      0000009d`7197d370 00007ffa`66e2b261 KERNELBASE!WaitForSingleObjectEx+0x8f
      (Inline Function) --------`-------- coreclr!CLREventWaitHelper2+0x11
      0000009d`7197d410 00007ffa`66e29f9c coreclr!`CLREventWaitHelper'::`4'::__Body::Run+0x81
      0000009d`7197d460 00007ffa`66e2b908 coreclr!CLREventWaitHelper+0x2c
      0000009d`7197d4c0 00007ffa`66e38766 coreclr!CLREventBase::WaitEx+0x2f8
      0000009d`7197d610 00007ffa`66e37b93 coreclr!ThreadSuspend::SuspendRuntime+0xa76
      0000009d`7197d880 00007ffa`66c77a38 coreclr!ThreadSuspend::SuspendEE+0x253
      0000009d`7197d950 00007ffa`66fd1a60 coreclr!GCToEEInterface::SuspendEE+0x48
      0000009d`7197d980 00007ffa`66fd1706 coreclr!WKS::GCHeap::GarbageCollectGeneration+0x2e0
      (Inline Function) --------`-------- coreclr!WKS::GCHeap::GarbageCollectTry+0x63
      0000009d`7197d9e0 00007ffa`66fd4b60 coreclr!WKS::GCHeap::GarbageCollect+0x246
      0000009d`7197da40 00007ffa`66afcf27 coreclr!WKS::GCHeap::StressHeap+0x5f0
      (Inline Function) --------`-------- coreclr!_GCStress::StressGcTriggerPolicy::Trigger+0x81
      (Inline Function) --------`-------- coreclr!_GCStress::GCSBase<0,_GCStress::EeconfigFastGcSPolicy,_GCStress::CoopGcModePolicy,mpl::null_type>::MaybeTrigger+0xd6
      0000009d`7197db00 00007ffa`66b0795a coreclr!MethodDesc::DoPrestub+0x5d7
      0000009d`7197deb0 00007ffa`6702cc95 coreclr!PreStubWorker+0x4aa
      0000009d`7197e0b0 00007ffa`0779032c coreclr!ThePreStub+0x55
      0000009d`7197e160 000001e6`c6c02790 0x00007ffa`0779032c
      0000009d`7197e168 000001e6`c6c02558 0x000001e6`c6c02790
      0000009d`7197e170 000001e6`c6c02800 0x000001e6`c6c02558
      0000009d`7197e178 00007ffa`673d5468 0x000001e6`c6c02800
      0000009d`7197e180 00000000`00000000 coreclr!`string'
      
         1  Id: 1460.18d0 Suspend: 0 Teb: 0000009d`716cf000 Unfrozen
      Child-SP          RetAddr           Call Site
      0000009d`71aff668 00007ffa`98c9eb5e ntdll!ZwWaitForWorkViaWorkerFactory+0x14
      0000009d`71aff670 00007ffa`98a284d4 ntdll!RtlReleaseSRWLockExclusive+0x296e
      0000009d`71affa70 00007ffa`98cd1791 kernel32!BaseThreadInitThunk+0x14
      0000009d`71affaa0 00000000`00000000 ntdll!RtlUserThreadStart+0x21
      
         2  Id: 1460.169c Suspend: 0 Teb: 0000009d`716d1000 Unfrozen
      Child-SP          RetAddr           Call Site
      0000009d`71c7f568 00007ffa`95534d6f ntdll!ZwWaitForMultipleObjects+0x14
      0000009d`71c7f570 00007ffa`95534c6e KERNELBASE!WaitForMultipleObjectsEx+0xef
      0000009d`71c7f870 00007ffa`670fb28c KERNELBASE!WaitForMultipleObjects+0xe
      0000009d`71c7f8b0 00007ffa`671d4792 coreclr!ds_ipc_poll+0x16c
      0000009d`71c7fb40 00007ffa`671d8b3c coreclr!ds_ipc_stream_factory_get_next_available_stream+0x2b2
      0000009d`71c7fc30 00007ffa`98a284d4 coreclr!server_thread+0xfc
      0000009d`71c7fcb0 00007ffa`98cd1791 kernel32!BaseThreadInitThunk+0x14
      0000009d`71c7fce0 00000000`00000000 ntdll!RtlUserThreadStart+0x21
      
         3  Id: 1460.124c Suspend: 0 Teb: 0000009d`716d3000 Unfrozen
      Child-SP          RetAddr           Call Site
      0000009d`71dff438 00007ffa`95534d6f ntdll!ZwWaitForMultipleObjects+0x14
      0000009d`71dff440 00007ffa`670f1a94 KERNELBASE!WaitForMultipleObjectsEx+0xef
      0000009d`71dff740 00007ffa`670f3f7b coreclr!DebuggerRCThread::MainLoop+0x5a4
      0000009d`71dffa40 00007ffa`670f40e9 coreclr!DebuggerRCThread::ThreadProc+0x4bb
      0000009d`71dffb80 00007ffa`98a284d4 coreclr!DebuggerRCThread::ThreadProcStatic+0x79
      0000009d`71dffbb0 00007ffa`98cd1791 kernel32!BaseThreadInitThunk+0x14
      0000009d`71dffbe0 00000000`00000000 ntdll!RtlUserThreadStart+0x21
      
      #  4  Id: 1460.fe8 Suspend: 0 Teb: 0000009d`716d5000 Unfrozen
      Child-SP          RetAddr           Call Site
      0000009d`71f7ade0 00007ffa`670716b5 KERNELBASE!RaiseFailFastException+0xae
      0000009d`71f7b3b0 00007ffa`67070f65 coreclr!_DbgBreakCheck+0x2e5
      0000009d`71f7c520 00007ffa`6707116f coreclr!_DbgBreakCheckNoThrow+0x55
      0000009d`71f7c5d0 00007ffa`67049796 coreclr!DbgAssertDialog+0x1af
      0000009d`71f7c6f0 00007ffa`669b93b5 coreclr!CHECK::Trigger+0x1e6
      0000009d`71f7c7f0 00007ffa`669b8c91 coreclr!CLRVectoredExceptionHandlerPhase3+0x4b5
      0000009d`71f7ce10 00007ffa`669b8bc6 coreclr!CLRVectoredExceptionHandlerPhase2+0x91
      0000009d`71f7cf10 00007ffa`669b9635 coreclr!CLRVectoredExceptionHandler+0xd36
      0000009d`71f7d4b0 00007ffa`98ce6b40 coreclr!CLRVectoredExceptionHandlerShim+0x1b5
      0000009d`71f7d520 00007ffa`98cb46cb ntdll!RtlInitializeCriticalSection+0x190
      0000009d`71f7d5c0 00007ffa`98d2989a ntdll!RtlImageNtHeaderEx+0x17b
      0000009d`71f7dcc0 00007ffa`671da10b ntdll!KiUserExceptionDispatcher+0x3a
      (Inline Function) --------`-------- coreclr!list_insert_node_after+0xc
      0000009d`71f7ea30 00007ffa`671c0341 coreclr!dn_list_insert+0xab
      (Inline Function) --------`-------- coreclr!dn_list_push_back+0x69
      0000009d`71f7ea80 00007ffa`671c60c2 coreclr!buffer_manager_allocate_buffer_for_thread+0x511
      0000009d`71f7eb80 00007ffa`671ce918 coreclr!ep_buffer_manager_write_event+0x3d2
      0000009d`71f7f2a0 00007ffa`671d2135 coreclr!ep_session_write_event+0x218
      0000009d`71f7f330 00007ffa`671cfd82 coreclr!write_event_2+0x1f5
      (Inline Function) --------`-------- coreclr!write_event+0x77
      0000009d`71f7f3a0 00007ffa`671b4427 coreclr!ep_write_event+0x92
      0000009d`71f7f410 00007ffa`671a65ad coreclr!EventPipeAdapter::WriteEvent+0xd7
      0000009d`71f7f4e0 00007ffa`66c6e9c5 coreclr!EventPipeWriteEventGCFinalizersBegin_V1+0x6d
      (Inline Function) --------`-------- coreclr!FireEtwGCFinalizersBegin_V1+0xd
      0000009d`71f7f570 00007ffa`66c6fbc4 coreclr!FinalizerThread::FinalizeAllObjects+0x35
      0000009d`71f7f600 00007ffa`66b60784 coreclr!FinalizerThread::FinalizerThreadWorker+0x884
      0000009d`71f7f910 00007ffa`66b60886 coreclr!ManagedThreadBase_DispatchInner+0xb4
      0000009d`71f7f9d0 00007ffa`66b6301b coreclr!ManagedThreadBase_DispatchMiddle+0xb6
      0000009d`71f7fb60 00007ffa`66b63313 coreclr!``ManagedThreadBase_DispatchOuter'::`11'::__Body::Run'::`5'::__Body::Run+0x7b
      0000009d`71f7fbb0 00007ffa`66b6099d coreclr!`ManagedThreadBase_DispatchOuter'::`11'::__Body::Run+0x93
      0000009d`71f7fc30 00007ffa`66b60bcb coreclr!ManagedThreadBase_DispatchOuter+0xdd
      0000009d`71f7fcb0 00007ffa`66c6f06c coreclr!ManagedThreadBase_NoADTransition+0xcb
      0000009d`71f7fd90 00007ffa`98a284d4 coreclr!FinalizerThread::FinalizerThreadStart+0x19c
      0000009d`71f7fdc0 00007ffa`98cd1791 kernel32!BaseThreadInitThunk+0x14
      0000009d`71f7fdf0 00000000`00000000 ntdll!RtlUserThreadStart+0x21
      
         5  Id: 1460.e90 Suspend: 0 Teb: 0000009d`716d7000 Unfrozen
      Child-SP          RetAddr           Call Site
      0000009d`720ffa98 00007ffa`98c9eb5e ntdll!ZwWaitForWorkViaWorkerFactory+0x14
      0000009d`720ffaa0 00007ffa`98a284d4 ntdll!RtlReleaseSRWLockExclusive+0x296e
      0000009d`720ffea0 00007ffa`98cd1791 kernel32!BaseThreadInitThunk+0x14
      0000009d`720ffed0 00000000`00000000 ntdll!RtlUserThreadStart+0x21
      
         6  Id: 1460.dfc Suspend: 0 Teb: 0000009d`716e1000 Unfrozen
      Child-SP          RetAddr           Call Site
      0000009d`7287e0a8 00007ffa`95516d1f ntdll!ZwWaitForSingleObject+0x14
      0000009d`7287e0b0 00007ffa`67009ad1 KERNELBASE!WaitForSingleObjectEx+0x8f
      0000009d`7287e150 00007ffa`67006495 coreclr!WKS::gc_heap::wait_for_gc_done+0x41
      0000009d`7287e180 00007ffa`66fd0661 coreclr!WKS::gc_heap::try_allocate_more_space+0x35
      (Inline Function) --------`-------- coreclr!WKS::gc_heap::allocate_more_space+0x11
      (Inline Function) --------`-------- coreclr!WKS::gc_heap::allocate+0x60
      0000009d`7287e200 00007ffa`66c79a09 coreclr!WKS::GCHeap::Alloc+0x301
      0000009d`7287e2f0 00007ffa`66c7beff coreclr!Alloc+0x1e9
      0000009d`7287e3e0 00007ffa`66c7c0b8 coreclr!AllocateSzArray+0x1ef
      0000009d`7287e500 00007ffa`66c7b500 coreclr!AllocateSzArray+0xe8
      0000009d`7287e5e0 00007ffa`66876d97 coreclr!AllocateObjectArray+0x1f0
      0000009d`7287e750 00007ffa`6687fea0 coreclr!ThreadStaticHandleBucket::ThreadStaticHandleBucket+0x297
      0000009d`7287e950 00007ffa`66b69e06 coreclr!ThreadStaticHandleTable::AllocateHandles+0x1c0
      0000009d`7287ea70 00007ffa`66b6ab29 coreclr!ThreadLocalBlock::AllocateStaticFieldObjRefPtrs+0x1d6
      0000009d`7287eb90 00007ffa`66b696d7 coreclr!ThreadLocalBlock::AllocateThreadStaticHandles+0x299
      0000009d`7287ed00 00007ffa`66b6ba6c coreclr!ThreadStatics::AllocateAndInitTLM+0x147
      0000009d`7287edf0 00007ffa`66cf176e coreclr!ThreadStatics::GetTLM+0x14c
      0000009d`7287eeb0 00007ffa`66cd7459 coreclr!JIT_GetGCThreadStaticBase_Helper+0x26e
      0000009d`7287f1b0 00007ffa`077cf34b coreclr!JIT_GetSharedGCThreadStaticBase+0x2c9
      0000009d`7287f280 0000009d`7287f3b0 0x00007ffa`077cf34b
      0000009d`7287f288 000001e6`c6c02c28 0x0000009d`7287f3b0
      0000009d`7287f290 000001e6`c6c02c28 0x000001e6`c6c02c28
      0000009d`7287f298 00000000`00000000 0x000001e6`c6c02c28
      0:004> !clrstack -f -all
      OS Thread Id: 0x1274
              Child SP               IP Call Site
      0000009D7197D368 00007FFA98D25D14 ntdll!ZwWaitForSingleObject + 20
      0000009D7197D370 00007FFA95516D1F KERNELBASE!WaitForSingleObjectEx + 143
      0000009D7197D410 00007FFA66E2B261 coreclr!`CLREventWaitHelper'::`4'::__Body::Run + 129
      0000009D7197D460 00007FFA66E29F9C coreclr!CLREventWaitHelper + 44
      0000009D7197D4C0 00007FFA66E2B908 coreclr!CLREventBase::WaitEx + 760
      0000009D7197D610 00007FFA66E38766 coreclr!ThreadSuspend::SuspendRuntime + 2678
      0000009D7197D880 00007FFA66E37B93 coreclr!ThreadSuspend::SuspendEE + 595
      0000009D7197D950 00007FFA66C77A38 coreclr!GCToEEInterface::SuspendEE + 72
      0000009D7197D980 00007FFA66FD1A60 coreclr!WKS::GCHeap::GarbageCollectGeneration + 736
      0000009D7197D9E0 00007FFA66FD1706 coreclr!WKS::GCHeap::GarbageCollect + 582
      0000009D7197DA40 00007FFA66FD4B60 coreclr!WKS::GCHeap::StressHeap + 1520
      0000009D7197DB00 00007FFA66AFCF27 coreclr!MethodDesc::DoPrestub + 1495
      0000009D7197DEB0 00007FFA66B0795A coreclr!PreStubWorker + 1194
      0000009D7197DFB8                  [PrestubMethodFrame: 0000009d7197dfb8] BinderTracingTest.ResolutionFlow.dll!BinderTracingTests.BinderTracingTest.AppDomainAssemblyResolve_ReturnNull()
      0000009D7197E0B0 00007FFA6702CC95 coreclr!ThePreStub + 85
      0000009D7197E160 00007FFA0779032C BinderTracingTest.ResolutionFlow.dll!BinderTracingTests.BinderTracingTest.RunSingleTest(System.Reflection.MethodInfo) + 1292
      0000009D7197E168 000001E6C6C02790 
      0000009D7197E170 000001E6C6C02558 
      0000009D7197E178 000001E6C6C02800 
      0000009D7197E180 00007FFA673D5468 coreclr!`string'
      0000009D7197E210 00007FFA0778A867 BinderTracingTest.ResolutionFlow.dll!BinderTracingTests.BinderTracingTest.RunAllTests() + 711
      0000009D7197E2A0 00007FFA0778A2FD BinderTracingTest.ResolutionFlow.dll!BinderTracingTests.BinderTracingTest.Main(System.String[]) + 61
      OS Thread Id: 0xfe8
              Child SP               IP Call Site
      0000009D71F7ADE0 00007FFA955B96CE KERNELBASE!RaiseFailFastException + 174
      0000009D71F7B3B0 00007FFA670716B5 coreclr!_DbgBreakCheck + 741
      0000009D71F7C520 00007FFA67070F65 coreclr!_DbgBreakCheckNoThrow + 85
      0000009D71F7C5D0 00007FFA6707116F coreclr!DbgAssertDialog + 431
      0000009D71F7C6F0 00007FFA67049796 coreclr!CHECK::Trigger + 486
      0000009D71F7C7F0 00007FFA669B93B5 coreclr!CLRVectoredExceptionHandlerPhase3 + 1205
      0000009D71F7CE10 00007FFA669B8C91 coreclr!CLRVectoredExceptionHandlerPhase2 + 145
      0000009D71F7CF10 00007FFA669B8BC6 coreclr!CLRVectoredExceptionHandler + 3382
      0000009D71F7D4B0 00007FFA669B9635 coreclr!CLRVectoredExceptionHandlerShim + 437
      0000009D71F7D520 00007FFA98CE6B40 ntdll!RtlInitializeCriticalSection + 400
      0000009D71F7D5C0 00007FFA98CB46CB ntdll!RtlImageNtHeaderEx + 379
      0000009D71F7DCC0 00007FFA98D2989A ntdll!KiUserExceptionDispatcher + 58
      0000009D71F7EA30 00007FFA671DA10B coreclr!dn_list_insert + 171
      0000009D71F7EA80 00007FFA671C0341 coreclr!buffer_manager_allocate_buffer_for_thread + 1297
      0000009D71F7EB80 00007FFA671C60C2 coreclr!ep_buffer_manager_write_event + 978
      0000009D71F7F2A0 00007FFA671CE918 coreclr!ep_session_write_event + 536
      0000009D71F7F330 00007FFA671D2135 coreclr!write_event_2 + 501
      0000009D71F7F3A0 00007FFA671CFD82 coreclr!ep_write_event + 146
      0000009D71F7F410 00007FFA671B4427 coreclr!EventPipeAdapter::WriteEvent + 215
      0000009D71F7F4E0 00007FFA671A65AD coreclr!EventPipeWriteEventGCFinalizersBegin_V1 + 109
      0000009D71F7F570 00007FFA66C6E9C5 coreclr!FinalizerThread::FinalizeAllObjects + 53
      0000009D71F7F600 00007FFA66C6FBC4 coreclr!FinalizerThread::FinalizerThreadWorker + 2180
      0000009D71F7F910 00007FFA66B60784 coreclr!ManagedThreadBase_DispatchInner + 180
      0000009D71F7F9D0 00007FFA66B60886 coreclr!ManagedThreadBase_DispatchMiddle + 182
      0000009D71F7FB60 00007FFA66B6301B coreclr!``ManagedThreadBase_DispatchOuter'::`11'::__Body::Run'::`5'::__Body::Run + 123
      0000009D71F7FBB0 00007FFA66B63313 coreclr!`ManagedThreadBase_DispatchOuter'::`11'::__Body::Run + 147
      0000009D71F7FC30 00007FFA66B6099D coreclr!ManagedThreadBase_DispatchOuter + 221
      0000009D71F7FC68                  [DebuggerU2MCatchHandlerFrame: 0000009d71f7fc68] 
      0000009D71F7FCB0 00007FFA66B60BCB coreclr!ManagedThreadBase_NoADTransition + 203
      0000009D71F7FD90 00007FFA66C6F06C coreclr!FinalizerThread::FinalizerThreadStart + 412
      0000009D71F7FDC0 00007FFA98A284D4 kernel32!BaseThreadInitThunk + 20
      0000009D71F7FDF0 00007FFA98CD1791 ntdll!RtlUserThreadStart + 33
      OS Thread Id: 0xdfc
              Child SP               IP Call Site
      0000009D7287E0A8 00007FFA98D25D14 ntdll!ZwWaitForSingleObject + 20
      0000009D7287E0B0 00007FFA95516D1F KERNELBASE!WaitForSingleObjectEx + 143
      0000009D7287E150 00007FFA67009AD1 coreclr!WKS::gc_heap::wait_for_gc_done + 65
      0000009D7287E180 00007FFA67006495 coreclr!WKS::gc_heap::try_allocate_more_space + 53
      0000009D7287E200 00007FFA66FD0661 coreclr!WKS::GCHeap::Alloc + 769
      0000009D7287E2F0 00007FFA66C79A09 coreclr!Alloc + 489
      0000009D7287E3E0 00007FFA66C7BEFF coreclr!AllocateSzArray + 495
      0000009D7287E500 00007FFA66C7C0B8 coreclr!AllocateSzArray + 232
      0000009D7287E5E0 00007FFA66C7B500 coreclr!AllocateObjectArray + 496
      0000009D7287E750 00007FFA66876D97 coreclr!ThreadStaticHandleBucket::ThreadStaticHandleBucket + 663
      0000009D7287E950 00007FFA6687FEA0 coreclr!ThreadStaticHandleTable::AllocateHandles + 448
      0000009D7287EA70 00007FFA66B69E06 coreclr!ThreadLocalBlock::AllocateStaticFieldObjRefPtrs + 470
      0000009D7287EB90 00007FFA66B6AB29 coreclr!ThreadLocalBlock::AllocateThreadStaticHandles + 665
      0000009D7287ED00 00007FFA66B696D7 coreclr!ThreadStatics::AllocateAndInitTLM + 327
      0000009D7287EDF0 00007FFA66B6BA6C coreclr!ThreadStatics::GetTLM + 332
      0000009D7287EEB0 00007FFA66CF176E coreclr!JIT_GetGCThreadStaticBase_Helper + 622
      0000009D7287F088                  [HelperMethodFrame: 0000009d7287f088] 
      0000009D7287F1B0 00007FFA66CD7459 coreclr!JIT_GetSharedGCThreadStaticBase + 713
      0000009D7287F280 00007FFA077CF34B System.Private.CoreLib.dll!System.Threading.Tasks.ThreadPoolTaskScheduler+<>c.<.cctor>b__10_0(System.Object) + 155
      0000009D7287F288 0000009D7287F3B0 
      0000009D7287F290 000001E6C6C02C28 
      0000009D7287F298 000001E6C6C02C28 
      0000009D7287F998                  [DebuggerU2MCatchHandlerFrame: 0000009d7287f998] 
      0:004> q
      quit:
      NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\atlmfc.natvis'
      NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\concurrency.natvis'
      NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\cpp_rest.natvis'
      NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\stl.natvis'
      NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\Windows.Data.Json.natvis'
      NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\Windows.Devices.Geolocation.natvis'
      NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\Windows.Devices.Sensors.natvis'
      NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\Windows.Media.natvis'
      NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\windows.natvis'
      NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\winrt.natvis'
      
      Test Harness Exitcode is : 1
      To run the test:
      > set CORE_ROOT=C:\h\w\9EA008B3\p
      > C:\h\w\9EA008B3\w\ADFD0960\e\Loader\binding\tracing\BinderTracingTest.ResolutionFlow\BinderTracingTest.ResolutionFlow.cmd
      Expected: True
      Actual:   False

@BruceForstall BruceForstall added the blocking-clean-ci-optional Blocking optional rolling runs label May 10, 2023
@v-wenyuxu
Copy link
Author

Failed again in: runtime-coreclr gcstress-extra 20230509.1

Failed tests:

coreclr windows x64 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.10.Amd64.Open
    - Loader\\binding\\tracing\\BinderTracingTest.ResolutionFlow\\BinderTracingTest.ResolutionFlow.cmd

Error message:

 
Assert failure(PID 5216 [0x00001460], Thread: 4072 [0x0fe8]): Consistency check failed: AV in clr at this callstack:
------
CORECLR! dn_list_insert + 0xAB (0x00007ffa`671da10b)
CORECLR! buffer_manager_allocate_buffer_for_thread + 0x511 (0x00007ffa`671c0341)
CORECLR! ep_buffer_manager_write_event + 0x3D2 (0x00007ffa`671c60c2)
CORECLR! ep_session_write_event + 0x218 (0x00007ffa`671ce918)
CORECLR! write_event_2 + 0x1F5 (0x00007ffa`671d2135)
CORECLR! ep_write_event + 0x92 (0x00007ffa`671cfd82)
CORECLR! EventPipeAdapter::WriteEvent + 0xD7 (0x00007ffa`671b4427)
CORECLR! EventPipeWriteEventGCFinalizersBegin_V1 + 0x6D (0x00007ffa`671a65ad)
CORECLR! FinalizerThread::FinalizeAllObjects + 0x35 (0x00007ffa`66c6e9c5)
CORECLR! FinalizerThread::FinalizerThreadWorker + 0x884 (0x00007ffa`66c6fbc4)
CORECLR! ManagedThreadBase_DispatchInner + 0xB4 (0x00007ffa`66b60784)
CORECLR! ManagedThreadBase_DispatchMiddle + 0xB6 (0x00007ffa`66b60886)
CORECLR! ``ManagedThreadBase_DispatchOuter&#39;::`11&#39;::__Body::Run&#39;::`5&#39;::__Body::Run + 0x7B (0x00007ffa`66b6301b)
CORECLR! `ManagedThreadBase_DispatchOuter&#39;::`11&#39;::__Body::Run + 0x93 (0x00007ffa`66b63313)
CORECLR! ManagedThreadBase_DispatchOuter + 0xDD (0x00007ffa`66b6099d)
CORECLR! ManagedThreadBase_NoADTransition + 0xCB (0x00007ffa`66b60bcb)
CORECLR! FinalizerThread::FinalizerThreadStart + 0x19C (0x00007ffa`66c6f06c)
KERNEL32! BaseThreadInitThunk + 0x14 (0x00007ffa`98a284d4)
NTDLL! RtlUserThreadStart + 0x21 (0x00007ffa`98cd1791)
-----
.AV on tid=0xfe8 (4072), cxr=0000009D71F7DCC0, exr=0000009D71F7E1B0
FAILED: false

CORECLR! CHECK::Trigger + 0x1E6 (0x00007ffa`67049796)
CORECLR! CLRVectoredExceptionHandlerPhase3 + 0x4B5 (0x00007ffa`669b93b5)
CORECLR! CLRVectoredExceptionHandlerPhase2 + 0x91 (0x00007ffa`669b8c91)
CORECLR! CLRVectoredExceptionHandler + 0xD36 (0x00007ffa`669b8bc6)
CORECLR! CLRVectoredExceptionHandlerShim + 0x1B5 (0x00007ffa`669b9635)
NTDLL! RtlInitializeCriticalSection + 0x190 (0x00007ffa`98ce6b40)
NTDLL! RtlImageNtHeaderEx + 0x17B (0x00007ffa`98cb46cb)
NTDLL! KiUserExceptionDispatcher + 0x3A (0x00007ffa`98d2989a)
CORECLR! dn_list_insert + 0xAB (0x00007ffa`671da10b)
CORECLR! buffer_manager_allocate_buffer_for_thread + 0x511 (0x00007ffa`671c0341)
    File: D:\a\_work\1\s\src\coreclr\vm\excep.cpp Line: 7115
    Image: C:\h\w\9EA008B3\p\corerun.exe


Return code:      1
Raw output file:      C:\h\w\9EA008B3\w\ADFD0960\uploads\Reports\Loader.binding\tracing\BinderTracingTest.ResolutionFlow\BinderTracingTest.ResolutionFlow.output.txt
Raw output:
BEGIN EXECUTION
 &quot;C:\h\w\9EA008B3\p\corerun.exe&quot; -p &quot;System.Reflection.Metadata.MetadataUpdater.IsSupported=false&quot; -p &quot;System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true&quot;  BinderTracingTest.ResolutionFlow.dll 
[9:54:06 PM] Running AssemblyLoadContextResolving_ReturnNull...
[9:54:29 PM] Running AssemblyLoadContextResolving_LoadAssembly...
[9:54:32 PM] Running AssemblyLoadContextResolving_NameMismatch...
[9:54:36 PM] Running AssemblyLoadContextResolving_MultipleHandlers...
[9:54:37 PM] Running AppDomainAssemblyResolve_ReturnNull...
Expected: 100
Actual: -1073740286
END EXECUTION - FAILED
FAILED
Test failed. Trying to see if dump file was created in C:\cores since 5/9/2023 9:54:02 PM
Processing C:\cores\corerun.exe.5216.dmp
Invoking: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe -c &quot;$&lt;C:\h\w\9EA008B3\t\tmpbh5rqi.tmp&quot; -z &quot;C:\cores\corerun.exe.5216.dmp&quot;
stdout: 
Microsoft (R) Windows Debugger Version 10.0.18362.1 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\cores\corerun.exe.5216.dmp]
User Mini Dump File with Full Memory: Only application data is available


************* Path validation summary **************
Response                         Time (ms)     Location
OK                                             C:\h\w\9EA008B3\p\PDB
Symbol search path is: C:\h\w\9EA008B3\p\PDB
Executable search path is: 
Windows 10 Ve

Stack trace:

    at Loader_binding._tracing_BinderTracingTest_ResolutionFlow_BinderTracingTest_ResolutionFlow_._tracing_BinderTracingTest_ResolutionFlow_BinderTracingTest_ResolutionFlow_cmd()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

@davmason davmason mentioned this issue May 12, 2023
Merged
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label May 12, 2023
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label May 15, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Jun 14, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@davmason davmason

Labels
arch-x64 area-Tracing-coreclr blocking-clean-ci-optional Blocking optional rolling runs JitStress CLR JIT issues involving JIT internal stress modes os-windows
Projects
None yet
Milestone
8.0.0
Development

Successfully merging a pull request may close this issue.

Fix EventPipe shutdown logic davmason/runtime
6 participants
@vcsjones @jeffschwMSFT @BruceForstall @davmason @tommcdon @v-wenyuxu