[Crossgen2] compiler crash in Interop\PInvoke\Vector2_3_4\Vector2_3_4 on x86

[trylek]

OS: Windows
Architecture: x86
Example run: https://dev.azure.com/dnceng/public/_build/results?buildId=1094848&view=ms.vss-test-web.build-test-results-tab&runId=33520836&resultId=101221&paneView=debug

Diagnostics:

D:\git\runtime2\artifacts\tests\coreclr\windows.x86.Checked\Tests\Core_Root\crossgen2\crossgen2.dll" @"D:\git\runtime2\artifacts\tests\coreclr\windows.x86.Checked\Interop\PInvoke\Vector2_3_4\Vector2_3_4\\composite-r2r.dll.rsp"   --composite"
D:\git\runtime2\src\coreclr\jit\codegenxarch.cpp:1983
Assertion failed '!"Multireg store to SIMD reg not supported on X64 Windows"' in 'Vector2_3_4TestNative:CreateVector2FromFloats(float,float):System.Numerics.Vector2' during 'Generate code' (IL size 18)

With Crossgen1 the test compiles and passes but it's unclear to me whether it's something that's wrong in Crossgen2 or whether we're just more aggressive in using SIMD instructions or whatnot and we end up tripping a JIT bug; @davidwrighton, could you please share your thoughts?

/cc @dotnet/crossgen-contrib @dotnet/jit-contrib

[mangod9]

Maybe some logic is missing a case, since the assert seems to claim its running on windows x64?

[trylek]

NgenDump: https://gist.github.com/trylek/a3a106a5f1010949660bb791067e11af

[trylek]

Yes, that's what also occurred to me. The offending code bit is here:

runtime/src/coreclr/jit/codegenxarch.cpp

Line 1983 in 2aba0c5

assert(!"Multireg store to SIMD reg not supported on X64 Windows");

Interestingly enough, I believe I'm getting the same error when compiling using the x86- and x64-hosted Crossgen2 (I would naively assume that codegenxarch is somehow related to cross-compilation which shouldn't be happening with x86-hosted Crossgen2).

[trylek]

Assertion call stack:

>	clrjit_win_x86_x64.dll!assertAbort(const char * why, const char * file, unsigned int line) Line 306	C++
 	clrjit_win_x86_x64.dll!CodeGen::genMultiRegStoreToSIMDLocal(GenTreeLclVar * lclNode) Line 1983	C++
 	clrjit_win_x86_x64.dll!CodeGen::genMultiRegStoreToLocal(GenTreeLclVar * lclNode) Line 11319	C++
 	clrjit_win_x86_x64.dll!CodeGen::genCodeForStoreLclVar(GenTreeLclVar * lclNode) Line 4421	C++
 	clrjit_win_x86_x64.dll!CodeGen::genCodeForTreeNode(GenTree * treeNode) Line 1616	C++
 	clrjit_win_x86_x64.dll!CodeGen::genCodeForBBlist() Line 452	C++
 	clrjit_win_x86_x64.dll!CodeGen::genGenerateMachineCode() Line 2312	C++
 	clrjit_win_x86_x64.dll!CodeGenPhase::DoPhase() Line 1594	C++
 	clrjit_win_x86_x64.dll!Phase::Run() Line 61	C++
 	clrjit_win_x86_x64.dll!DoPhase(CodeGen * _codeGen, Phases _phase, void(CodeGen::*)() _action) Line 1608	C++
 	clrjit_win_x86_x64.dll!CodeGen::genGenerateCode(void * * codePtr, unsigned int * nativeSizeOfCode) Line 2111	C++
 	clrjit_win_x86_x64.dll!Compiler::compCompile(void * * methodCodePtr, unsigned int * methodCodeSize, JitFlags * compileFlags) Line 5184	C++
 	clrjit_win_x86_x64.dll!Compiler::compCompileHelper(CORINFO_MODULE_STRUCT_ * classPtr, ICorJitInfo * compHnd, CORINFO_METHOD_INFO * methodInfo, void * * methodCodePtr, unsigned int * methodCodeSize, JitFlags * compileFlags) Line 6442	C++
 	clrjit_win_x86_x64.dll!`Compiler::compCompile'::`67'::__Body::Run(Compiler::compCompile::__l2::__JITParam * __JITpParam) Line 5717	C++
 	clrjit_win_x86_x64.dll!Compiler::compCompile(CORINFO_MODULE_STRUCT_ * classPtr, void * * methodCodePtr, unsigned int * methodCodeSize, JitFlags * compileFlags) Line 5721	C++
 	clrjit_win_x86_x64.dll!``jitNativeCode'::`8'::__Body::Run'::`5'::__Body::Run(jitNativeCode::__l8::__Body::Run::__l4::__JITParam * __JITpParam) Line 7085	C++
 	clrjit_win_x86_x64.dll!`jitNativeCode'::`8'::__Body::Run(jitNativeCode::__l2::__JITParam * __JITpParam) Line 7088	C++
 	clrjit_win_x86_x64.dll!jitNativeCode(CORINFO_METHOD_STRUCT_ * methodHnd, CORINFO_MODULE_STRUCT_ * classPtr, ICorJitInfo * compHnd, CORINFO_METHOD_INFO * methodInfo, void * * methodCodePtr, unsigned int * methodCodeSize, JitFlags * compileFlags, void * inlineInfoPtr) Line 7112	C++
 	clrjit_win_x86_x64.dll!CILJit::compileMethod(ICorJitInfo * compHnd, CORINFO_METHOD_INFO * methodInfo, unsigned int flags, unsigned char * * entryAddress, unsigned int * nativeSizeOfCode) Line 276	C++
 	jitinterface_x64.dll!JitCompileMethod(CorInfoExceptionClass * * ppException, ICorJitCompiler * pJit, void * thisHandle, void * * callbacks, CORINFO_METHOD_INFO * methodInfo, unsigned int flags, unsigned char * * entryAddress, unsigned int * nativeSizeOfCode) Line 36	C++

[trylek]

WinDbg capture of the x86-hosted assert:

01 2cf6ea78 7c073bf0     KERNELBASE!wil::details::DebugBreak+0x2 [onecore\internal\sdk\inc\wil\opensource\wil\result_macros.h @ 1737] 
02 2cf6ea78 7c15cc62     clrjit_win_x86_x86!assertAbort+0xcb [D:\git\runtime2\src\coreclr\jit\error.cpp @ 306] 
03 2cf6ead8 7c057e83     clrjit_win_x86_x86!CodeGen::genMultiRegStoreToSIMDLocal+0x14 [D:\git\runtime2\src\coreclr\jit\codegenxarch.cpp @ 1983] 
04 2cf6ead8 7c1591db     clrjit_win_x86_x86!CodeGen::genMultiRegStoreToLocal+0x168 [D:\git\runtime2\src\coreclr\jit\codegencommon.cpp @ 11319] 
05 2cf6eb0c 7c159948     clrjit_win_x86_x86!CodeGen::genCodeForStoreLclVar+0x65 [D:\git\runtime2\src\coreclr\jit\codegenxarch.cpp @ 4421] 
06 2cf6ec48 7c05c3ae     clrjit_win_x86_x86!CodeGen::genCodeForTreeNode+0x2cc [D:\git\runtime2\src\coreclr\jit\codegenxarch.cpp @ 1616] 
07 2cf6ed1c 7c056c9f     clrjit_win_x86_x86!CodeGen::genCodeForBBlist+0x79a [D:\git\runtime2\src\coreclr\jit\codegenlinear.cpp @ 452] 
08 2cf6ed34 7c051a04     clrjit_win_x86_x86!CodeGen::genGenerateMachineCode+0x36f [D:\git\runtime2\src\coreclr\jit\codegencommon.cpp @ 2312] 
09 2cf6ed74 7c130f68     clrjit_win_x86_x86!CodeGenPhase::DoPhase+0x14 [D:\git\runtime2\src\coreclr\jit\codegen.h @ 1594] 
0a 2cf6ed74 7c0568c2     clrjit_win_x86_x86!Phase::Run+0x38 [D:\git\runtime2\src\coreclr\jit\phase.cpp @ 62] 
0b (Inline) --------     clrjit_win_x86_x86!DoPhase+0x41 [D:\git\runtime2\src\coreclr\jit\codegen.h @ 1607] 
0c 2cf6ed9c 7c06336c     clrjit_win_x86_x86!CodeGen::genGenerateCode+0x72 [D:\git\runtime2\src\coreclr\jit\codegencommon.cpp @ 2111] 
0d 2cf6ef50 7c064462     clrjit_win_x86_x86!Compiler::compCompile+0xa35 [D:\git\runtime2\src\coreclr\jit\compiler.cpp @ 5184] 
0e 2cf6efb4 7c06391f     clrjit_win_x86_x86!Compiler::compCompileHelper+0x5f6 [D:\git\runtime2\src\coreclr\jit\compiler.cpp @ 6442] 
0f (Inline) --------     clrjit_win_x86_x86!Compiler::compCompile::__l67::__Body::Run+0x15 [D:\git\runtime2\src\coreclr\jit\compiler.cpp @ 5717] 
10 2cf6f030 7c06231c     clrjit_win_x86_x86!Compiler::compCompile+0x4af [D:\git\runtime2\src\coreclr\jit\compiler.cpp @ 5721] 
11 2cf6f04c 7c06235c     clrjit_win_x86_x86!``jitNativeCode'::`8'::__Body::Run'::`5'::__Body::Run+0xc5 [D:\git\runtime2\src\coreclr\jit\compiler.cpp @ 7085] 
12 2cf6f09c 7c067e27     clrjit_win_x86_x86!`jitNativeCode'::`8'::__Body::Run+0x38 [D:\git\runtime2\src\coreclr\jit\compiler.cpp @ 7088] 
13 2cf6f49c 7c06a3cb     clrjit_win_x86_x86!jitNativeCode+0x132 [D:\git\runtime2\src\coreclr\jit\compiler.cpp @ 7112] 
14 2cf6f524 7bff47cc     clrjit_win_x86_x86!CILJit::compileMethod+0xdb [D:\git\runtime2\src\coreclr\jit\ee_il_dll.cpp @ 276] 
15 2cf6f590 2dc439e6     jitinterface_x86!JitCompileMethod+0x9c [D:\git\runtime2\src\coreclr\tools\aot\jitinterface\jitwrapper.cpp @ 36] 

[trylek]

Also, please note that, according to the preprocessor directives, the method genMultiRegStoreToSIMDLocal only works when targeting Linux x64 ABI so the problem seems to be higher up the call stack (we shouldn't be calling the method at all when targeting x86 on Windows).

[davidwrighton]

We are definitely much more aggressive about using SIMD instructions in crossgen2 than in crossgen1. However, this looks really odd. The assert message is discussing 64bit windows, but based on the call stack, you're compiling targeting x86 windows. You need a JIT expert to look into this.

[trylek]

Thanks David for your feedback. I guess this should be trivial to triage for JIT experts like @BruceForstall, @sandreenko, or @AndyAyersMS. As I'm not a JIT expert myself, it's hard for me to estimate whether it's a mainline issue or an obscure corner case, in general 32-bit architectures often automatically imply the latter these days. Understanding that is probably the first thing we need here.

[BruceForstall]

@sandreenko Maybe you're the best one to look?

I see in the dump:

Generating: N037 ( 26,  9) [000002] --CXG-------         t2 = *  CALL r2r_ind struct Vector2_3_4TestNative.CreateVector2FromFloats REG esi,edi $180
...
                                                              /--*  t2     struct                                                               
Generating: N049 ( 30, 12) [000005] DA-XG-------              *  STORE_LCL_VAR simd8 <System.Numerics.Vector2> V05 loc2         d:2 mm0 REG mm0 

I'm not sure if this is a new form, or why the code didn't appear before.

My repro is:

C:\gh\runtime\artifacts\tests\coreclr\windows.x86.Checked\Tests\Core_Root\corerun.exe  C:\gh\runtime\artifacts\tests\coreclr\windows.x86.Checked\Tests\Core_Root\crossgen2\crossgen2.dll @C:\gh\runtime\artifacts\tests\coreclr\windows.x86.Checked\Interop\PInvoke\Vector2_3_4\Vector2_3_4\\composite-r2r.dll.rsp --composite --singlemethodtypename "Vector2_3_4TestNative,Vector2_3_4" --singlemethodname CreateVector2FromFloats --singlemethodindex 1 --codegenopt NgenDump=* --codegenopt JitStdOutFile=dump.txt

with:

C:\gh\runtime\artifacts\tests\coreclr\windows.x86.Checked\Interop\PInvoke\Vector2_3_4\Vector2_3_4>type composite-r2r.dll.rsp
C:\gh\runtime\artifacts\tests\coreclr\windows.x86.Checked\Interop\PInvoke\Vector2_3_4\Vector2_3_4\IL-CG2\*.dll
-o:C:\gh\runtime\artifacts\tests\coreclr\windows.x86.Checked\Interop\PInvoke\Vector2_3_4\Vector2_3_4\\composite-r2r.dll
--targetarch:x86
--verify-type-and-field-layout
-r:C:\gh\runtime\artifacts\tests\coreclr\windows.x86.Checked\Tests\Core_Root\System.*.dll
-r:C:\gh\runtime\artifacts\tests\coreclr\windows.x86.Checked\Tests\Core_Root\Microsoft.*.dll
-r:C:\gh\runtime\artifacts\tests\coreclr\windows.x86.Checked\Tests\Core_Root\mscorlib.dll
-r:C:\gh\runtime\artifacts\tests\coreclr\windows.x86.Checked\Tests\Core_Root\netstandard.dll
-O

cc @tannergooding

[sandreenko]

I will take a look.

[sandreenko]

The fix could be to support [eax,edx]->xmm store on x86 windows but I am not convinced that this behavior is expected.
A few questions that I need to answer first:

1. Where do we get managed code for Vector2_3_4TestNative:CreateVector2FromFloats(float,float):System.Numerics.Vector if it is a native method declared here:

   runtime/src/tests/Interop/PInvoke/Vector2_3_4/Vector2_3_4TestNative.cpp

   Lines 63 to 70 in a629767

   	extern "C" DLL_EXPORT Vector2 STDMETHODCALLTYPE CreateVector2FromFloats(float x, float y)
   	{
   	Vector2 result;
   	result.x = x;
   	result.y = y;
   	return result;
   	}

   
   I guess it is some kind of a stub created between native and managed method, but why don't we see it without crossgen2?
2. when the result is in [eax,edx] according to the native calling convention it is not in the regs according to the managed calling convention, meaning a native call is more efficient? Is there a problem to return all 8-byte structs in regs with managed ABI? link https://github.com/dotnet/runtime/pull/46238/files as related, cc @jkoritzinsky