Remove hardcoded list of compiler versions in init-compiler.sh

[omajid]

If a version-specific compiler (eg, clang-17) is passed to the script, it works as before. However, if a generic compiler (eg, clang) is passed to the script, it uses the version in $PATH. This is a change from earlier, where it was iterating through a known list of versions to find the latest version of the compiler and using that.

This change can break or modify existing build scenarios where the following are all true:

• The script is called with a generic compiler name, without a version
• Multiple versions of compilers are installed on the system
• The default compiler ($compiler) is not the latest version

In this scenario, an older version of the compiler will be used, which may not be desired.

With a generic compiler, this script now also invokes it via $compiler -dumpversion (which works on both clang and gcc) to find the version for various (existing) version checks. The compiler was already being invoked to check if -fuse-ld=lld is supported.

Fixes: #14632

[omajid]

I tested building the VMR on Fedora with this. That worked. Any particular environments I should try out?

[omajid]

cc @akoeplinger @janvorli @tmds

[akoeplinger]

the old script errored for clang < 3.5 and gcc < 4.9, I think those are probably old enough that we don't need to care, right?

[omajid]

Release dates:

• clang 3.5: 03 September 2014
• gcc 4.9: 22 April 2014

These will be more than 10 years old by the time of .NET 9 GA. I am okay with dropping these both.

[akoeplinger]

/cc @am11

[akoeplinger]

I tested building the VMR on Fedora with this. That worked. Any particular environments I should try out?

Building runtime should be a good test case, I see you're hitting some issues in dotnet/runtime#100761 so let's hold off on merging this.

[am11]

The rationale behind having explicit versions was @janvorli's: we validate each version before adding it to the list. We do not want to use any version on the PATH. That's why we have maintained it this way over years.

[akoeplinger]

Yeah he mentioned that in #14572 (comment).

I don't think it's worth being explicit anymore, especially given that we use clang from PATH on macOS and Apple ships new clang releases constantly so we need to fix issues anyway and there's also still the option of setting CLR_CC=clang-XX to override the detection.

[am11]

Apple versions are always behind the Linux versions in our environments.

[omajid]

I have reworked the change entirely. Please see the new commit message:

    Make init-compiler.sh fall back to and work with generic compiler
    
    If a version-specific compiler (eg, clang-17) is passed to the script,
    it works as before. And like before, if a generic compiler (eg clang) is
    passed to the script, it prefers to find and use the few
    specific/known/tested versions of compilers (eg, clang means use
    clang-18 if possible). However, unlike before, if none of those specific
    known versions are found, it correctly falls back to the generic
    compiler binary (eg clang) in $PATH.
    
    The script was actually supposed to do all this already, but it seems to
    have been broken when warning messages were added. These warning
    messages were parsed by msbuild. Msbuild would see the warning messages
    and fail the build.
    
    With a generic compiler, this script now also invokes it via $compiler
    -dumpversion (which works on both clang and gcc) to find the version for
    various (existing) version checks. The compiler was already being
    invoked to check if -fuse-ld=lld is supported.
    
    Fixes: #14632

And here's CI exercising the change: dotnet/runtime#100828

[omajid]

Any further thoughts or comments? Is this okay to go in?

[janvorli]

LGTM, thank you!

[akoeplinger]

We're seeing a build break caused by this in dotnet/installer#19562, reverting this for now: #14734

       Commencing build of "install" target in "native libraries component" for iossimulator.arm64.Release in /Users/runner/work/1/vmr/src/runtime/artifacts/obj/native/net9.0-iossimulator-Release-arm64
      Invoking "/Users/runner/work/1/vmr/src/runtime/eng/native/gen-buildsys.sh" "/Users/runner/work/1/vmr/src/runtime/src/native/libs" "/Users/runner/work/1/vmr/src/runtime/artifacts/obj/native/net9.0-iossimulator-Release-arm64" arm64 iossimulator clang Release ""  -DCMAKE_OSX_ARCHITECTURES="arm64" -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_SYSROOT=iphonesimulator -DCMAKE_OSX_DEPLOYMENT_TARGET=11.0 -C /Users/runner/work/1/vmr/src/runtime/eng/native/tryrun_ios_tvos.cmake -DCMAKE_USE_PTHREADS=0 -DCMAKE_OSX_ARCHITECTURES="arm64" -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_SYSROOT=iphonesimulator -DCMAKE_OSX_DEPLOYMENT_TARGET=11.0 
    EXEC : error : Specific version of clang not found [/Users/runner/work/1/vmr/src/runtime/src/native/libs/build-native.proj]
      Failed to generate "native libraries component" build project!

[akoeplinger]

shouldn't this be "$(uname)" != "Darwin? we won't have majorVersion set on Darwin.
It should also stay a warning and not fail the build on other platforms I think?

[omajid]

Yeah, good point. Let me think about this some more.

Any warning printed by this script (especially if it is prefixed by error: or warning:) will fail the build.

[jakubch1]

This change caused issue in coverage repo on macOS:

EXEC : error : Specific version of clang not found [/Users/runner/work/1/s/src/Microsoft.CleverTestsSelection.InstrumentationMethod/Microsoft.CleverTestsSelection.InstrumentationMethod.proj]
##[error]EXEC(0,0): error : (NETCORE_ENGINEERING_TELEMETRY=Build) Specific version of clang not found

before this change output was:

  -- The CXX compiler identification is AppleClang 13.0.0.13000029
  -- The CXX compiler identification is AppleClang 13.0.0.13000029

I've tried to specify: clang-13 and clang-13.0 but also got errors. Any suggestion how I can fix it?