Add options to control PIC

[MatthewFluet]

Add -native-pic {false,true} and -pic-runtime {false,true} expert options that can be used to override the target-determined default. Along with -cc-opt, -llvm-lc-opt, and -link-opt, most configurations can be explicitly achieved:

• .s -> .o

  • static: -native-pic false
    Default when target is considered to require PIC.
  • PIC: -native-pic true
    Default when target is not considered to require PIC.
  • PIE: not supported
    Not supported, in the sense that the native codegen's have no support for PIE vs PIC. Nonetheless, amd64-linux seems forgiving about linking -native-pic true object files with -pie.

• .c -> .o

  • default:
    Default when the target is not considered to require PIC; cc will be called with no -fPIC, -fno-pic, -fPIE, or -fno-pie option and, hence, the C compiler's default behavior will be followed.
  • static: -cc-opt -fno-pic
    Use to override the -fPIC added to cc flags when the target is considered to require PIC. Also, even if the target is not considered to require PIC, the C compiler may be configured with --enable-default-pie and an explicit -cc-opt -fno-pic may be required to override the implicit -fPIE of the C compiler's default behavior.
  • PIC: -cc-opt -fPIC
    Default when target is considered to require PIC. Note that if the C compiler is configured with --enable-default-pie, then this overrides the implicit -fPIE of the C compiler's default behavior.
  • PIE: -cc-opt -fPIE

• .ll -> .o

  • default:
    Default when the target is not considered to require PIC; llc will be called with no -relocation-model option.
  • static: -llvm-llc-opt -relocation-model=static
    Use to override the -relocation-mode=pic added to llc flags when the target is considered to require PIC.
  • PIC: -llvm-llc-opt -relocation-model=pic
    Default when target is considered to require PIC
  • PIE: ???
    llc has other -relocation-model options; perhaps one corresponds to a C compiler's -fPIE behavior.

• .o & .a -> exe

  • default:
    cc will be called with either -lmlton or -lmlton-pic, depending on whether or not the target is considered to require PIC, and no -static, -no-pie, or -pie option and, hence, the C compiler's default behavior will be followed.
  • statically linked: -pic-runtime false -link-opt -static
  • dynamically linked (to non-MLton libraries), non-PIE: -pic-runtime true -link-opt -no-pie
    Even if the target is not considered to require PIC, the C compiler may be configured with --enable-default-pie and an explicit -link-opt -no-pie may be required to override the implicit -pie of the C compiler's default behavior.
  • dynamically linked (to non-MLton libraries), PIE: -pic-runtime true -link-opt -pie

As can be surmised, the non-PI vs PIC vs PIC is complex and this is hardly a complete solution. However, in the absence of additional understanding and a clear design, a variety of explicit controls seems best.

Closes (albeit, incompletely) #191

[MatthewFluet]

Add options to control position-independent code

Add -native-pic {false,true} and -pi-style {default|npi|pic|pie} expert options that can be used to override the target-determined default. Along with -cc-opt, -llvm-lc-opt, and -link-opt, most configurations can be explicitly achieved.

Building on the changes in #361, a total of 12 libmlton{,-dbg}{-npi,-pic,-pie}.a configurations can be built, along two axes:

• mode: opt (no suffix) and -dbg
• position-independent style: -npi (built with -fno-pic -fno-pie), -pic (built with -fPIC), -pie (built with -fPIE)

The explicit -fno-pic -fno-pie for %-npi.a libraries are necessary if the C compiler's default is to generate PIE (e.g., Ubuntu gcc configured with --enable-default-pie) or PIC.

The runtime/Makefile determines the default behavior of the C compiler by observing the expansions of __pic__ and __pie__ with -f{,no-}{pic,pie} and only builds the meaningful libraries. (For example, -npi and -pie are not meaningful on MacOS.) Furthermore, the %.a libraries are copied from one of the %-npi.a, %-pic.a, or %-pie.a libraries. The default behavior of the C compiler is also written to the target-specific constants file, which makes it available to the compiler proper.

Compiling with one of -pi-style {npi,pic,pie} forces a particular style:

• npi: Control.Native.pic is set to false (unless overridden by an explicit -native-pic true), .c files are compiled with -fno-pic -fno-pie, .ll/.bc files are compiled with no extra flags (using the default --relocation-model=static), executables are linked with -no-pie, and archives, libraries, and libarchives are linked with no extra flags.
• pic: Control.Native.pic is set to true (unless overridden by an explicit -native-pic false), .c files are compiled with -fPIC, .ll/.bc files are compiled with --relocation-model=pic, executables are linked with -no-pie, and archives, libraries, and libarchives are linked with no extra flags.
• pie: Control.Native.pic is set to true (unless overridden by an explicit -native-pic false), .c files are compiled with -fPIE, .ll/.bc files are compiled with --relocation-model=pic, executables are linked with -fPIE -pie, and archives, libraries, and libarchives are linked with no extra flags.

Compiling with one of -format {executable,archive} and with -pi-style default attempts to follow the default behavior of the C compiler (as recorded in the target-specific constants file).

• __pic__ == 0 && __pie__ == 0: Control.Native.pic is set to false (unless overridden by an explicit -native-pic true) and .ll/.bc files are compiled with no extra flags (using the default --relocation-model=static).
• __pic__ > 0 || __pie__ > 0: Control.Native.pic is set to true (unless overridden by an explicit -native-pic false) and .ll/.bc files are compiled with --relocation-model=pic.
• .c files are compiled with no extra flags and executables and archives are linked with no extra flags.

Compiling with one of -format {library,libarchive} (on non-Windows platforms) and with -pi-style default behaves like -pi-style pic.

Note that user-supplied -cc-opt, -llvm-llc-opt, and -link-opt options are placed last in the generated command; hence, they can be used to override specific behavior. For example, compiling with -pi-style pie -link-opt -no-pie will result in .c files being compiled with -fPIC but executables being linked with -fPIE -pie -no-pie (effectively, -no-pie).

Native codegen and LLVM codegen support for -pi-style pie may be limited. The gcc manual states that -fPIC generates position-independent code suitable for dynamic linking and that -fPIE generates position-independent code only to be linked into executables. On amd64-linux, gcc will compile a .c file to different assembly with -fPIC and -fPIE. However, the native codegens do not generate different assembly for -pi-style pic and -pi-style pie. Similarly, the LLVM codegen does not generate different LLVM IR for -pi-style pic and -pi-style pie and llc is invoked with --relocation-model=pic for both -pi-style pic and -pi-style pie. Nonetheless, amd64-linux seems forgiving about linking native codegen and LLVM codegen object files with -pie.

As can be surmised, the non-PI vs PIC vs PIC is complex and there are certainly other variants (e.g., -fno-plt in gcc and clang, --relocation-model={dynamic-no-pic,ropi,rwpi,ropi-rwpi} in llc, dso_local runtime preemption specifier in LLVM IR).

The approach taken here is to make the default case follow the behavior of the C compiler (that was used to build the runtime) when compiling an executable. There may be (small) issues with downstream packaging and/or binary releases. In particular, if the C compiler used to build a package or binary release was not configured with --enable-default-pie, then libmlton.a will not be position-independent code and the native and LLVM codegens will not generate position-independent code. If this package or binary release is then used on a system where the C compiler is configured with --enable-default-pie, then the linking command will have no extra flags (but will have an implicit -pie) and the link will fail with an error. Nonetheless, the control afforded by -pi-style, -native-pic, -cc-opt, -llvm-llc-opt, and -link-opt should suffice to work around such situations. Moreover, a binary release could be "updated" to work with the new C compiler by updating the default::pie and default::pic entries of the target-specific constants file and overwriting libmlton.a with libmlton-pie.a.

Closes #191