Configuration with float and complex

[marcfehling]

I noticed differences in the configuration when enabling complex numbers and float datatypes with the following set of parameters:

1. Trilinos_ENABLE_COMPLEX_DOUBLE=ON and Trilinos_ENABLE_COMPLEX_FLOAT=ON
2. Trilinos_ENABLE_COMPLEX=ON and Trilinos_ENABLE_FLOAT=ON

In the deal.II library, we advised its users to use parameter set 1 (in this readme).

I noticed that Trilinos will not provide instantions for std::complex<float> in Tpetra classes and functions with the parameter set 1. If I read the documentation correctly, both options schould work.

The affected packages seem to be kokkos-kernels and tpetra, so I'll mark @ndellingwood and @csiefer2. Maybe other packages are affected as well.

Steps to Reproduce

The following Cmake configurations were generated with Trilinos 15.0.0. I get the same results with Trilinos 14.4.0 and 13.4.1.

1. cmake -DTrilinos_ENABLE_ALL_PACKAGES=ON -DTPL_ENABLE_Netcdf=OFF -DTrilinos_ENABLE_COMPLEX_DOUBLE=ON -DTrilinos_ENABLE_COMPLEX_FLOAT=ON ..
   CMakeCache.txt

KokkosKernels_INST_COMPLEX_DOUBLE:BOOL=ON
KokkosKernels_INST_COMPLEX_FLOAT:BOOL=OFF
Tpetra_INST_COMPLEX_DOUBLE:BOOL=ON
Tpetra_INST_COMPLEX_FLOAT:BOOL=OFF

Trilinos_ENABLE_COMPLEX:BOOL=OFF
Trilinos_ENABLE_COMPLEX_DOUBLE:BOOL=ON
Trilinos_ENABLE_COMPLEX_FLOAT:BOOL=ON
Trilinos_ENABLE_FLOAT:BOOL=OFF

2. cmake -DTrilinos_ENABLE_ALL_PACKAGES=ON -DTPL_ENABLE_Netcdf=OFF -DTrilinos_ENABLE_COMPLEX=ON -DTrilinos_ENABLE_FLOAT=ON ..
   CMakeCache.txt

KokkosKernels_INST_COMPLEX_DOUBLE:BOOL=ON
KokkosKernels_INST_COMPLEX_FLOAT:BOOL=ON
Tpetra_INST_COMPLEX_DOUBLE:BOOL=ON
Tpetra_INST_COMPLEX_FLOAT:BOOL=ON

Trilinos_ENABLE_COMPLEX:BOOL=ON
Trilinos_ENABLE_COMPLEX_DOUBLE:BOOL=ON
Trilinos_ENABLE_COMPLEX_FLOAT:BOOL=ON
Trilinos_ENABLE_FLOAT:BOOL=ON

[ccober6]

@trilinos/kokkos-kernels and @trilinos/tpetra

[cgcgcg]

I think the issue is that the combination of Trilinos_ENABLE_COMPLEX_FLOAT=ON and Trilinos_ENABLE_FLOAT=OFF is not enough for several packages. (We need a magnitude type.) Therefore packages disable their complex<float> usage. If we're not already printing a warning somewhere we should add that.

[cgcgcg]

See #12755.

[cgcgcg]

@marcfehling Can you try again with the current develop branch?

[marcfehling]

Thank you for the quick response. I tried the following combinations:

• develop branch, Trilinos_ENABLE_COMPLEX_DOUBLE=ON, Trilinos_ENABLE_COMPLEX_FLOAT=ON

KokkosKernels_INST_COMPLEX_DOUBLE ON
KokkosKernels_INST_COMPLEX_FLOAT  OFF
KokkosKernels_INST_DOUBLE         ON
KokkosKernels_INST_FLOAT          OFF

Tpetra_INST_COMPLEX_DOUBLE        ON
Tpetra_INST_COMPLEX_FLOAT         OFF
Tpetra_INST_DOUBLE                ON
Tpetra_INST_FLOAT                 OFF

• develop branch, Trilinos_ENABLE_COMPLEX=ON, Trilinos_ENABLE_FLOAT=ON

KokkosKernels_INST_COMPLEX_DOUBLE ON
KokkosKernels_INST_COMPLEX_FLOAT  ON
KokkosKernels_INST_DOUBLE         ON
KokkosKernels_INST_FLOAT          ON

Tpetra_INST_COMPLEX_DOUBLE        ON
Tpetra_INST_COMPLEX_FLOAT         ON
Tpetra_INST_DOUBLE                ON
Tpetra_INST_FLOAT                 ON

The error message introduced with #12755 was not triggered in either configuration. So the configuration of these packages is consistent.

I guess the problem was rather that we expected that Trilinos_ENABLE_COMPLEX_FLOAT=ON enables complex float instantiations in all packages of the library, but this is not the case. You pointed that out in #12750 (comment).

[cgcgcg]

@marcfehling Whoops, sorry. I'll try this again. #12756

[marcfehling]

With Trilinos_ENABLE_COMPLEX_DOUBLE=ON and Trilinos_ENABLE_COMPLEX_FLOAT=ON I now get a warning:

CMake Warning at packages/tpetra/CMakeLists.txt:964 (MESSAGE):
  Trilinos_ENABLE_COMPLEX_FLOAT=ON, but Tpetra_INST_FLOAT=OFF.  Disabling
  Scalar=std::complex<float> in Tpetra by default.

This is helpful! Thank you!

Do you think a similar warning would be useful for kokkos-kernels as well?

[brian-kelley]

@marcfehling We do actually need one change in KokkosKernels, see kokkos/kokkos-kernels#2115.

Unlike Tpetra, it's fine to build KokkosKernels with a complex type enabled, and its corresponding real type not enabled.
But there was one case (graph unit tests) where a configuration like this caused undefined symbol errors late in the build. This PR adds a cmake check to error out at configure time instead.