configure script throws SEGFAULT under conditions involving multiple versions of PROJ libraries

[cboettig]

Describe the bug

./configure step in package installation throws the error:

#9 3500.3 checking GDAL: checking whether PROJ is available for running:... ./configure: line 3665: 33527 Segmentation fault      (core dumped) ./gdal_proj

You can see the full log associated with this error in this GitHub Actions build: https://github.com/boettiger-lab/nasa-topst-env-justice/actions/runs/6315364948/job/17147738622#step:4:9711, which should include all the details about the system necessary to reproduce.

To Reproduce

I believe this occurs whenever we have PROJ installed from source (e.g. in /usr/local) and also installed from the ubuntu repos by apt-get (e.g. in /usr/) . More precisely, this script provides a build recipe that will reproduce this problem, i.e. building this Dockerfile will reproduce the error

FROM ghcr.io/rocker-org/devcontainer/tidyverse:4.2
ADD https://raw.githubusercontent.com/rocker-org/rocker-versioned2/blob/d4d4f05f583e8e18c5f44f88b9373539b6c15dce/scripts/install_dev_osgeo.sh install_dev_osge
o.sh
RUN bash install_dev_osgeo.sh

Perhaps a failure on my part to run the appropriate linkers or path config? My installation recipe is based on the Docker Ubutnu 22.04 recipe from osgeo/gdal, which includes several packages from the ubuntu repos that implicitly depend on libproj-dev, such as libgeotiff-dev. Including any of these in the GDAL build results in the segfault crash. Most provide optional support for additional but less common drivers so for now I'm building without them. It's not entirely clear to me why gdal recommends libs like libgeotiff-dev for anyway since it can obviously read geotif without this. More generally I'd like to provide as 'complete' a build of GDAL as possible to users, but with an emphasis on support for the most recent and most performant drivers, legacy and proprietary drivers are of less interest anyway.

[rsbivand]

I do not use either ubuntu or containers for detailed work like this, I build GEOS, PROJ and GDAL (and libkea) from source (updated current Fedora release), and make sure that there is only one of each on the system. Your case is likely to be GDAL finding another version of PROJ than it was built with. I seem to recall seeing traffic on https://lists.osgeo.org/pipermail/gdal-dev/ or https://lists.osgeo.org/pipermail/proj/ which was not dissimilar.

The other question was whether geotiff must be external for GDAL, answer is no, as the gdal sources bundle geotiff sources (and those of a fair number of other smaller libraries). I do however use Fedora binaries for most other upstream dependencies of GDAL, where version drag isn't so important, or where security matters more (curl, etc.).

[cboettig]

Thanks for the help, this all makes sense to me. Don't mean to belabor the point, but just want to clarify this is does appear to be a (very minor) bug in sf configuration, and in no way impacts gdal or other osgeo libraries (i.e. why I'm reporting it here and not in osgeo). Happy to be corrected if I'm wrong about this!

I believe the configure script in sf should detect the correct version of proj, e.g. the source build in /usr/local/lib, not the older system libraries in /usr/lib. If for some reason this combination is untenable, configure should throw an appropriate error message as it does in most other cases, rather than Segmentation fault core dump. I do not believe there is a fundamental problem with having both versions of proj installed in the same system -- note that the OSGeo gdal configuration (via cmake) has no problem compiling when both the older system PROJ and newer user-compiled PROJ exist on the same system, as seen in the osgeo dockerfile linked above. I think this issue is specific to sf configuration installation, gdal itself is installing operating fine. GDAL itself is being built with the latest PROJ and is not confused by the presence of the libproj-dev binary from the repos. only sf install gets confused.

If we avoid installing any of the libraries that bring in libproj-dev, we can indeed install gdal from source and sf installs fine. We simply get a GDAL that lacks support for these optional drivers and whatever other features are associated with those dependencies (in my experimentation, these also include libspatialite-dev, librasterlite-dev, maybe others. Also there's obviously many other packages with a reverse dependency on libproj-dev. In my opinion, a user installing any such library that brings in this reverse dependency shouldn't result in breaking the ability for sf to compile from source, even though most users can probably live without sqlite spatial driver etc.

[edzer]

Leaving aside whether this should be considered a bug or not, it is a well-known issue mentioned in the README. It would be great if the configure step would tell you what the problem is, but I don't see how that can be realised (and not be a wild guess).

IIRC the experiments I did when this came to light it had more to do with what R does with LD_LIBRARY_PATH than what sf does, as the cause. Maybe setting that variable the way R does during the configure steps could solve it, I don't know.

[cboettig]

thanks @edzer , I'll poke around with LD_LIBRARY_PATH.

I thought there may be a way I can do a staged build where I can successfully remove Ubuntu's libproj-dev etc after building GDAL from source (binding the newest proj but with optional dependencies that bring in repo proj). Unsurprisingly though sf can't be built then since removing libproj-dev remove any dependent libs like libspatiallite-dev, and when sf tries to compile against gdal it complains about not being able to find the dev libs it was built against!

[edzer]

I think this has been sufficiently diagnosed and discussed at #844 ; please reopen if you disagree or have new insights.