sf installation problem under Mac OS 11 - Big Sur

[codyevers]

Upgrading to Mac OS 11 - Big Sur appears to have created an issue in installing and loading 'sf'. The issue seems specific to the 'rgdal' package, which is looking for the library "libpq.5.dylib" which no longer exists in "/usr/lib/". Found another post on the same problem here: R-macos/recipes#14, though no solutions yet. Anyone have thoughts on how to resolve this?

Error reported in R when install sf and/or rgdal:

unable to load shared object '/Library/Frameworks/R.framework/Versions/4.0/Resources/library/sf/libs/sf.so':
dlopen(/Library/Frameworks/R.framework/Versions/4.0/Resources/library/sf/libs/sf.so, 6): Library not loaded: /usr/lib/libpq.5.dylib
Referenced from: /Library/Frameworks/R.framework/Versions/4.0/Resources/library/sf/libs/sf.so
Reason: image not found

[rsbivand]

https://twitter.com/opencpu/status/1326902988334657537?s=19 A known issue, and good advice not to upgrade until those who understand Apple (I do not) confirm that it is safe to do so. Following R-sig-mac is also advised https://stat.ethz.ch/mailman/listinfo/r-sig-mac. Roger Bivand Falsensvei 32 5063 Bergen fre. 13. nov. 2020, 20:24 skrev codyevers <[email protected]>:

…

Upgrading to Mac OS 11 - Big Sur appears to have created an issue in installing and loading 'sf'. The issue seems specific to the 'rgdal' package, which is looking for the library "libpq.5.dylib" which no longer exists in "/usr/lib/". Found another post on the same problem here: R-macos/recipes#14 <R-macos/recipes#14>, though no solutions yet. Anyone have thoughts on how to resolve this? Error reported in R when install sf and/or rgdal: unable to load shared object '/Library/Frameworks/R.framework/Versions/4.0/Resources/library/sf/libs/sf.so': dlopen(/Library/Frameworks/R.framework/Versions/4.0/Resources/library/sf/libs/sf.so, 6): Library not loaded: /usr/lib/libpq.5.dylib Referenced from: /Library/Frameworks/R.framework/Versions/4.0/Resources/library/sf/libs/sf.so Reason: image not found — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1536>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACNZ3BABGYG3AIHBKOF2M6LSPWBWTANCNFSM4TU62MQQ> .

[abuabara]

These steps worked for me ...

PART 1: Using Terminal

1. Install brew

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

2. Install the packages:

brew install pkg-config
brew install gdal proj geos

2.1) Not sure it's necessary ... I included it on terminal.

export LDFLAGS="-L/usr/local/opt/libpq/lib"
export CPPFLAGS="-I/usr/local/opt/libpq/include"

PART 2: Using R

install.packages("rgeos", repos="http://R-Forge.R-project.org", type="source")
install.packages("rgdal", repos="http://R-Forge.R-project.org", type="source")
library(devtools)
install_github("r-spatial/sf", configure.args = "--with-proj-lib=/usr/local/lib/")

Restart R and tidycensus, rgdal, rgeos, and sf will load properly.

[codyevers]

Fantastic @abuabara. I can confirm that this fixed the problem on my end, and I assume will work for anyone else running into this issue after installing Mac OS 11 - Big Sur. Thanks for the quick fix!

[franzkappa]

Hi there. I get this when running the last line of code:

> install_github("r-spatial/sf", configure.args = "--with-proj-lib=/usr/local/lib/")
Downloading GitHub repo r-spatial/sf@HEAD
Error: Failed to install 'sf' from GitHub:
  Could not find tools necessary to compile a package
Call `pkgbuild::check_build_tools(debug = TRUE)` to diagnose the problem.
Warning message:
1 components of `...` were not used.

We detected these problematic arguments:
* `configure.args`

Did you misspecify an argument?

I then tried this, but without success:

> options(buildtools.check = function(action) TRUE )
> install_github("r-spatial/sf", configure.args = "--with-proj-lib=/usr/local/lib/")
Downloading GitHub repo r-spatial/sf@HEAD
✓  checking for file ‘/private/var/folders/np/x2bs8n2s6d996fkt00vssqnw0000gn/T/RtmpABGiuf/remotes10853f3de773/r-spatial-sf-af12048/DESCRIPTION’ ...
─  preparing ‘sf’:
✓  checking DESCRIPTION meta-information ...
─  cleaning src
─  running ‘cleanup’
─  checking for LF line-endings in source and make files and shell scripts (454ms)
─  checking for empty or unneeded directories
─  building ‘sf_0.9-7.tar.gz’
   
* installing *source* package ‘sf’ ...
** using staged installation
configure: CC: /usr/local/clang4/bin/clang
configure: CXX: /usr/local/clang4/bin/clang++ -std=gnu++11
checking for gdal-config... /usr/local/bin/gdal-config
checking gdal-config usability... yes
configure: GDAL: 3.2.0
checking GDAL version >= 2.0.1... yes
checking for gcc... /usr/local/clang4/bin/clang
checking whether the C compiler works... no
configure: error: in `/private/var/folders/np/x2bs8n2s6d996fkt00vssqnw0000gn/T/RtmpJKE5hT/R.INSTALL10d0ad0d386/sf':
configure: error: C compiler cannot create executables
See `config.log' for more details
ERROR: configuration failed for package ‘sf’
* removing ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library/sf’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library/sf’
Error: Failed to install 'sf' from GitHub:
  (converted from warning) installation of package ‘/var/folders/np/x2bs8n2s6d996fkt00vssqnw0000gn/T//RtmpABGiuf/file108579264f74/sf_0.9-7.tar.gz’ had non-zero exit status

[thierrygosselin]

This is my brew less version... to fix the install of sf on macOS 11.0.1 (Big Sur)

Note: change -j56 below in the code to the number of CPU on your machine

macOS terminal

udunits

cd ~/Downloads
curl -L https://www.unidata.ucar.edu/downloads/udunits/udunits-2.2.26.tar.gz | tar xf -
cd udunits*
./configure && make -j56 && sudo make install
cd ..
sudo rm -R udunits*

sqlite

cd ~/Downloads
curl -L https://www.sqlite.org/2020/sqlite-autoconf-3330000.tar.gz | tar xf -
cd sqlite-autoconf*
./configure && make -j56 && sudo make install
cd ..
sudo rm -R sqlite-autoconf*

libtiff

cd ~/Downloads
curl -L https://download.osgeo.org/libtiff/tiff-4.1.0.tar.gz | tar xf -
cd tiff*
./configure && make -j56 && sudo make install
cd ..
sudo rm -R tiff*

proj

cd ~/Downloads
curl -L  http://download.osgeo.org/proj/proj-7.2.0.tar.gz | tar xf -
cd proj*
./configure --libdir=/usr/local/lib && make -j56 && sudo make install
cd ..
sudo rm -R proj*

gdal

export LDFLAGS="-L/usr/local/lib"
export CPPFLAGS="-I/usr/local/include"

cd ~/Downloads
curl -L https://github.com/OSGeo/gdal/releases/download/v3.2.0/gdal-3.2.0.tar.gz | tar xf -
cd gdal*
./configure --prefix=/usr/local --libdir=/usr/local/lib --with-proj=/usr/local && make -j56 && sudo make install
cd ..
sudo rm -R gdal*

geos

cd ~/Downloads
git clone https://git.osgeo.org/gitea/geos/geos.git
cd geos*
./autogen.sh
./configure --libdir=/usr/local/lib && make -j56 && sudo make install
cd ..
sudo rm -R geos*

Back in R console

install.packages("rgeos", repos="http://R-Forge.R-project.org", type="source")
install.packages("rgdal", repos="http://R-Forge.R-project.org", type="source")
devtools::install_github("r-spatial/sf", configure.args = "--with-proj-lib=/usr/local")

[jeroen]

@thierrygosselin you should be really careful using sudo make install because there is no way to uninstall those libs, they will linger on the system forever and may start conflicting with other things. Also if you build gdal like this, you are missing all of the drivers that are provided by 3rd party dependency libraries.

If you're going to install GDAL manually, I much recommend using brew or another package manager, so that you can upgrade or uninstall the gdal files when you don't need them anymore.

[thierrygosselin]

@jeroen not sure for GDAL, I'm barely using it, I need sf to run adegenet, and for 99% of the other genomics related packages and software I use, I see only advantages of using sudo make install:

• no permission issues
• Installing in /usr/local/ is in the PATH by defaults, that's a plus for novice users
• as for remaining in the system forever, I have stories with homebrew that required erase/install of the main drive. I'm usually advertising to do an erase/install every year or so... With external hard drive and important personal files / preferences synced to the cloud these days, it makes the operation effortless, but that's probably outside the scope of this thread...
• I'm no fan of homebrew and MacPorts, as you might have guess...
• most importantly, it's not folders protected by macOS System Integrity Protection so there is no harm done by writing/moving/deleting them.
• 3rd party dependency libraries: I have no problem with packages that depends on it, but please make correction if you have a better way to install/compile without homebrew, I'll update my pipeline with it!

Best,
Thierry

[markegge]

Thanks @abuabara - ran with a slight simplification. These steps worked for me ...

PART 1: Using Terminal

Using Homebrew, install gdal:

brew install pkg-config
brew install gdal proj geos

(Homebrew installs gdal, proj, and geos into a Cellar, and then symlinks these into the /usr/local/lib folder. You can ignore the postgres formula "keg only" warning produced when libpq is installed.)

PART 2: Using R

devtools::install_github("r-spatial/sf", configure.args="--with-proj-lib=/usr/local/lib/")

[angel-langdon]

Thanks @abuabara - ran with a slight simplification. These steps worked for me ...

PART 1: Using Terminal

Using Homebrew, install gdal:

brew install pkg-config
brew install gdal proj geos

(Homebrew installs gdal, proj, and geos into a Cellar, and then symlinks these into the /usr/local/lib folder. You can ignore the postgres formula "keg only" warning produced when libpq is installed.)

PART 2: Using R

devtools::install_github("r-spatial/sf", configure.args = "--with-proj-lib=/usr/local/lib/")

Restart R and sf will load properly.

In my case no need for restarting R.

[jeroen]

This problem is now fixed on CRAN. You can now install the sf and rgdal packages from CRAN in the regular way on MacOS big sur.

[Yizhi0102]

I am having problems when installing packages in R
It shows:

install.packages("rgeos", repos="http://R-Forge.R-project.org", type="source")
试开URL’http://R-Forge.R-project.org/src/contrib/rgeos_0.5-7.tar.gz'
Content type 'application/x-gzip' length 276293 bytes (269 KB)
==================================================
downloaded 269 KB

• installing source package ‘rgeos’ ...
  ** using staged installation
  configure: CC: clang -mmacosx-version-min=10.13
  configure: CXX: clang++ -mmacosx-version-min=10.13 -std=gnu++14
  configure: rgeos: 0.5-7
  checking for /usr/bin/svnversion... no
  cat: inst/SVN_VERSION: No such file or directory
  configure: svn revision:
  checking for geos-config... no
  no
  configure: error: geos-config not found or not executable.
  ERROR: configuration failed for package ‘rgeos’
• removing ‘/Library/Frameworks/R.framework/Versions/4.1/Resources/library/rgeos’
  Warning in install.packages :
  installation of package ‘rgeos’ had non-zero exit status

The downloaded source packages are in
‘/private/var/folders/78/9n9r7b590z9_vy2trjt6nnhc0000gn/T/RtmpDtHvAI/downloaded_packages’
install.packages("rgdal", repos="http://R-Forge.R-project.org", type="source")
试开URL’http://R-Forge.R-project.org/src/contrib/rgdal_1.5-25.tar.gz'
Content type 'application/x-gzip' length 4379698 bytes (4.2 MB)
==============
downloaded 1.2 MB

Warning in install.packages :
downloaded length 1245184 != reported length 4379698
Warning in install.packages :
URL 'https://download.r-forge.r-project.org/src/contrib/rgdal_1.5-25.tar.gz': Timeout of 60 seconds was reached
Error in download.file(url, destfile, method, mode = "wb", ...) :
download from 'http://R-Forge.R-project.org/src/contrib/rgdal_1.5-25.tar.gz' failed
Warning in install.packages :
download of package ‘rgdal’ failed
library(devtools)
载入需要的程辑包：usethis
install_github("r-spatial/sf", configure.args = "--with-proj-lib=/usr/local/lib/")
Downloading GitHub repo r-spatial/sf@HEAD
✓ checking for file ‘/private/var/folders/78/9n9r7b590z9_vy2trjt6nnhc0000gn/T/RtmpDtHvAI/remotes4a0181ffb26/r-spatial-sf-7a0ec2c/DESCRIPTION’ ...
─ preparing ‘sf’:
✓ checking DESCRIPTION meta-information ...
─ cleaning src
─ running ‘cleanup’
─ checking for LF line-endings in source and make files and shell scripts (443ms)
─ checking for empty or unneeded directories
─ building ‘sf_1.0-3.tar.gz’

• installing source package ‘sf’ ...
  ** using staged installation
  configure: CC: clang -mmacosx-version-min=10.13
  configure: CXX: clang++ -mmacosx-version-min=10.13 -std=gnu++11
  checking for gdal-config... no
  no
  configure: error: gdal-config not found or not executable.
  ERROR: configuration failed for package ‘sf’
• removing ‘/Library/Frameworks/R.framework/Versions/4.1/Resources/library/sf’
  Warning message:
  In i.p(...) :
  installation of package ‘/var/folders/78/9n9r7b590z9_vy2trjt6nnhc0000gn/T//RtmpDtHvAI/file4a049f26abe/sf_1.0-3.tar.gz’ had non-zero exit status

But I am sure that gdal proj geos have been installed by brew

Warning: pkg-config 0.29.2_3 is already installed and up-to-date.
To reinstall 0.29.2_3, run:
brew reinstall pkg-config
Warning: gdal 3.3.1_3 is already installed and up-to-date.
To reinstall 3.3.1_3, run:
brew reinstall gdal
Warning: proj 8.1.1 is already installed and up-to-date.
To reinstall 8.1.1, run:
brew reinstall proj
Warning: geos 3.9.1 is already installed and up-to-date.
To reinstall 3.9.1, run:
brew reinstall geos

So, what should I do next?

[rsbivand]

Do not use source packages unless you can handle installation yourself, based on for example https://github.com/r-spatial/sf/#macos. In the rgeos case, geos-config is not found, so if you must install from R-Forge (why, CRAN has the same source package and a binary for your platform?), you'll need to provide configure argument: --with-geos-config=GEOS_CONFIG the location of geos-config. Similarly for the others. But really you should avoid source installs unless you have very good reason.

[aourednik]

@abuabara
Your solution lead me on the right path but it does not work anymore on the arm64 (M1) architecture.
See #1848

[cheuerde]

install_github("r-spatial/sf", configure.args = "--with-proj-lib=/usr/local/lib/")

For a homebrew installation of proj, use this:

install_github("r-spatial/sf", configure.args = "--with-proj-lib=/opt/homebrew/lib")

[rsbivand]

No, it always depends on the architecture, so without stating which architecture, this advice is not accurate, since on M1 rosetta can be brought into play with the wrong path. Always best to use CRAN binaries only, also when installing from source.