Ubuntu 20.04 -> R-4.0.3 -> Keras -> Tensorflow -> CUDA -> Quadro GP100

[dslate1]

Sorry for asking yet another toolchain compatibility question, but I'm stymied by so many "moving parts", and I hope someone can point me in the right direction. For years I have successfully run some Intel Xeon-based workstations (mostly offline) with Ubuntu 16.04 LTS, R, python 3.6, keras, tensorflow, and cuda (for my machine with the GPU). Now I wish to \move up to Ubuntu 20.04 and have set aside a test machine to see if I can get the whole toolchain working again. So far I seem to have Ubuntu 20.04.1 and R (I maintain my own R installation in /usr/local/) working pretty well. The missing pieces are keras, tensorflow, and cuda. On my Ubuntu 16.04 systems I had my own python 3.6 and cuda 10.0 installations. I was hoping that on Ubuntu 20.04, I could use the system versions of these: python 3.8 and cuda 10.1, but I ran into problems. For example, trying to import keras into python3 produces:

python3
Python 3.8.5 (default, Jul 28 2020, 12:59:40)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import keras
Using TensorFlow backend.
2021-01-13 13:39:20.195264: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-01-13 13:39:20.195294: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

So I gather that tensorflow 2.4.0, which python 3.8 installed, wants cuda 11.0, not the cuda 10.1 toolkit that comes with Ubuntu 20.04. From my researches on the web it also seems that earlier versions of tensorflow like 1.14.0 (which I use on my Ubuntu 16.04 machines), are not compatible with python 3.8. If I give up on trying to use python 3.8 and just go back to manually installing and maintaining python 3.6, with its keras and tensorflow versions, I could probably get everything working. But is there a better and simpler way to get an R and keras toolchain going on Ubuntu 20.04 that will support my Nvidia Quadro GP100 GPU?

Thanks for any help.

[dfalbel]

I would say that the simplest solution is to install CUDA 11. Multiple cuda installations are allowed to coexist so that wouldn't mess with other software.

Also I highly recommend not using the python system installation. I would rather use a miniconda installation as provided by reticulate: https://rstudio.github.io/reticulate/reference/install_miniconda.html

[dslate1]

Thanks dfalbel for your reply. Everything actually seems to be working now, although I'm still using the system python (I may try miniconda as you suggest). The last piece of the puzzle was to add this to my R code:

Sys.setenv( TF_FORCE_GPU_ALLOW_GROWTH = "true")

This fixes a "CUDNN_STATUS_INTERNAL_ERROR", which apparently is related to GPU memory pre-allocation.

Here is my current setup:

CPU: Intel Xeon W-1270
OS: Ubuntu 20.04.1 LTS
R: 4.0.3
Python: 3.8.5
Keras: 2.2.4
Tensorflow: 2.4.0
CUDA: 11.2
Cudnn: libcudnn8_8.0.5.39-1+cuda11.1_amd64
Nvidia driver: 460.27.04
GPU: Nvidia Quadro T2000 (just in my test machine, will eventually move to a Quadro GP 100)

[t-kalinowski]

Hello,

The code that installs tensorflow and keras has been refactored.
Please update to the latest versions of reticulate, tensorflow, and keras:

install.packages("remotes")
remotes::install_github(paste0("rstudio/", c("reticulate", "tensorflow", "keras")))
reticulate::install_miniconda() # skip this if you want to self-install conda or use venv
keras::install_keras()

Please let us know if after updating you are still encountering installation issues:
https://github.com/rstudio/tensorflow/issues/new