Binder environment solve failing

[kafitzgerald]

First, thanks so much for all of the great resources you've put together!

In trying to launch the tutorial on the cloud (using the launch binder link), I noticed the environment creation on Binder seems to be failing (see details below). It looks like the [file for the specific version of the] package of interest is marked broken on conda-forge.

Waiting for build to start...
Picked Git content provider.
Cloning into '/tmp/repo2dockerw2qrr8o4'...
HEAD is now at 4e29164 [pre-commit.ci] pre-commit autoupdate (#253)
Building conda environment for python=3.11
Using CondaBuildPack builder
Step 1/50 : FROM docker.io/library/buildpack-deps:jammy
 ---> b6235acc32ce
Step 2/50 : ENV DEBIAN_FRONTEND=noninteractive
 ---> Using cache
 ---> 755cd1b1e980
Step 3/50 : RUN apt-get -qq update &&     apt-get -qq install --yes --no-install-recommends locales > /dev/null &&     apt-get -qq purge &&     apt-get -qq clean &&     rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> de9f0e4a05cd
Step 4/50 : RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen &&     locale-gen
 ---> Using cache
 ---> 92f56f191f0c
Step 5/50 : ENV LC_ALL=en_US.UTF-8     LANG=en_US.UTF-8     LANGUAGE=en_US.UTF-8
 ---> Using cache
 ---> 183e604467e3
Step 6/50 : ENV SHELL=/bin/bash
 ---> Using cache
 ---> 29d870378c0e
Step 7/50 : ARG NB_USER
 ---> Using cache
 ---> 3294b005ebab
Step 8/50 : ARG NB_UID
 ---> Using cache
 ---> fb48a5f7c4f6
Step 9/50 : ENV USER=${NB_USER}     HOME=/home/${NB_USER}
 ---> Using cache
 ---> 6f69538c7a7f
Step 10/50 : RUN groupadd         --gid ${NB_UID}         ${NB_USER} &&     useradd         --comment "Default user"         --create-home         --gid ${NB_UID}         --no-log-init         --shell /bin/bash         --uid ${NB_UID}         ${NB_USER}
 ---> Using cache
 ---> eda0cac86361
Step 11/50 : RUN apt-get -qq update &&     apt-get -qq install --yes --no-install-recommends        gettext-base        less        unzip        > /dev/null &&     apt-get -qq purge &&     apt-get -qq clean &&     rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 1ff96132f912
Step 12/50 : EXPOSE 8888
 ---> Using cache
 ---> 5a4faedfdb99
Step 13/50 : ENV APP_BASE=/srv
 ---> Using cache
 ---> 99cded126c26
Step 14/50 : ENV CONDA_DIR=${APP_BASE}/conda
 ---> Using cache
 ---> 2cdab4c7820c
Step 15/50 : ENV NB_PYTHON_PREFIX=${CONDA_DIR}/envs/notebook
 ---> Using cache
 ---> 1fe020df2668
Step 16/50 : ENV NPM_DIR=${APP_BASE}/npm
 ---> Using cache
 ---> 860da8dad357
Step 17/50 : ENV NPM_CONFIG_GLOBALCONFIG=${NPM_DIR}/npmrc
 ---> Using cache
 ---> 98eea01df937
Step 18/50 : ENV NB_ENVIRONMENT_FILE=/tmp/env/environment.lock
 ---> Using cache
 ---> 8cbdc0f7937a
Step 19/50 : ENV MAMBA_ROOT_PREFIX=${CONDA_DIR}
 ---> Using cache
 ---> 73d9bc52ee1b
Step 20/50 : ENV MAMBA_EXE=${CONDA_DIR}/bin/mamba
 ---> Using cache
 ---> 02bf661ccfc9
Step 21/50 : ENV CONDA_PLATFORM=linux-64
 ---> Using cache
 ---> e84d65f49470
Step 22/50 : ENV KERNEL_PYTHON_PREFIX=${NB_PYTHON_PREFIX}
 ---> Using cache
 ---> f3920c4a7f2a
Step 23/50 : ENV PATH=${NB_PYTHON_PREFIX}/bin:${CONDA_DIR}/bin:${NPM_DIR}/bin:${PATH}
 ---> Using cache
 ---> b4e06e806525
Step 24/50 : COPY --chown=1000:1000 build_script_files/-2fopt-2fvenv-2flib-2fpython3-2e11-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2factivate-2dconda-2esh-e70a7b /etc/profile.d/activate-conda.sh
 ---> Using cache
 ---> 7de2973d9eb0
Step 25/50 : COPY --chown=1000:1000 build_script_files/-2fopt-2fvenv-2flib-2fpython3-2e11-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2fenvironment-2epy-2d3-2e11-2dlinux-2d64-2elock-4045c1 /tmp/env/environment.lock
 ---> Using cache
 ---> d5e0338e1261
Step 26/50 : COPY --chown=1000:1000 build_script_files/-2fopt-2fvenv-2flib-2fpython3-2e11-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2finstall-2dbase-2denv-2ebash-6a6072 /tmp/install-base-env.bash
 ---> Using cache
 ---> 8b966e71692c
Step 27/50 : RUN TIMEFORMAT='time: %3R' bash -c 'time /tmp/install-base-env.bash' && rm -rf /tmp/install-base-env.bash /tmp/env
 ---> Using cache
 ---> c02f05a98b6e
Step 28/50 : RUN mkdir -p ${NPM_DIR} && chown -R ${NB_USER}:${NB_USER} ${NPM_DIR}
 ---> Using cache
 ---> c4693ba673c1
Step 29/50 : USER root
 ---> Using cache
 ---> 9ef8ec836ad9
Step 30/50 : ARG REPO_DIR=${HOME}
 ---> Using cache
 ---> 9bdf2484928b
Step 31/50 : ENV REPO_DIR=${REPO_DIR}
 ---> Using cache
 ---> d27859b6f602
Step 32/50 : RUN if [ ! -d "${REPO_DIR}" ]; then         /usr/bin/install -o ${NB_USER} -g ${NB_USER} -d "${REPO_DIR}";     fi
 ---> Using cache
 ---> 1046374f9614
Step 33/50 : WORKDIR ${REPO_DIR}
 ---> Using cache
 ---> b45884b871ce
Step 34/50 : RUN chown ${NB_USER}:${NB_USER} ${REPO_DIR}
 ---> Using cache
 ---> b830b434c664
Step 35/50 : ENV PATH=${HOME}/.local/bin:${REPO_DIR}/.local/bin:${PATH}
 ---> Using cache
 ---> 74f4cdfb8892
Step 36/50 : ENV CONDA_DEFAULT_ENV=${KERNEL_PYTHON_PREFIX}
 ---> Using cache
 ---> 12e1a5c324a3
Step 37/50 : COPY --chown=1000:1000 src/.binder/environment.yml ${REPO_DIR}/.binder/environment.yml
 ---> Using cache
 ---> ef3ef735f463
Step 38/50 : USER ${NB_USER}
 ---> Using cache
 ---> 61bad9173e36
Step 39/50 : RUN TIMEFORMAT='time: %3R' bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_PREFIX} --file ".binder/environment.yml" && time ${MAMBA_EXE} clean --all -f -y && ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} '
 ---> Running in 292066c511c8
Channels:
 - conda-forge
 - defaults
Platform: linux-64
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed

PackagesNotFoundError: The following packages are not available from current channels:

  - rdma-core==50.0=hd3aeb46_0

Current channels:

  - https://conda.anaconda.org/conda-forge/linux-64
  - https://repo.anaconda.com/pkgs/main/linux-64
  - https://repo.anaconda.com/pkgs/r/linux-64

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.


time: 60.367
 ---> Removed intermediate container 9d18d00af3eb
The command '/bin/sh -c TIMEFORMAT='time: %3R' bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_Pment.yml" && time ${MAMBA_EXE} clean --all -f -y && ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} '' retur

[scottyhq]

PackagesNotFoundError: The following packages are not available from current channels:

  - rdma-core==50.0=hd3aeb46_0

Thanks for pointing this out @kafitzgerald. Looks like this rdma package is a transitive dependency of libarrow (from mamba repoquery whoneeds rdma-core)

Hopefully just bumping to more recent versions of pyarrow fixes this, until then it seems running via codespaces still works b/c we use a lockfile for that and so it isn't aware of packages being later labeled as "broken" https://github.com/xarray-contrib/xarray-tutorial?tab=readme-ov-file#github-codespaces

[scottyhq]

huh... turns out repo2docker (& therefore mybinder.org doesn't support python 3.12 yet)
jupyterhub/repo2docker#1225