diff --git a/NEWS.md b/NEWS.md index e8f0777dbf..5955997f4f 100644 --- a/NEWS.md +++ b/NEWS.md @@ -6,7 +6,7 @@ We have a new release [Recommenders 0.7.0](https://github.com/microsoft/recommen In this, we have changed the names of the folders which contain the source code, so that they are more informative. This implies that you will need to change any import statements that reference the recommenders package. Specifically, the folder `reco_utils` has been renamed to `recommenders` and its subfolders have been renamed according to [issue 1390](https://github.com/microsoft/recommenders/issues/1390). -The recommenders package now supports three types of environments: [venv](https://docs.python.org/3/library/venv.html) and [virtualenv](https://virtualenv.pypa.io/en/latest/index.html#) with Python 3.6, [conda](https://docs.conda.io/projects/conda/en/latest/glossary.html?highlight=environment#conda-environment) with Python versions 3.6 and 3.7. +The recommenders package now supports three types of environments: [venv](https://docs.python.org/3/library/venv.html), [virtualenv](https://virtualenv.pypa.io/en/latest/index.html#) and [conda](https://docs.conda.io/projects/conda/en/latest/glossary.html?highlight=environment#conda-environment) with Python versions 3.6 and 3.7. We have also added new evaluation metrics: _novelty, serendipity, diversity and coverage_ (see the [evalution notebooks](examples/03_evaluate/README.md)). diff --git a/README.md b/README.md index 27f650606b..f9fad2d13b 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ We have changed the names of the folders which contain the source code, so that The previous release ([0.6.0](https://github.com/microsoft/recommenders/releases/tag/0.6.0)) is compatible with the old style of naming of modules. -The recommenders package now supports three types of environments: [venv](https://docs.python.org/3/library/venv.html) and [virtualenv](https://virtualenv.pypa.io/en/latest/index.html#) with Python 3.6, [conda](https://docs.conda.io/projects/conda/en/latest/glossary.html?highlight=environment#conda-environment) with Python versions 3.6 and 3.7. +The recommenders package now supports three types of environments: [venv](https://docs.python.org/3/library/venv.html), [virtualenv](https://virtualenv.pypa.io/en/latest/index.html#) and [conda](https://docs.conda.io/projects/conda/en/latest/glossary.html?highlight=environment#conda-environment) with Python versions 3.6 and 3.7. We have also added new evaluation metrics: _novelty, serendipity, diversity and coverage_ (see the [evalution notebooks](examples/03_evaluate/README.md)). @@ -42,8 +42,7 @@ For a more detailed overview of the repository, please see the documents on the Please see the [setup guide](SETUP.md) for more details on setting up your machine locally, on a [data science virtual machine (DSVM)](https://azure.microsoft.com/en-gb/services/virtual-machines/data-science-virtual-machines/) or on [Azure Databricks](SETUP.md#setup-guide-for-azure-databricks). The installation of the recommenders package has been tested with -- Python version 3.6 and [venv](https://docs.python.org/3/library/venv.html) or [virtualenv](https://virtualenv.pypa.io/en/latest/index.html#) -- Python versions 3.6, 3.7 and [conda](https://docs.conda.io/projects/conda/en/latest/glossary.html?highlight=environment#conda-environment) +- Python versions 3.6, 3.7 and [venv](https://docs.python.org/3/library/venv.html), [virtualenv](https://virtualenv.pypa.io/en/latest/index.html#) or [conda](https://docs.conda.io/projects/conda/en/latest/glossary.html?highlight=environment#conda-environment) and currently does not support version 3.8 and above. It is recommended to install the package and its dependencies inside a clean environment (such as [conda](https://docs.conda.io/projects/conda/en/latest/glossary.html?highlight=environment#conda-environment), [venv](https://docs.python.org/3/library/venv.html) or [virtualenv](https://virtualenv.pypa.io/en/latest/index.html#)). @@ -68,11 +67,6 @@ pip install --upgrade pip pip install --upgrade setuptools pip install recommenders[examples] ``` -In the case of conda, you also need to -```bash -conda install numpy-base -``` -after the pip installation. 4. Register your (conda or virtual) environment with Jupyter: diff --git a/SETUP.md b/SETUP.md index b9a47db28c..7592ce955e 100644 --- a/SETUP.md +++ b/SETUP.md @@ -151,6 +151,7 @@ create the file `%RECO_ENV%\etc\conda\deactivate.d\env_vars.bat` and add: It is straightforward to install the recommenders package within a [virtual environment](https://docs.python.org/3/library/venv.html). However, setting up CUDA for use with a GPU can be cumbersome. We thus recommend setting up [Nvidia docker](https://github.com/NVIDIA/nvidia-docker) and running the virtual environment within a container, as the most convenient way to do this. +In the following `3.6` should be replaced with the Python version you are using. # Start docker daemon if not running sudo dockerd & diff --git a/conda.md b/conda.md index a6e41f7c7d..2c2ee29bbf 100644 --- a/conda.md +++ b/conda.md @@ -1,6 +1,6 @@ One possible way to use the repository is to run all the recommender utilities directly from a local copy of the source code (without building the package). This requires installing all the necessary dependencies from Anaconda and PyPI. -To this end we provide a script, [generate_conda_file.py](tools/generate_conda_file.py), to generate a conda-environment yaml file which you can use to create the target environment using Python 3.6 with all the correct dependencies. +To this end we provide a script, [generate_conda_file.py](tools/generate_conda_file.py), to generate a conda-environment yaml file which you can use to create the target environment using Python 3.6 or 3.7 with all the correct dependencies. Assuming the repo is cloned as `Recommenders` in the local system, to install **a default (Python CPU) environment**: diff --git a/recommenders/README.md b/recommenders/README.md index d520b3d6f2..7d3f4030a1 100644 --- a/recommenders/README.md +++ b/recommenders/README.md @@ -22,11 +22,6 @@ To install core utilities, CPU-based algorithms, and dependencies pip install --upgrade pip pip install recommenders ``` -In the case of conda, you also need to -```bash -conda install numpy-base -``` -after the pip installation. ## Optional Dependencies diff --git a/setup.py b/setup.py index 0aba6982be..52c59814fa 100644 --- a/setup.py +++ b/setup.py @@ -22,7 +22,7 @@ version += ".post" + str(int(time.time())) install_requires = [ - "numpy>=1.14", + "numpy>=1.19", "pandas>1.0.3,<2", "scipy>=1.0.0,<2", "tqdm>=4.31.1,<5", @@ -43,7 +43,8 @@ "pyyaml>=5.4.1,<6", "requests>=2.0.0,<3", "cornac>=1.1.2,<2", - "scikit-surprise>=0.19.1,<=1.1.1", + # For Surprise, specify the tarball in order to avoid incompatibilities of compiled .pyx files with numpy versions < 1.20 + "scikit-surprise@https://files.pythonhosted.org/packages/97/37/5d334adaf5ddd65da99fc65f6507e0e4599d092ba048f4302fe8775619e8/scikit-surprise-1.1.1.tar.gz", "retrying>=1.3.3", ] diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile index 588ebacba9..3d7525617f 100644 --- a/tools/docker/Dockerfile +++ b/tools/docker/Dockerfile @@ -21,7 +21,7 @@ RUN if [ "${VIRTUAL_ENV}" != "conda" ] && [ "${VIRTUAL_ENV}" != "venv" ] && [ "$ RUN apt-get update && \ apt-get install -y curl build-essential cmake RUN if [ "${VIRTUAL_ENV}" = "conda" ] ; then apt-get install -y libpython3.7; fi -RUN if [ "${VIRTUAL_ENV}" = "venv" ] || [ "${VIRTUAL_ENV}" = "virtualenv" ]; then apt-get install -y libpython3.6; fi +RUN if [ "${VIRTUAL_ENV}" = "venv" ] || [ "${VIRTUAL_ENV}" = "virtualenv" ]; then apt-get install -y libpython3.7; fi # Install Anaconda ARG ANACONDA="https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh" @@ -38,19 +38,19 @@ SHELL ["/bin/bash", "--login", "-c"] # Python version supported by recommenders RUN if [ "${VIRTUAL_ENV}" = "conda" ] ; then conda install python=3.7; fi -RUN if [ "${VIRTUAL_ENV}" = "venv" ] ; then apt-get -y install python3.6; \ +RUN if [ "${VIRTUAL_ENV}" = "venv" ] ; then apt-get -y install python3.7; \ apt-get -y install python3-pip; \ - apt-get -y install python3.6-venv; fi -RUN if [ "${VIRTUAL_ENV}" = "virtualenv" ] ; then apt-get -y install python3.6; \ + apt-get -y install python3.7-venv; fi +RUN if [ "${VIRTUAL_ENV}" = "virtualenv" ] ; then apt-get -y install python3.7; \ apt-get -y install python3-pip; \ - python3.6 -m pip install --user virtualenv; fi + python3.7 -m pip install --user virtualenv; fi # Activate the virtual environment -RUN if [ "${VIRTUAL_ENV}" = "venv" ] ; then python3.6 -m venv --system-site-packages $HOME/${VIRTUAL_ENV}; \ +RUN if [ "${VIRTUAL_ENV}" = "venv" ] ; then python3.7 -m venv --system-site-packages $HOME/${VIRTUAL_ENV}; \ source $HOME/${VIRTUAL_ENV}/bin/activate; \ pip install --upgrade pip; \ pip install --upgrade setuptools; fi -RUN if [ "${VIRTUAL_ENV}" = "virtualenv" ] ; then python3.6 -m virtualenv $HOME/${VIRTUAL_ENV}; \ +RUN if [ "${VIRTUAL_ENV}" = "virtualenv" ] ; then python3.7 -m virtualenv $HOME/${VIRTUAL_ENV}; \ source $HOME/${VIRTUAL_ENV}/bin/activate; \ pip install --upgrade pip; \ pip install --upgrade setuptools; fi @@ -62,10 +62,7 @@ FROM base AS cpu RUN if [ "${VIRTUAL_ENV}" = "venv" ] || [ "${VIRTUAL_ENV}" = "virtualenv" ]; then source $HOME/${VIRTUAL_ENV}/bin/activate; \ pip install recommenders[xlearn,examples]; fi -# Note that "conda install numpy-base" needs to be called after the pip installation -# so that numpy works smoothly with other dependencies in python 3.7 -RUN if [ "${VIRTUAL_ENV}" = "conda" ] ; then pip install recommenders[xlearn,examples]; \ - conda install -n base numpy-base; fi +RUN if [ "${VIRTUAL_ENV}" = "conda" ] ; then pip install recommenders[xlearn,examples]; fi ############### @@ -84,8 +81,7 @@ ENV JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64" \ # Install dependencies in virtual environment RUN if [ "${VIRTUAL_ENV}" = "venv" ] || [ "${VIRTUAL_ENV}" = "virtualenv" ]; then source $HOME/${VIRTUAL_ENV}/bin/activate; \ pip install recommenders[spark,xlearn,examples]; fi -RUN if [ "${VIRTUAL_ENV}" = "conda" ] ; then pip install recommenders[spark,xlearn,examples]; \ - conda install -n base numpy-base; fi +RUN if [ "${VIRTUAL_ENV}" = "conda" ] ; then pip install recommenders[spark,xlearn,examples]; fi ########### @@ -105,7 +101,7 @@ RUN if [ "${VIRTUAL_ENV}" != "conda" ] && [ "${VIRTUAL_ENV}" != "venv" ] && [ "$ RUN apt-get update && \ apt-get install -y curl build-essential cmake RUN if [ "${VIRTUAL_ENV}" = "conda" ] ; then apt-get install -y libpython3.7; fi -RUN if [ "${VIRTUAL_ENV}" = "venv" ] || [ "${VIRTUAL_ENV}" = "virtualenv" ]; then apt-get install -y libpython3.6; fi +RUN if [ "${VIRTUAL_ENV}" = "venv" ] || [ "${VIRTUAL_ENV}" = "virtualenv" ]; then apt-get install -y libpython3.7; fi # Install Anaconda ARG ANACONDA="https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh" @@ -121,28 +117,27 @@ SHELL ["/bin/bash", "--login", "-c"] # Python version supported by recommenders RUN if [ "${VIRTUAL_ENV}" = "conda" ] ; then conda install python=3.7; fi -RUN if [ "${VIRTUAL_ENV}" = "venv" ] ; then apt-get -y install python3.6; \ +RUN if [ "${VIRTUAL_ENV}" = "venv" ] ; then apt-get -y install python3.7; \ apt-get -y install python3-pip; \ - apt-get -y install python3.6-venv; fi -RUN if [ "${VIRTUAL_ENV}" = "virtualenv" ] ; then apt-get -y install python3.6; \ + apt-get -y install python3.7-venv; fi +RUN if [ "${VIRTUAL_ENV}" = "virtualenv" ] ; then apt-get -y install python3.7; \ apt-get -y install python3-pip; \ - python3.6 -m pip install --user virtualenv; fi + python3.7 -m pip install --user virtualenv; fi # Activate the virtual environment -RUN if [ "${VIRTUAL_ENV}" = "venv" ] ; then python3.6 -m venv --system-site-packages $HOME/${VIRTUAL_ENV}; \ +RUN if [ "${VIRTUAL_ENV}" = "venv" ] ; then python3.7 -m venv --system-site-packages $HOME/${VIRTUAL_ENV}; \ source $HOME/${VIRTUAL_ENV}/bin/activate; \ pip install --upgrade pip; \ pip install --upgrade setuptools; \ pip install recommenders[gpu,xlearn,examples]; fi -RUN if [ "${VIRTUAL_ENV}" = "virtualenv" ] ; then python3.6 -m virtualenv $HOME/${VIRTUAL_ENV}; \ +RUN if [ "${VIRTUAL_ENV}" = "virtualenv" ] ; then python3.7 -m virtualenv $HOME/${VIRTUAL_ENV}; \ source $HOME/${VIRTUAL_ENV}/bin/activate; \ pip install --upgrade pip; \ pip install --upgrade setuptools; \ pip install recommenders[gpu,xlearn,examples]; fi RUN if [ "${VIRTUAL_ENV}" = "conda" ] ; then \ - pip install recommenders[gpu,xlearn,examples] -f https://download.pytorch.org/whl/cu100/torch_stable.html && \ - conda install -n base numpy-base; fi + pip install recommenders[gpu,xlearn,examples] -f https://download.pytorch.org/whl/cu100/torch_stable.html ; fi ############ @@ -168,10 +163,6 @@ RUN if [ "${VIRTUAL_ENV}" = "venv" ] || [ "${VIRTUAL_ENV}" = "virtualenv" ]; the pip install recommenders[all]; fi RUN if [ "${VIRTUAL_ENV}" = "conda" ] ; then pip install recommenders[all]; fi -# It is not clear why but force-reinstall is needed at this place, -# otherwise numpy-base does not appear in conda list -RUN if [ "${VIRTUAL_ENV}" = "conda" ] ; then conda update --force-reinstall -y -n base numpy-base; fi - ############# # Final Stage