Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TensorFlow 2.6 upgrade #1565

Merged
merged 66 commits into from
Dec 6, 2021
Merged
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
66e5d3c
Change TF and Cuda versions
anargyri Sep 22, 2021
3396f2c
Fix numpy version
anargyri Sep 27, 2021
153099e
Explain numpy version
anargyri Sep 27, 2021
d67955e
Merge branch 'staging' into andreas/tf
anargyri Oct 7, 2021
b572bbe
Merge branch 'staging' into andreas/tf
anargyri Oct 11, 2021
e8bc100
Merge branch 'staging' into andreas/tf
anargyri Oct 11, 2021
68af516
Specify no-binary install for Surprise
anargyri Oct 11, 2021
aafbe1a
Update CUDA version
anargyri Oct 11, 2021
899b43a
Modifications made by tf_upgrade_v2 script provided by TF + fix for i…
anargyri Oct 12, 2021
cb60a2f
Modifications made by tf_upgrade_v2 script provided by TF + fix for i…
anargyri Oct 12, 2021
9f39411
Replace tf.contrib.estimator.build_supervised_input_receiver_fn_from_…
anargyri Oct 12, 2021
edf9b4c
Fix tf_utils
anargyri Oct 13, 2021
a98750b
Merge branch 'andreas/tf' of github.com:microsoft/recommenders into a…
anargyri Oct 13, 2021
10cd5a7
reapply TF2
anargyri Oct 13, 2021
e1560f9
Fix wide and deep tuning notebook
anargyri Oct 13, 2021
d2b03b7
Replace tf.contrib.training.HParams with custom class
anargyri Oct 18, 2021
2efcac0
Fix DKN model
anargyri Oct 19, 2021
e5ec621
Fix XDeepFM hparams
anargyri Oct 19, 2021
f3d0437
Fix tf version
anargyri Oct 20, 2021
ba702e5
Replace tf.logging in notebooks
anargyri Oct 20, 2021
1b83ffe
Fix tf.summary in notebook
anargyri Oct 20, 2021
d258bfe
Hparams for newsrec
anargyri Oct 20, 2021
1d1b211
Fix LSTUR run
anargyri Oct 25, 2021
f205d15
Replace batch_dot with einsum in self attention layer
anargyri Oct 25, 2021
df7ed26
Change Adam in RBM
anargyri Oct 25, 2021
c29bc41
Fix RNNs
anargyri Oct 26, 2021
de2b3f7
Remove TF addons
anargyri Oct 27, 2021
f5d28e4
Remove LazyAdam optimizer
anargyri Oct 28, 2021
b6ceeec
Fix RNN cells import
anargyri Oct 28, 2021
68979f1
Fix sum.py
anargyri Oct 28, 2021
06c93ba
Fix asserts
anargyri Oct 28, 2021
0e57c04
Get fully connected layer from tf-slim
anargyri Oct 28, 2021
26eb236
Update conda script
anargyri Oct 28, 2021
2fd0726
Merge branch 'staging' into andreas/tf
anargyri Oct 29, 2021
6f1d85e
Remove pip installation cell from VAE notebooks
anargyri Oct 29, 2021
774c8eb
Changes required for pymanopt
anargyri Nov 2, 2021
93fb27b
Remove inference from SLIRec notebook
anargyri Nov 4, 2021
1d94f13
Update TF version in notebooks
anargyri Nov 4, 2021
8503c51
Replace pytorch location for cuda 11
anargyri Nov 8, 2021
a743eba
Add no binary pip install option in ADO test pipeline
anargyri Nov 9, 2021
7d36296
Changed version to 1.0.0
anargyri Nov 9, 2021
d5491b7
Edit nvidia docker image name
anargyri Nov 11, 2021
f325a2c
Java version for virtual env
anargyri Nov 11, 2021
11e4c86
Resolved flake8 issues and blacked
anargyri Nov 11, 2021
a8e316d
Whitespace
anargyri Nov 11, 2021
1f9c24a
Add keras in setup.py
anargyri Nov 11, 2021
bd32d2f
Fix Surprise issue in test environments
anargyri Nov 15, 2021
fde38fb
Undo 'Fix Surprise issue in test environments'
anargyri Nov 16, 2021
679d50c
Change heading in sequential notebook
anargyri Nov 16, 2021
7688dc1
Move pandera to core dependencies in order to fix movielens
anargyri Nov 16, 2021
2523447
Fix issue with RLRMC test
anargyri Nov 16, 2021
3ef366a
Remove numpy 1.19 from GitHub workflows
anargyri Nov 17, 2021
ecff395
Fix mock movielens duplicates
anargyri Nov 19, 2021
dd0b29e
Fix mock movielens test
anargyri Nov 24, 2021
8f32dda
Add blank line
anargyri Nov 24, 2021
23ae6e0
Move surprise to experimental
anargyri Nov 26, 2021
55a8199
Update tests and docs for experimental
anargyri Nov 29, 2021
383c475
Update CI tests
anargyri Nov 30, 2021
5c18fb6
Update GitHb actions
anargyri Nov 30, 2021
dfc8eac
No binary option in tox.ini
anargyri Nov 30, 2021
1dc5d5c
Remove experimental from GitHub actions
anargyri Nov 30, 2021
292f139
Disable experimental tests from workflow
anargyri Nov 30, 2021
2bce47a
Wrap surprise import in test
anargyri Nov 30, 2021
2eb8f93
Fix bug in tf_utils
anargyri Nov 30, 2021
bfbb4b0
Ignore coverage errors while reading from source files
anargyri Dec 6, 2021
3eae4b1
Merge branch 'staging' into andreas/tf
anargyri Dec 6, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ On Windows you will need [Microsoft C++ Build Tools](https://visualstudio.micros
```bash
pip install --upgrade pip
pip install --upgrade setuptools
pip install recommenders[examples]
pip install --no-cache --no-binary scikit-surprise recommenders[examples]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--no-cache --no-binary scikit-surprise how can we remove these from the installation? before we didn't have it, we do we need it now?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we need this because we want pip to build Surprise from the source (I changed setup.py in an earlier PR to link to the source on GitHub). Otherwise we get the conflict between Surprise and Numpy 1.19 (array size mismatch).
Maybe there is a way to do this inside setup.py, any ideas?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NicolasHug hey Nicolas, we are upgrading to TF 2 in recommenders, we are having a conflict between surprise and TF 2.

What @anargyri is doing at the moment is adding the tarball and also we have to install --no-cache --no-binary scikit-surprise before recommenders.

Do you know what is the best way to install surprise with TF2 and add it in the extra_requires like we had before?

Copy link
Collaborator Author

@anargyri anargyri Nov 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@miguelgfierro See this issue and discussion
@NicolasHug one possible alternative would be to publish another version of Surprise on PyPI that has .pyx files compiled with numpy version 1.19.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi all, sorry for the troubles - any chance you could just rely on conda-forge to install surprise binaries directly instead of building from source?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, we'd really appreciate it 🙏

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your support @NicolasHug!

Copy link
Collaborator

@NicolasHug NicolasHug Nov 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had time to look into this a little. I'm a bit surprised that we're getting this "size changed" issue. According to numpy/numpy#18709, this issue happens when one is trying to run code that was compiled against a more recent numpy version than what is currently installed.

But since I released surprise 1.1.1 in July 2020, the latest possible version I could have possibly used is numpy 1.19 (which was released one month earlier).

So I'm not quite sure why you're getting this error with 1.19 TBH. Is it possible that you're installing / building surprise with e.g. numpy 1.20/1.21 and then downgrade to numpy 1.19? If that's the case, then the error is expected.

On my end, I'm only able to reproduce the issue if I build on (e.g.) 1.20 and then try to import surprise with 1.19. If I build with 1.17, I can import surprise with all versions all the way up to 1.21.


Anyway, it's possible that I'm missing something. Since 1.17 seems to work fine upwards (and since it's older and thus less constraining than 1.19 in terms of dependency satisfaction), I uploaded a new release where Cython files were compiled against 1.17. I only updated it to the pypi-test channel for now. Before I push it for good on the official PyPI channel, could you please check that it actually works for you guys? You can install from pypi-test with e.g.

pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple scikit-surprise

Note: when trying that locally you might need a --no-cache-dir flag to make sure stuff gets re-compiled when you switch numpy versions.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @NicolasHug, I still get the same behavior with the new release, however. As before, pip installing without the no-binary option leads to the error, using this option works fine.
According to the issue you linked, it seems that others have been affected by this too. It looks like this behavior is not really about the version of numpy itself but has to do with the Cython compilation. So that everything works when python setup.py is run during the pip installation (which is the effect of the no-binary option). Gensim has observed the same.
Maybe we should wait until there is a fix by numpy or cython. Until then, we could require users to use --no-binary. Thanks for trying anyway!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NicolasHug @miguelgfierro I did some further investigation. To summarize,

pip install numpy==1.19 cython
pip install scikit-surprise

installs but then importing surprise raises ValueError.

pip install numpy==1.19 cython
pip install scikit-surprise@https://github.com/NicolasHug/Surprise/archive/refs/tags/v1.1.1.tar.gz

works correctly.
So there is a workaround for people who want to use numpy <=1.19 along with surprise.
In our case, it still doesn't work via our setup.py (unless you pre-install cython), but this must be some pip / setuptools bug.

```

4. Register your (conda or virtual) environment with Jupyter:
Expand Down
10 changes: 5 additions & 5 deletions SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,13 +151,13 @@ create the file `%RECO_ENV%\etc\conda\deactivate.d\env_vars.bat` and add:

It is straightforward to install the recommenders package within a [virtual environment](https://docs.python.org/3/library/venv.html). However, setting up CUDA for use with a GPU can be cumbersome. We thus
recommend setting up [Nvidia docker](https://github.com/NVIDIA/nvidia-docker) and running the virtual environment within a container, as the most convenient way to do this.
In the following `3.6` should be replaced with the Python version you are using and `11` should be replaced with the appropriate Java version.
In the following `3.6` should be replaced with the Python version you are using and `8` should be replaced with the appropriate Java version.

# Start docker daemon if not running
sudo dockerd &
# Pull the image from the Nvidia docker hub (https://hub.docker.com/r/nvidia/cuda) that is suitable for your system
# E.g. for Ubuntu 18.04 do
sudo docker run --gpus all -it --rm nvidia/cuda:10.0-cudnn7-runtime-ubuntu18.04
sudo docker run --gpus all -it --rm nvidia/cuda:11.2.2-cudnn8-runtime-ubuntu18.04

# Within the container:

Expand All @@ -167,8 +167,8 @@ In the following `3.6` should be replaced with the Python version you are using
apt-get -y install python3.6-venv
apt-get -y install libpython3.6-dev
apt-get -y install cmake
apt-get install -y libgomp1 openjdk-11-jre
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
apt-get install -y libgomp1 openjdk-8-jre
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64

python3.6 -m venv --system-site-packages /venv
source /venv/bin/activate
Expand All @@ -179,7 +179,7 @@ In the following `3.6` should be replaced with the Python version you are using
export PYSPARK_DRIVER_PYTHON=/venv/bin/python
export PYSPARK_PYTHON=/venv/bin/python

pip install recommenders[all]
pip install --no-cache --no-binary scikit-surprise recommenders[all]

If you prefer to use [virtualenv](https://virtualenv.pypa.io/en/latest/index.html#) instead of venv, you may follow the above steps, except you will need to replace the line

Expand Down
4 changes: 2 additions & 2 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

To setup the documentation, first you need to install the dependencies of the full environment. For it please follow the [SETUP.md](../SETUP.md). Then type:

conda create -n reco_full python=3.6 cudatoolkit=10.0 "cudnn>=7.6"
conda create -n reco_full -c conda-forge python=3.6 cudatoolkit=11.2 cudnn=8.1
conda activate reco_full
pip install .[all]
pip install --no-cache --no-binary scikit-surprise .[all]
pip install sphinx_rtd_theme


Expand Down
26 changes: 18 additions & 8 deletions examples/00_quick_start/dkn_MIND.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -62,20 +62,28 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 1,
"metadata": {
"pycharm": {
"is_executing": false
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/anaconda/envs/tf2/lib/python3.7/site-packages/papermill/iorw.py:50: FutureWarning: pyarrow.HadoopFileSystem is deprecated as of 2.0.0, please use pyarrow.fs.HadoopFileSystem instead.\n",
" from pyarrow import HadoopFileSystem\n"
]
},
Comment on lines +72 to +79
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this error is weird, it seems that it comes from papermill but we don't import papermill on this notebook.

I see this appears in all notebooks, it would be good to clear this error. This can be done by executing twice this cell or by upgrading the papermill version to one that has pyarrow.fs.HadoopFileSystem

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually this cell imports scrapbook, which imports papermill. The versions of papermill and scrapbook are the latest ones. I think the only thing we can do is wait for scrapbook to change this in a later version.

{
"name": "stdout",
"output_type": "stream",
"text": [
"System version: 3.6.11 | packaged by conda-forge | (default, Aug 5 2020, 20:09:42) \n",
"System version: 3.7.11 (default, Jul 27 2021, 14:32:16) \n",
"[GCC 7.5.0]\n",
"Tensorflow version: 1.15.2\n"
"Tensorflow version: 2.6.1\n"
miguelgfierro marked this conversation as resolved.
Show resolved Hide resolved
]
}
],
Expand Down Expand Up @@ -359,10 +367,12 @@
],
"metadata": {
"celltoolbar": "Tags",
"interpreter": {
"hash": "3a9a0c422ff9f08d62211b9648017c63b0a26d2c935edc37ebb8453675d13bb5"
},
Comment on lines +370 to +372
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this hash?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure. It looks like metadata added by VSCode, maybe it identifies the Python interpreter VSCode uses.
This hash appears in staging too (in other notebooks such as als_deep_dive.ipynb). When you select another kernel this hash changes.

"kernelspec": {
"display_name": "Python (reco_gpu)",
"language": "python",
"name": "reco_gpu"
"display_name": "Python 3.7.11 64-bit ('tf2': conda)",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
Expand All @@ -374,7 +384,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.11"
"version": "3.7.11"
},
"pycharm": {
"stem_cell": {
Expand All @@ -388,4 +398,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}
24 changes: 17 additions & 7 deletions examples/00_quick_start/lstur_MIND.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -75,13 +75,21 @@
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/anaconda/envs/tf2/lib/python3.7/site-packages/papermill/iorw.py:50: FutureWarning: pyarrow.HadoopFileSystem is deprecated as of 2.0.0, please use pyarrow.fs.HadoopFileSystem instead.\n",
" from pyarrow import HadoopFileSystem\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"System version: 3.6.11 | packaged by conda-forge | (default, Aug 5 2020, 20:09:42) \n",
"System version: 3.7.11 (default, Jul 27 2021, 14:32:16) \n",
"[GCC 7.5.0]\n",
"Tensorflow version: 1.15.2\n"
"Tensorflow version: 2.6.1\n"
]
}
],
Expand Down Expand Up @@ -529,10 +537,12 @@
],
"metadata": {
"celltoolbar": "Tags",
"interpreter": {
"hash": "3a9a0c422ff9f08d62211b9648017c63b0a26d2c935edc37ebb8453675d13bb5"
},
"kernelspec": {
"display_name": "Python (reco_gpu)",
"language": "python",
"name": "reco_gpu"
"display_name": "Python 3.7.11 64-bit ('tf2': conda)",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
Expand All @@ -544,9 +554,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.11"
"version": "3.7.11"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
}
26 changes: 18 additions & 8 deletions examples/00_quick_start/naml_MIND.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -72,16 +72,24 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/anaconda/envs/tf2/lib/python3.7/site-packages/papermill/iorw.py:50: FutureWarning: pyarrow.HadoopFileSystem is deprecated as of 2.0.0, please use pyarrow.fs.HadoopFileSystem instead.\n",
" from pyarrow import HadoopFileSystem\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"System version: 3.6.11 | packaged by conda-forge | (default, Aug 5 2020, 20:09:42) \n",
"System version: 3.7.11 (default, Jul 27 2021, 14:32:16) \n",
"[GCC 7.5.0]\n",
"Tensorflow version: 1.15.2\n"
"Tensorflow version: 2.6.1\n"
]
}
],
Expand Down Expand Up @@ -526,10 +534,12 @@
],
"metadata": {
"celltoolbar": "Tags",
"interpreter": {
"hash": "3a9a0c422ff9f08d62211b9648017c63b0a26d2c935edc37ebb8453675d13bb5"
},
"kernelspec": {
"display_name": "Python (reco_gpu)",
"language": "python",
"name": "reco_gpu"
"display_name": "Python 3.7.11 64-bit ('tf2': conda)",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
Expand All @@ -541,9 +551,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.11"
"version": "3.7.11"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
}
24 changes: 17 additions & 7 deletions examples/00_quick_start/npa_MIND.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -75,13 +75,21 @@
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/anaconda/envs/tf2/lib/python3.7/site-packages/papermill/iorw.py:50: FutureWarning: pyarrow.HadoopFileSystem is deprecated as of 2.0.0, please use pyarrow.fs.HadoopFileSystem instead.\n",
" from pyarrow import HadoopFileSystem\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"System version: 3.6.11 | packaged by conda-forge | (default, Aug 5 2020, 20:09:42) \n",
"System version: 3.7.11 (default, Jul 27 2021, 14:32:16) \n",
"[GCC 7.5.0]\n",
"Tensorflow version: 1.15.2\n"
"Tensorflow version: 2.6.1\n"
]
}
],
Expand Down Expand Up @@ -504,10 +512,12 @@
],
"metadata": {
"celltoolbar": "Tags",
"interpreter": {
"hash": "3a9a0c422ff9f08d62211b9648017c63b0a26d2c935edc37ebb8453675d13bb5"
},
"kernelspec": {
"display_name": "Python (reco_gpu)",
"language": "python",
"name": "reco_gpu"
"display_name": "Python 3.7.11 64-bit ('tf2': conda)",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
Expand All @@ -519,9 +529,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.11"
"version": "3.7.11"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
}
24 changes: 17 additions & 7 deletions examples/00_quick_start/nrms_MIND.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -75,13 +75,21 @@
"scrolled": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/anaconda/envs/tf2/lib/python3.7/site-packages/papermill/iorw.py:50: FutureWarning: pyarrow.HadoopFileSystem is deprecated as of 2.0.0, please use pyarrow.fs.HadoopFileSystem instead.\n",
" from pyarrow import HadoopFileSystem\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"System version: 3.6.11 | packaged by conda-forge | (default, Aug 5 2020, 20:09:42) \n",
"System version: 3.7.11 (default, Jul 27 2021, 14:32:16) \n",
"[GCC 7.5.0]\n",
"Tensorflow version: 1.15.2\n"
"Tensorflow version: 2.6.1\n"
]
}
],
Expand Down Expand Up @@ -523,10 +531,12 @@
],
"metadata": {
"celltoolbar": "Tags",
"interpreter": {
"hash": "3a9a0c422ff9f08d62211b9648017c63b0a26d2c935edc37ebb8453675d13bb5"
},
"kernelspec": {
"display_name": "Python (reco_gpu)",
"language": "python",
"name": "reco_gpu"
"display_name": "Python 3.7.11 64-bit ('tf2': conda)",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
Expand All @@ -538,9 +548,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.11"
"version": "3.7.11"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
}
28 changes: 19 additions & 9 deletions examples/00_quick_start/rbm_movielens.ipynb

Large diffs are not rendered by default.

Loading