67 install probatus in colab notebooks

.github/workflows/publish_to_pypi.yml

@@ -20,8 +20,7 @@ jobs:
         pip install pytest
     - name: Make sure unit tests succeed
       run: |
-        pip install .
-        pip install -r requirements_test.txt
+        pip install ".[all]"
         pytest
     - name: Build package & publish to PyPi
       env:

.github/workflows/unit_tests.yml

@@ -36,12 +36,7 @@ jobs:
         SKIP_LIGHTGBM: ${{ matrix.SKIP_LIGHTGBM }}
       run: |
         pip install --upgrade setuptools pip
-        pip install pytest
-        pip install pytest-cov
-        pip install pyflakes
-        pip install mypy
-        pip install .
-        pip install -r tests/requirements_test.txt
+        pip install ".[all]"
         pytest --cov=probatus/binning --cov=probatus/metric_volatility --cov=probatus/sample_similarity --cov=probatus/stat_tests --cov=probatus/utils --cov=probatus/interpret/ --ignore==tests/interpret/test_inspector.py --cov-report=xml
         pyflakes probatus
         mypy probatus --ignore-missing-imports

CONTRIBUTING.md

@@ -47,24 +47,28 @@ https://github.com/jonschlinkert/markdown-toc
 ## Code development
 
 ### Project workflow
-For each new version of Probatus we loosely follow the steps:
+
+For each new version of `probatus` we loosely follow the steps:
+
 * Set up a next release milestone, with the release date
 * Plan issues that are picked up in a given milestone, and assign contributors
-* Deploy new version of Probatus
+* Deploy new version of `probatus`
 
-Milestones are created by the Maintainer, in collaboration with contributors, based on contributors availability and priority of the backlog.
+Milestones are created by the maintainer, in collaboration with contributors, based on contributors availability and priority of the backlog.
 
 ### Developing new functionality
 
-For new functionality that can be developed in Probatus, we follow the workflow:
+For new functionality that can be developed in `probatus`, we follow the workflow:
+
 1. Make an issue on the board, clearly describe the idea and discuss it in the comments with other contributors.
 2. Plan development of the feature, as part of one or multiple releases.
 3. Implement the feature: feature code, unit tests and documentation (including notebook in `docs/` with tutorial on how to use the feature). Each of these can be implemented as a separate Merge Request.
 4. Maintainer deploys new version of Probatus and expose it and new docs to the users.
 
 ### Ad-hoc development
 
-For any other work picked up in the project, e.g. refactoring code, fixing bugs, refining docs we loosely follow this workflow:  
+For any other work picked up in the project, e.g. refactoring code, fixing bugs, refining docs we loosely follow this workflow: 
+
 1. Make an issue on the board, clearly describe the idea and discuss it in the comments with other contributors.
 2. Assign a contributor, and implement the change using Merge Request functionality.
 3. Maintainer deploys new version of Probatus and expose it and new docs to the users.
@@ -74,20 +78,24 @@ For any other work picked up in the project, e.g. refactoring code, fixing bugs,
 We are always glad if new contributors want to spend time improving the package, do not hesitate to reach out to us.
 
 For new contributors, a good starting point would be the following:
-* Make an issues, describing and discussing ideas for improvements
+
+* Make an issue, describing and discussing ideas for improvements
 * Look over issues that do not have an assignee, and assign yourself. Best to start with simple ones, e.g. improving docs, improving code style etc. This way you can get to know the package better.
-* Create a Merge Request to master, that implements the issue that you are assigned to. Start with something small :)
+* Create a Pull Request to master, that implements the issue that you are assigned to. Start with something small :)
 
 ## Technical Standards
 
+To get started with a local development environment, install `probatus` as an editable package:
+
+```shell
+pip install -e ".[all]"
+```
+
 ### Python
 
 * Use Python 3.6 or higher depends on availability in your dev/prod environment.
-* Use environments (conda). To create one, run `conda env create -f requirements.yml`.
-* Keep track of dependencies in `requirements.yml` and run `conda install --file requirements.yml` to install them. Remember to notify your colleagues of the change in requirements since they'll need to conda install them.
 * Line length is 120
 * Follow [PEP8](http://pep8.org/) as closely as possible (except line length)
-* Use ```__name__ == '__main__'```, otherwise Sphinx will execute your script upon building.
 * Use the [google docstring format](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/)
 
 ```python
@@ -108,6 +116,7 @@ def func(arg1, arg2):
 ```
 
 ### Code structure
+
 * Model validation modules assume that trained models passed for validation are developed in scikit-learn framework (have predict_proba and other standard functions), or follows scikit-learn API e.g. XGBoost.
 * Every python file used for model validation, needs to be in `/probatus/`
 * Class structure for a given module should have a base class, and specific functionality classes that inherit from base. If a given module implements only single way of computing the output, the base class is not required. 
@@ -129,13 +138,14 @@ other functions. This make the code more readable, and easier to test.
 
 
 ### Probatus API
+
 Classes follow the probatus API structure:
 * Each class implements fit(), compute() and fit_compute() methods. Fit is used to fit object with provided data (unless no fit is required), and compute calculates the output e.g. DataFrame with report for the user. Lastly, fit_compute applies one after the other.
 * If applicable, plot() method presents user with the appropriate graphs.
 * For compute(), and plot(), check if the object is fitted first.
 
-
 ### Unit tests
+
 Unit tests follow the following convention:
 * Use pytest framework
 * Structure tests in `tests/` folder the same way as `probatus/` modules. Each test file and function should start its name with `test_`.
@@ -177,6 +187,7 @@ We develop the docs in the following way:
 ## Code Reviews
 
 Code reviews are very important: 
+
 - improve knowledge of both reviewer and reviewee
 - improve consistency of code, easier to work on in the future
 - shorter development time as you catch bugs earlier
@@ -245,10 +256,6 @@ Create feature X
 This is a longer description of what the commit contains. See #123
 ```
 
-## Issue tracking
-
-Issue tracking is recommended even for solo projects. In this project we use Gitlab issue board for issue tracking.
-
 
 ## Versioning and Deployment
 

README.md

@@ -1,4 +1,4 @@
-<img src="docs/img/logo_large.png" width="120" align="right">
+<img src="https://github.com/ing-bank/probatus/raw/main/docs/img/logo_large.png" width="120" align="right">
 
 [![pytest](https://github.com/ing-bank/probatus/workflows/Release/badge.svg)](https://github.com/ing-bank/probatus/actions?query=workflow%3A%22Release%22)
 [![pytest](https://github.com/ing-bank/probatus/workflows/Development/badge.svg)](https://github.com/ing-bank/probatus/actions?query=workflow%3A%22Development%22)

docs/tutorials/nb_binning.ipynb

@@ -9,6 +9,16 @@
     "[![open in colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ing-bank/probatus/blob/master/docs/tutorials/nb_binning.ipynb)"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!pip install probatus"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,
@@ -628,7 +638,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.4"
+   "version": "3.8.3-final"
   }
  },
  "nbformat": 4,

docs/tutorials/nb_custom_scoring.ipynb

@@ -20,6 +20,16 @@
     "Let's prepare some data:"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!pip install probatus"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,

docs/tutorials/nb_distribution_statistics.ipynb

@@ -36,6 +36,16 @@
     "## Setup"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!pip install probatus"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,

docs/tutorials/nb_metric_volatility.ipynb

@@ -18,6 +18,16 @@
     "### Setup"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!pip install probatus"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 14,

docs/tutorials/nb_sample_similarity.ipynb

@@ -31,6 +31,16 @@
     "## Setup"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!pip install probatus"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,

docs/tutorials/nb_shap_dependence.ipynb

@@ -14,6 +14,16 @@
     "First let's import some dependencies and set some settings:"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!pip install probatus"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 14,

docs/tutorials/nb_shap_feature_elimination.ipynb

@@ -43,6 +43,17 @@
     "- 1 feature with missing values"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!pip install probatus\n",
+    "!pip install lightgbm"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,

docs/tutorials/nb_shap_model_interpreter.ipynb

@@ -31,6 +31,16 @@
     "Let's assume we want to analyse the following model:"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!pip install probatus"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,

setup.py

@@ -1,10 +1,49 @@
 import setuptools
 import os
 
-
 def read(fname):
     return open(os.path.join(os.path.dirname(__file__), fname)).read()
 
+base_packages = [
+    "scikit-learn>=0.22.2",
+    "pandas>=1.0.0",
+    "matplotlib>=3.1.1",
+    "scipy>=1.4.0",
+    "joblib>=0.13.2",
+    "tqdm>=4.41.0",
+    "shap>=0.38.1",
+    "numpy>=1.19.0"
+]
+
+dev_dep = [
+    "flake8>=3.8.3",
+    "black>=19.10b0",
+    "pre-commit>=2.5.0",
+    "mypy>=0.770",
+    "flake8-docstrings>=1.4.0",
+    "pytest>=6.0.0",
+    "pytest-cov>=2.10.0",
+    "pyflakes",
+    "seaborn>=0.9.0",
+    "joblib>=0.13.2",
+    "lightgbm>=3.1.0",
+    "jupyter>=1.0.0",
+    "tabulate>=0.8.7",
+    "nbconvert>=6.0.7",
+]
+
+docs_dep = [
+    "mkdocs-material>=6.1.0",
+    "mkdocs-git-revision-date-localized-plugin>=0.7.2",
+    "mkdocs-git-authors-plugin>=0.3.2",
+    "mkdocs-table-reader-plugin>=0.4.1",
+    "mkdocs-enumerate-headings-plugin>=0.4.3",
+    "mkdocs-awesome-pages-plugin>=2.4.0",
+    "mkdocs-minify-plugin>=0.3.0",
+    "mknotebooks>=0.6.2",
+    "mkdocs-print-site-plugin>=0.8.2",
+    "mkdocs-markdownextradata-plugin>=0.1.9",
+]
 
 setuptools.setup(
     name="probatus",
@@ -22,16 +61,10 @@ def read(fname):
         "License :: OSI Approved :: MIT License",
         "Operating System :: OS Independent",
     ],
-    install_requires=[
-        "scikit-learn>=0.22.2",
-        "pandas>=1.0.0",
-        "matplotlib>=3.1.1",
-        "scipy>=1.4.0",
-        "joblib>=0.13.2",
-        "tqdm>=4.41.0",
-        "shap>=0.38.1",
-        "numpy>=1.19.0"
-    ],
+    install_requires=base_packages,
+    extras_require={
+        "all": base_packages + dev_dep + docs_dep,
+    },
     url="https://github.com/ing-bank/probatus",
     zip_safe=False,
 )