Merge pull request #232 from quantumblacklabs/release/0.15.1

[KED-1003] Release 0.15.1

.coveragerc

@@ -1,7 +1,8 @@
 [report]
 fail_under=100
 show_missing=True
-omit = kedro/template*
+omit =
+    kedro/template*
 exclude_lines =
     pragma: no cover
     raise NotImplementedError

.github/ISSUE_TEMPLATE/bug-report.md

@@ -1,6 +1,6 @@
 ---
-name: Bug Report
-about: Create a report to help us improve
+name: Bug report
+about: If something isn't working
 title: '<Title>'
 labels: 'Issue: Bug Report'
 assignees: ''
@@ -16,7 +16,7 @@ How has this bug affected you? What were you trying to accomplish?
 ## Steps to Reproduce
 1. [First Step]
 2. [Second Step]
-3. [and so on...]
+3. [And so on...]
 
 ## Expected Result
 Tell us what should happen.

.github/ISSUE_TEMPLATE/feature-request.md

@@ -1,6 +1,6 @@
 ---
-name: Feature Request
-about: Suggest a feature or enhancement for this project
+name: Feature request
+about: Let us know if you have a feature request or enhancement
 title: '<Title>'
 labels: 'Issue: Feature Request'
 assignees: ''

.github/ISSUE_TEMPLATE/thank-you.md

@@ -0,0 +1,21 @@
+---
+name: Say thank you
+about: Tell us how you use Kedro and help us grow a community
+title: '<Title>'
+labels: 'Issue: Thank You'
+assignees: ''
+
+---
+
+## Let us know
+If you (or your company) are using Kedro - please let us know. We'd love to hear from you!
+
+## Making Kedro even better
+If you would like to help Kedro - any of the following is greatly appreciated.
+
+- [ ] Give the repository a star
+- [ ] Help out with issues
+- [ ] Review pull requests
+- [ ] Blog about Kedro
+- [ ] Make tutorials
+- [ ] Give talks

.gitignore

@@ -150,3 +150,4 @@ docs/tmp-build-artifacts
 docs/build
 docs/node_modules
 docs/source/04_user_guide/source/.ipynb
+tests/template/fake_project/

.readthedocs.yml

@@ -9,6 +9,7 @@ version: 2
 sphinx:
   builder: html
   configuration: docs/conf.py
+  fail_on_warning: true
 
 # Build documentation with MkDocs
 # mkdocs:

CONTRIBUTING.md

@@ -65,6 +65,9 @@ def count_truthy(elements: List[Any]) -> int:
 
 > *Note:* We only accept contributions under the Apache 2.0 license and you should have permission to share the submitted code.
 
+Please note that each code file should have a licence header, include the content of [`legal_header.txt`](https://github.com/quantumblacklabs/kedro/blob/master/legal_header.txt).
+There is an automated check to verify that it exists. The check will highlight any issues and suggest a solution.
+
 ### Branching conventions
 We use a branching model that helps us keep track of branches in a logical, consistent way. All branches should have the hyphen-separated convention of: `<type-of-change>/<short-description-of-change>` e.g. `contrib/io-dataset`
 

Makefile

@@ -1,11 +1,13 @@
 install: build-docs
+	rm -rf kedro/html
 	cp -r docs/build/html kedro
 	pip install .
 
 clean:
-	rm -rf build dist docs/build kedro/html pip-wheel-metadata
+	rm -rf build dist docs/build kedro/html pip-wheel-metadata .mypy_cache .pytest_cache
 	find . -regex ".*/__pycache__" -exec rm -rf {} +
 	find . -regex ".*\.egg-info" -exec rm -rf {} +
+	pre-commit clean || true
 
 install-pip-setuptools:
 	python -m pip install -U "pip>=18.0, <19.0" "setuptools>=38.0, <39.0" wheel

README.md

@@ -70,6 +70,10 @@ For more detailed installation instructions, including how to setup Python virtu
   - [Kedro-Docker](https://github.com/quantumblacklabs/kedro-docker), a tool for packaging and shipping Kedro projects within containers
 - Kedro can be deployed locally, on-premise and cloud (AWS, Azure and GCP) servers, or clusters (EMR, Azure HDinsight, GCP and Databricks)
 
+## What are the main Kedro building blocks?
+
+You can find the overview of Kedro architecture [here](https://kedro.readthedocs.io/en/latest/06_resources/02_architecture_overview.html).
+
 ## How do I use Kedro?
 
 Our [documentation](https://kedro.readthedocs.io/en/latest/) explains:
@@ -118,3 +122,7 @@ pip install kedro -U
 ## What licence do you use?
 
 Kedro is licensed under the [Apache 2.0](https://github.com/quantumblacklabs/kedro/blob/master/LICENSE.md) License.
+
+## We're hiring!
+
+Do you want to be part of the team that builds Kedro and [other great products](https://quantumblack.com/labs) at QuantumBlack? If so, you're in luck! QuantumBlack is currently hiring Software Engineers who love using data to drive their decisions. Take a look at [our open positions](https://www.quantumblack.com/careers/current-openings#content) and see if you're a fit.

RELEASE.md

@@ -1,3 +1,28 @@
+# Release 0.15.1
+
+## Major features and improvements
+* Extended `versioning` support to cover the tracking of environment setup, code and datasets.
+* Added the following datasets:
+  - `FeatherLocalDataSet` in `contrib` for usage with Pandas. (by [@mdomarsaleem](https://github.com/mdomarsaleem))
+* Added `get_last_load_version` and `get_last_save_version` to `AbstractVersionedDataSet`.
+* Implemented `__call__` method on `Node` to allow for users to execute `my_node(input1=1, input2=2)` as an alternative to `my_node.run(dict(input1=1, input2=2))`.
+* Added new `--from-inputs` run argument.
+
+## Bug fixes and other changes
+* Fixed a bug in `load_context()` not loading context in non-Kedro Jupyter Notebooks.
+* Fixed a bug in `ConfigLoader.get()` not listing nested files for `**`-ending glob patterns.
+* Fixed a logging config error in Jupyter Notebook.
+* Updated documentation in `03_configuration` regarding how to modify the configuration path.
+* Documented the architecture of Kedro showing how we think about library, project and framework components.
+* `extras/kedro_project_loader.py` renamed to `extras/ipython_loader.py` and now runs any IPython startup scripts without relying on the Kedro project structure.
+* Fixed TypeError when validating partial function's signature
+* After a node failure during a pipeline run, a resume command will be suggested in the logs. This command will not work if the required inputs are MemoryDataSets.
+
+## Breaking changes to the API
+
+## Thanks for supporting contributions
+[Omar Saleem](https://github.com/mdomarsaleem), [Mariana Silva](https://github.com/marianansilva), [Anil Choudhary](https://github.com/aniryou), [Craig](https://github.com/cfranklin11)
+
 # Release 0.15.0
 
 ## Major features and improvements

docs/build-docs.sh

@@ -30,16 +30,17 @@
 
 set -e
 
-rm -rf kedro/html
 pip install -e ".[docs]"
 pip install -r test_requirements.txt
 python -m ipykernel install --user --name=kedro --display-name=Kedro
-rm -rf docs/tmp-build-artifacts
-cp -a docs/source docs/tmp-build-artifacts
-mv docs/tmp-build-artifacts/05_api_docs/* docs/tmp-build-artifacts/
-rm -rf docs/tmp-build-artifacts/05_api_docs/
+
+# Move some files around. We need a separate build directory, which would
+# have all the files, build scripts would shuffle the files,
+# we don't want that happening on the actual code locally.
+# When running on ReadTheDocs, sphinx-build would run directly on the original files,
+# but we don't care about the code state there.
 rm -rf docs/build
-mkdir -p docs/build/html/_static
-cp -r docs/source/css docs/build/html/_static
-sphinx-build -c docs/ -W -E -a -j auto docs/tmp-build-artifacts docs/build/html
-rm -rf docs/tmp-build-artifacts
+mkdir docs/build/
+cp -r docs/_templates docs/conf.py docs/*.svg docs/*.json  docs/build/
+
+sphinx-build -c docs/ -WETa -j auto -D language=en docs/build/ docs/build/html

docs/conf.py

@@ -14,7 +14,6 @@
 # serve to show the default.
 
 import importlib
-import os
 import re
 import shutil
 import sys
@@ -39,10 +38,6 @@
 
 
 # -- General configuration ---------------------------------------------------
-
-on_rtd = os.getenv("READTHEDOCS") == "True"  # running on ReadTheDocs
-
-
 # If your documentation needs a minimal Sphinx version, state it here.
 #
 # needs_sphinx = '1.0'
@@ -91,9 +86,7 @@
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path .
-exclude_patterns = ["**.ipynb_checkpoints", "_templates", "modules.rst"]
-if on_rtd:
-    exclude_patterns.append("source")
+exclude_patterns = ["**.ipynb_checkpoints", "_templates", "modules.rst", "source"]
 
 
 # The name of the Pygments (syntax highlighting) style to use.
@@ -389,22 +382,22 @@ def skip(app, what, name, obj, skip, options):
     return skip
 
 
-def _prepare_cwd_for_rtd(app, config):
+def _prepare_build_dir(app, config):
     """Get current working directory to the state expected
     by the ReadTheDocs builder. Shortly, it does the same as
     ./build-docs.sh script except not running `sphinx-build` step."""
-    copy_tree(str(here / "source"), str(here))
-    copy_tree(str(here / "05_api_docs"), str(here))
-    shutil.rmtree(str(here / "05_api_docs"))
-    shutil.rmtree(str(here / "_build"), ignore_errors=True)
-    _css = here / "_build" / "html" / "_static" / "css"
-    copy_tree(str(here / "css"), str(_css))
-    shutil.rmtree(str(here / "css"))
+    build_root = Path(app.srcdir)
+    build_out = Path(app.outdir)
+    copy_tree(str(here / "source"), str(build_root))
+    copy_tree(str(build_root / "05_api_docs"), str(build_root))
+    shutil.rmtree(str(build_root / "05_api_docs"))
+    shutil.rmtree(str(build_out), ignore_errors=True)
+    copy_tree(str(build_root / "css"), str(build_out / "_static" / "css"))
+    shutil.rmtree(str(build_root / "css"))
 
 
 def setup(app):
-    if on_rtd:
-        app.connect("config-inited", _prepare_cwd_for_rtd)
+    app.connect("config-inited", _prepare_build_dir)
     app.connect("autodoc-process-docstring", autodoc_process_docstring)
     app.connect("autodoc-skip-member", skip)
     app.add_stylesheet("css/qb1-sphinx-rtd.css")

docs/source/03_tutorial/03_set_up_data.md

@@ -18,18 +18,26 @@ The spaceflight tutorial has three files and uses two data formats: `.csv` and `
 * [companies.csv](https://raw.githubusercontent.com/quantumblacklabs/kedro/develop/docs/source/03_tutorial/data/companies.csv)
 * [shuttles.xlsx](https://github.com/quantumblacklabs/kedro/blob/develop/docs/source/03_tutorial/data/shuttles.xlsx?raw=true)
 
-You can [download the files from GitHub](https://www.quora.com/How-do-I-download-something-from-GitHub) using [cURL](https://curl.haxx.se/download.html) or [Wget](https://www.gnu.org/software/wget/).
-
-An example of downloading [reviews.csv](https://raw.githubusercontent.com/quantumblacklabs/kedro/develop/docs/source/03_tutorial/data/reviews.csv) to your current directory is done by running this in your terminal or command line:
+Here is an example of how you can [download the files from GitHub](https://www.quora.com/How-do-I-download-something-from-GitHub) to `data/01_raw` directory inside your project using [cURL](https://curl.haxx.se/download.html):
 
 ```bash
-curl -O https://raw.githubusercontent.com/quantumblacklabs/kedro/develop/docs/source/03_tutorial/data/reviews.csv
+# reviews
+curl -o data/01_raw/reviews.csv https://raw.githubusercontent.com/quantumblacklabs/kedro/develop/docs/source/03_tutorial/data/reviews.csv
+# companies
+curl -o data/01_raw/companies.csv https://raw.githubusercontent.com/quantumblacklabs/kedro/develop/docs/source/03_tutorial/data/companies.csv
+# shuttles
+curl -L -o data/01_raw/shuttles.xlsx https://github.com/quantumblacklabs/kedro/blob/develop/docs/source/03_tutorial/data/shuttles.xlsx?raw=true
 ```
 
-Or through using Wget:
+Or through using [Wget](https://www.gnu.org/software/wget/):
 
 ```bash
-wget https://raw.githubusercontent.com/quantumblacklabs/kedro/develop/docs/source/03_tutorial/data/reviews.csv
+# reviews
+wget -O data/01_raw/reviews.csv https://raw.githubusercontent.com/quantumblacklabs/kedro/develop/docs/source/03_tutorial/data/reviews.csv
+# companies
+wget -O data/01_raw/companies.csv https://raw.githubusercontent.com/quantumblacklabs/kedro/develop/docs/source/03_tutorial/data/companies.csv
+# shuttles
+wget -O data/01_raw/shuttles.xlsx https://github.com/quantumblacklabs/kedro/blob/develop/docs/source/03_tutorial/data/shuttles.xlsx?raw=true
 ```