Merge remote-tracking branch 'origin/master' into mariusz/6674

.gitignore

@@ -1,42 +1,80 @@
-*~
-*.pyc
-*.iml
-.idea/
-.metadata/
-.recommenders/
-.gradle/
-.settings/
-RemoteSystemsTempFiles/
-jupyter_contrib_nbextensions/
-Groovier-1.0.0.jar
-jupyter-groovy-kernel.log
-jupyter-groovy-kernel-1.0.0.jar
-jupyter-groovy-kernel.log
-.ipynb_checkpoints/
-/*.ipynb
-!StartHere.ipynb
-#.gitignore
-/gradle*
-beaker.egg-info/
+# BeakerX gitignore
+
+# General
+lib/
 
+# Beaker
+VERSION
 beakerx/static/
-beakerx.egg-info/
 beakerx/beakerx/static/
 beakerx/beakerx/kernel/
 beakerx/beakerx/javadoc/
-beakerx/dist
+beakerx/record.txt
+
+# Jupyter / iPython
+/.project
+jupyter_contrib_nbextensions/
+.ipynb_checkpoints/
+/*.ipynb
+!StartHere.ipynb
 
+# Gradle
+.gradle/
+/gradle*
 kernel/build/
 kernel/*/build/
 /build/
-/.classpath
-/.project
 /.nb-gradle/
 .nb-gradle-properties
+Groovier-1.0.0.jar
+jupyter-groovy-kernel.log
+jupyter-groovy-kernel-1.0.0.jar
+jupyter-groovy-kernel.log
+
+# Java
+/.classpath
+
+# Python
+*.py[cod]
 __pycache__/
-beakerx/__pycache__/
-beakerx/plot/__pycache__/
+beakerx/dist
+*.egg-info/
+
+# Node
 js/bower_components/
-lib/
-beakerx/record.txt
-VERSION
+
+# Ignore tags created by etags, ctags, gtags, or cscope
+TAGS
+.TAGS
+.TAGS/
+tags
+.tags
+!tags/
+gtags.files
+GTAGS
+GRTAGS
+GPATH
+GSYMS
+cscope.files
+cscope.out
+cscope.in.out
+cscope.po.out
+
+# Emacs
+*~
+
+# Vim
+[._]*.s[a-v][a-v]
+[._]*.sw[a-p]
+[._]*s[a-v][a-z]
+[._]sw[a-p]
+
+# IntelliJ
+.idea/
+.iml
+
+# Eclipse
+.metadata/
+.settings/
+.recommenders/
+RemoteSystemsTempFiles/

README.md

@@ -24,14 +24,14 @@
 [![NPM version](https://badge.fury.io/js/beakerx.svg)](http://badge.fury.io/js/beakerx)
 [![PyPI Version](https://badge.fury.io/py/beakerx.svg)](http://badge.fury.io/py/beakerx)
 [![Anaconda-Server Badge](https://anaconda.org/conda-forge/beakerx/badges/version.svg)](https://anaconda.org/conda-forge/beakerx)
-[![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/twosigma/beakerx/0.11.1?filepath=StartHere.ipynb)
+[![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/twosigma/beakerx/0.13.0?filepath=StartHere.ipynb)
 
 BeakerX is a collection of JVM kernels and interactive widgets for
 plotting, tables, autotranslation, and other extensions to Jupyter
 Notebook.  BeakerX is in beta and under active development.
 
 The [documentation](https://github.com/twosigma/beakerx/blob/master/StartHere.ipynb) consists of tutorial notebooks on GitHub.
-You can try it in the cloud for free with [Binder](https://mybinder.org/v2/gh/twosigma/beakerx/0.11.1?filepath=StartHere.ipynb).
+You can try it in the cloud for free with [Binder](https://mybinder.org/v2/gh/twosigma/beakerx/0.13.0?filepath=StartHere.ipynb).
 
 BeakerX is the successor to the [Beaker Notebook (source code
 archive)](https://github.com/twosigma/beaker-notebook-archive).  It
@@ -55,6 +55,7 @@ how to install and run BeakerX.
 ```
 conda create -y -n beakerx 'python>=3' nodejs pandas openjdk maven
 source activate beakerx
+conda config --env --add pinned_packages 'openjdk>8.0.121'
 conda install -y -c conda-forge ipywidgets
 (cd beakerx; pip install -e . --verbose)
 beakerx install
@@ -63,8 +64,9 @@ beakerx install
 ### Build and Install for Lab
 
 ```
-conda create -y -n labx 'python>=3' nodejs pandas openjdk maven pytest
+conda create -y -n labx 'python>=3' nodejs pandas openjdk maven
 source activate labx
+conda config --env --add pinned_packages 'openjdk>8.0.121'
 conda install -y -c conda-forge jupyterlab
 (cd beakerx; pip install -e . --verbose)
 beakerx install
@@ -107,6 +109,83 @@ The Java unit tests are run with every build. See [test/README.md] for how to ru
 docker run -p 8888:8888 beakerx/beakerx
 ```
 
+## Architecture and Code Overview
+
+BeakerX is a collection of kernels and extensions for Jupyter.
+The code is organized into subdirectories as follows:
+
+* [beakerx](beakerx) The Python packages.  The main beakerx package has:
+
+  * a customized KernelSpec to allow BeakerX to configure the JVMs
+    started to run the kernels,
+
+  * the beakerx command line program, which has the bkr2ipynb
+    converter as well as install and uninstall functions,
+
+  * the Python API for the runtime (tables, plots, easyform),
+    including automatically installing a displayer for pandas tables,
+    and autotranslation;
+
+  * the webpack (compiled JavaScript, TypeScript, CSS, fonts, images);
+    and
+
+  * the compiled Java JARs.
+
+  There is a seperate python package (beakerx_magics) for the
+  `%%groovy` magic so it can always be loaded *without* loading the
+  regular beakerx package (which would turn on display of pandas
+  tables with our table widget).
+
+* [doc](doc) Documentation consisting of executable tutorial
+  notebooks.  [StartHere.ipynb](StartHere.ipynb) at the top level
+  links to these and is the intended way to navigate them.  There is a
+  subdirectory for each language.
+
+* [docker](docker) configuration files for creating the Docker image.
+  There is a subdirectory [doc/base](doc/base) for an image with
+  BeakerX's dependencies (the Ubuntu and conda packages).  The main
+  image is built by compiling BeakerX and installing BeakerX in the
+  base image.
+
+* [js](js) There are two subdirectories of JavaScript and TypeScript,
+  [js/lab](js/lab) and [js/notebook](js/notebook).  New code is being
+  written in TypeScript.
+
+  The lab subdirectory has the extension for Jupyter Lab (distributed
+  by npm).  Notebook has two extensions, one for the widgets (which
+  are included in Lab as well, and are also seperately distributed
+  with npm for embedded applications such as nbviewer), and one for
+  the notebook application.  This adds a tab to the tree view with our
+  options panel.  And for regular notebook pages the extension
+  handles:
+
+  * running initialization cells,
+
+  * publication,
+
+  * autotranslation,
+
+  * the getCodeCells and runByTag APIs,
+
+  * callbacks for table and plot actions,
+
+  * UI customizations such as changing the fonts, allowing wide code
+    cells, and disabling autosave.
+
+* [kernel](kernel) The Java implementation of the kernels is here.
+  The main directory is [kernel/base](kernel/base) which has generic
+  code for all the languages.  The base kernel has classes for
+  Jupyter's Comm protocol (a layer over ZMQ), magics, the classpath
+  (including loading from maven), and the kernl parts of the widget
+  APIs.
+
+  There is also a subdirectory for each language which has the
+  evaluator for that language, plus scala has wrappers for the widgets
+  so they have native types.
+
+* [test](test) The e2e tests, which are made with wdio (selenium,
+  chromedriver, jasmine).
+
 ## Contributing
 
 See [CONTRIBUTING.md](CONTRIBUTING.md).
@@ -146,3 +225,4 @@ database engine (http://www.h2database.com/), which is dual licensed
 and available under the MPL 2.0 (Mozilla Public License) or under the
 EPL 1.0 (Eclipse Public License).  An original copy of the license
 agreement can be found at: http://www.h2database.com/html/license.html
+

RELEASE.md

@@ -64,12 +64,11 @@ git clone https://github.com/twosigma/beakerx-feedstock.git
 cd beakerx-feedstock
 git remote add upstream https://github.com/conda-forge/beakerx-feedstock.git
 git fetch upstream
-git checkout master
-git reset --hard upstream/master  
+git reset --hard upstream/master
 git push origin master --force
 ```
 - Update the `version` and `sha256` variable values in `recipe/meta.yaml`.
-  Return  build number to 0.
+  Return  build number to 1.
 
 Then test it locally:
 
@@ -95,11 +94,11 @@ conda install --use-local beakerx
 Publish on npmjs
 ----------------
 
-To update the embedded version of our widget library:
-
-- Do a full build.
-- Run `(cd js/notebook; npm publish)`
-- Run `(cd js/lab; npm publish)`
+To update the embedded version of our widget library, and our Lab extension:
+```
+(cd js/notebook; npm publish)
+(cd js/lab; npm publish)
+```
 
 Release to Docker Hub
 ---------------------

StartHere.ipynb

@@ -16,10 +16,13 @@
     "[Groovy](doc/groovy/Groovy.ipynb), [Java](doc/java/Java.ipynb), [Scala](doc/scala/Scala.ipynb), [Clojure](doc/clojure/Clojure.ipynb), [SQL](doc/sql/Sql.ipynb), [Kotlin](doc/kotlin/Kotlin.ipynb).\n",
     "\n",
     "### Magics\n",
-    "[Timing](doc/groovy/TimingMagicCommands.ipynb), [Classpath and Imports](doc/groovy/ClasspathMagicCommands.ipynb), [Properties, Heap Size and other JVM parameters](doc/groovy/JavaArgs.ipynb), [Defining New Magics](doc/groovy/LoadMagicCommand.ipynb).\n",
+    "[Timing](doc/groovy/TimingMagicCommands.ipynb), [Classpath and Imports](doc/groovy/ClasspathMagicCommands.ipynb), [Defining New Magics](doc/groovy/LoadMagicCommand.ipynb).\n",
+    "\n",
+    "### Options Panels\n",
+    "[Properties, Heap Size, and other JVM Options](doc/groovy/JavaArgs.ipynb), [UI Options](doc/groovy/UIOptions.ipynb).\n",
     "\n",
     "### Autotranslation\n",
-    "[Python](doc/python/AutoTranslation.ipynb), [Groovy](doc/groovy/AutoTranslation.ipynb)\n",
+    "[Python](doc/python/AutoTranslation.ipynb), [Groovy](doc/groovy/AutoTranslation.ipynb).\n",
     "\n",
     "### Groovy Plotting and Charting\n",
     "[Example and Interaction](doc/groovy/Charting.ipynb), [Time Series and General APIs and Features](doc/groovy/PlotFeatures.ipynb), [Category Plots and Bar Charts](doc/groovy/CategoryPlot.ipynb), [Levels of Detail](doc/groovy/LevelsOfDetails.ipynb), [Histograms](doc/groovy/Histogram.ipynb), [Heatmaps](doc/groovy/Heatmap.ipynb), [Treemaps](doc/groovy/Treemap.ipynb), [Plot Actions](doc/groovy/PlotActions.ipynb), [Plot Seamless Updates](doc/groovy/PlotsPythonStyle.ipynb), [Second Y Axis](doc/groovy/2ndYaxis.ipynb).\n",
@@ -37,12 +40,12 @@
     "### Scala\n",
     "[Tables](doc/scala/TableAPI.ipynb), [Plotting](doc/scala/Plot.ipynb), [EasyForm](doc/scala/EasyForm.ipynb).\n",
     "\n",
-    "### Media Outputs and Displayer Customization\n",
+    "### Rich Outputs and Displayer Customization\n",
     "\n",
-    "[Media and MIME Outputs](doc/groovy/Mime.ipynb), [Display of Null](doc/groovy/showNullExecutionResult.ipynb), [Custom Displayers and `jvm-repr`](doc/groovy/JvmRepr.ipynb).\n",
+    "[Media and MIME Outputs](doc/groovy/Mime.ipynb),  [Display of Null](doc/groovy/showNullExecutionResult.ipynb), [Custom Displayers and `jvm-repr`](doc/groovy/JvmRepr.ipynb).\n",
     "\n",
     "### Forms, Widgets, and Interaction\n",
-    "[EasyForm](doc/groovy/EasyForm.ipynb), [Output Containers and Layout Managers](doc/groovy/OutputContainers.ipynb), [Groovy interface to Jupyter JS Widgets](doc/groovy/JavaWidgets.ipynb), [Interactive recomputation](doc/groovy/Interactive.ipynb).\n",
+    "[EasyForm](doc/groovy/EasyForm.ipynb), [Output Widget](doc/groovy/Output.ipynb), [Output Containers and Layout Managers](doc/groovy/OutputContainers.ipynb), [Groovy interface to Jupyter JS Widgets](doc/groovy/JavaWidgets.ipynb), [Interactive recomputation](doc/groovy/Interactive.ipynb).\n",
     "\n",
     "### Automation\n",
     "[Progress Reporting API](doc/groovy/ProgressUpdateReporting.ipynb), [Initialization Cells](doc/python/InitCells.ipynb), [Get Code](doc/groovy/CodeCell.ipynb).\n",
@@ -53,8 +56,8 @@
     "### Spark\n",
     "[Spark](doc/scala/Spark.ipynb) cluster computing based on Scala, and the [Flint](doc/scala/Flint.ipynb) time series library.\n",
     "\n",
-    "### Deeplearning4j and DataVec\n",
-    "[DataVec, including automatic display](doc/groovy/DataVec.ipynb).\n",
+    "### More Integrations\n",
+    "[DataVec](doc/groovy/DataVec.ipynb) (DeepLearning4j), [STIL](doc/groovy/STIL.ipynb) (Starlink Tables Infrastructure Library).\n",
     "\n",
     "## Learn More\n",
     "On the web at the homepage [BeakerX.com](http://beakerx.com), on [GitHub](https://github.com/twosigma/beakerx), and on [Two Sigma's Open Source site](http://opensource.twosigma.com/).\n",

beakerx/.gitignore

@@ -1,5 +1,3 @@
 /build/
 /.classpath
 /.project
-!/beakerx/static/tree.js
-!/beakerx/static/img/

beakerx/beakerx/__init__.py

@@ -47,4 +47,4 @@ def run():
         parse()
     except KeyboardInterrupt:
         return 130
-    return 0
+    return 0

beakerx/beakerx/_version.py

@@ -1,2 +1,2 @@
-version_info = (0, 12, 0)
+version_info = (0, 13, 0)
 __version__ = '.'.join(map(str, version_info))