From 227d1022d62b5c3d0d2d49408bb307aa8a5c533d Mon Sep 17 00:00:00 2001 From: jlstevens Date: Mon, 8 Feb 2016 13:14:03 +0000 Subject: [PATCH 01/19] Initial commit of the Dynamic map tutorial --- doc/Tutorials/DynamicMap.ipynb | 55 ++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 doc/Tutorials/DynamicMap.ipynb diff --git a/doc/Tutorials/DynamicMap.ipynb b/doc/Tutorials/DynamicMap.ipynb new file mode 100644 index 0000000000..0af8af3143 --- /dev/null +++ b/doc/Tutorials/DynamicMap.ipynb @@ -0,0 +1,55 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import holoviews as hv\n", + "import numpy as np\n", + "import math\n", + "hv.notebook_extension()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def sine_image(phase, freq):\n", + " return hv.Image(np.sin(phase + (freq*x**2+freq*y**2)))\n", + "\n", + "x,y = np.mgrid[-50:51, -50:51] * 0.1\n", + "hv.DynamicMap(sine_image, kdims=[hv.Dimension('phase',range=(0, np.pi)),\n", + " hv.Dimension('frequency', range=(0,np.pi))])" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 2", + "language": "python", + "name": "python2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 2 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython2", + "version": "2.7.11" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} From 00f7c4830af239524895b611935f240df0e3cea6 Mon Sep 17 00:00:00 2001 From: jlstevens Date: Tue, 9 Feb 2016 01:11:03 +0000 Subject: [PATCH 02/19] Renamed DynamicMap.ipynb to Dynamic_Map.ipynb --- doc/Tutorials/{DynamicMap.ipynb => Dynamic_Map.ipynb} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename doc/Tutorials/{DynamicMap.ipynb => Dynamic_Map.ipynb} (100%) diff --git a/doc/Tutorials/DynamicMap.ipynb b/doc/Tutorials/Dynamic_Map.ipynb similarity index 100% rename from doc/Tutorials/DynamicMap.ipynb rename to doc/Tutorials/Dynamic_Map.ipynb From a2dbc1846cbd8cb724c71a428093b362a0b7989d Mon Sep 17 00:00:00 2001 From: jlstevens Date: Tue, 9 Feb 2016 02:39:28 +0000 Subject: [PATCH 03/19] Draft of initial section describing DynamicMap closed mode --- doc/Tutorials/Dynamic_Map.ipynb | 214 +++++++++++++++++++++++++++++++- 1 file changed, 209 insertions(+), 5 deletions(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index 0af8af3143..f2489211c2 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -1,5 +1,53 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The [Containers](Containers.ipynb) Tutorial introduced the [HoloMap](Containers.ipynb#HoloMap), a core HoloViews datastructure that allows easy exploration of parameter spaces. The essence of a HoloMap is that it contains a collection of [Elements](Elements.ipynb) that you can easily visualize and slice.\n", + "\n", + "HoloMaps are containers that hold elements at sampled points in a multidimensional space. Although this makes them useful for exploring high-dimensional parameter spaces, they can very quickly consume huge amounts of memory as a result. For instance, a hundred samples along four orthogonal dimensions would need a HoloMap containing a hundred *million* elements. This highlights some of the limitations of ``HoloMaps``:\n", + "\n", + "* HoloMaps may require the generation of millions of elements before they are fully defined.\n", + "* HoloMaps can easily exhaust all the memory available to Python.\n", + "* HoloMaps can simultaneously exhaust all the memory in the browser when displayed.\n", + "* Static export of a notebook containing HoloMaps can result in impractically large HTML files.\n", + "\n", + "The ``DynamicMap`` addresses these issues by computing and displaying elements dynamically, allowing exploration of much larger datasets:\n", + "\n", + "* DynamicMaps generate elements on the fly allowing the process of exploration to begin immediately.\n", + "* DynamicMaps do not require fixed sampling, allowing exploration of parameters with arbitrary resolution.\n", + "* DynamicMaps are lazy in the sense they only compute only as much data as the user wishes to explore.\n", + "\n", + "The limitations of ``DynamicMaps`` are:\n", + "\n", + "* DynamicMaps require a live notebook server and cannot be directly exported to static HTML.\n", + "* DynamicMaps do not hold data which reduces the utility of a pickling DynamicMap.\n", + "\n", + "Now we have enumerated the pros and cons of DynamicMaps, let's create one!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
To use visualize and use a **DynamicMap** you need to be running a live Jupyter server.
This tutorial assumes that it will be run in a live notebook environment.
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# ``DynamicMap`` " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's start by importing HoloViews and loading the extension:" + ] + }, { "cell_type": "code", "execution_count": null, @@ -10,10 +58,55 @@ "source": [ "import holoviews as hv\n", "import numpy as np\n", - "import math\n", "hv.notebook_extension()" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will now create the ``DynamicMap`` equivalent of the ``HoloMap`` introduced in the [Containers Tutorial](Containers.ipynb#HoloMap). The ``HoloMap`` in that tutorial consisted of ``Image`` elements containing sine ring arrays s defined by the ``sine_array`` function:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "x,y = np.mgrid[-50:51, -50:51] * 0.1\n", + "\n", + "def sine_array(phase, freq):\n", + " return np.sin(phase + (freq*x**2+freq*y**2))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This function returns NumPy arrays when called:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "sine_array(0,1).shape" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For a ``DynamicMap`` we will need a function that returns HoloViews elements. This is easy as a trivial extension of the ``sine_array`` function allows it to output [Image](Elements.ipynb#Image) elements:" + ] + }, { "cell_type": "code", "execution_count": null, @@ -23,11 +116,122 @@ "outputs": [], "source": [ "def sine_image(phase, freq):\n", - " return hv.Image(np.sin(phase + (freq*x**2+freq*y**2)))\n", + " return hv.Image(np.sin(phase + (freq*x**2+freq*y**2)))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "sine_image(0,1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we can demonstrate the first type of exploration enabled by a ``DynamicMap`` we call 'bounded' mode." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Bounded mode" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A 'closed' mode ``DynamicMap`` is simply one where all the key dimensions have finite bounds. Bounded mode has the following properties:\n", "\n", - "x,y = np.mgrid[-50:51, -50:51] * 0.1\n", - "hv.DynamicMap(sine_image, kdims=[hv.Dimension('phase',range=(0, np.pi)),\n", - " hv.Dimension('frequency', range=(0,np.pi))])" + "* The limits of the space that can be explored must be declared for all the key dimensions.\n", + "* You can explore within the declared bounds at any resolution.\n", + "* The ``DynamicMap`` is defined using a callable that *must* be a function of its arguments (i.e the output is strictly determined by the input arguments).\n", + "\n", + "We can now create a DynamicMap by simply declaring the ranges of the two dimensions and passing the ``sine_image`` function as the ``.data``:\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "dmap = hv.DynamicMap(sine_image, kdims=[hv.Dimension('phase',range=(0, np.pi)),\n", + " hv.Dimension('frequency', range=(0.01,np.pi))])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This object is created instantly as no data has been generated yet. We can now look at the repr of this object: " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "repr(dmap)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "All ``DynamicMaps`` will look similar, only differing in the listed dimensions. Now let's see how this dynamic map visualizes itself:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "dmap" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you are running this tutorial in a live notebook, you should now see something that looks like the ``HoloMap`` in the [Containers Tutorial](Containers.ipynb#HoloMap). There are some key differences though:\n", + "\n", + "* You can now pick any value of **phase** or **frequency** up to the precision allowed by the slider.\n", + "* What you see in the cell above will not be exported in any HTML snapshot of the notebook." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Using your own callable" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can use any callable to define a ``DynamicMap`` in closed mode. A valid ``DynamicMap`` is defined by the following criteria:\n", + "\n", + "* There must be as many positional arguments in the callable signature as key dimensions.\n", + "* The argument order in the callable signature must match the order of the declared key dimensions.\n", + "* All key dimensions are defined with a bounded ``range`` or ``values`` parameter (for categorical dimensions)." ] } ], From 437f690b522d344960659418eb4adee83c69d275 Mon Sep 17 00:00:00 2001 From: jlstevens Date: Tue, 9 Feb 2016 03:23:37 +0000 Subject: [PATCH 04/19] Added tutorial section on DynamicMap caches and HoloMap conversion --- doc/Tutorials/Dynamic_Map.ipynb | 122 +++++++++++++++++++++++++++++++- 1 file changed, 121 insertions(+), 1 deletion(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index f2489211c2..9a05b910ee 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -210,7 +210,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "If you are running this tutorial in a live notebook, you should now see something that looks like the ``HoloMap`` in the [Containers Tutorial](Containers.ipynb#HoloMap). There are some key differences though:\n", + "If you are running this tutorial in a live notebook, you should now see something that looks like the ``HoloMap`` in the [Containers Tutorial](Containers.ipynb#HoloMap). ``DynamicMap`` is in fact a subclass of ``HoloMap`` with some crucial differences:\n", "\n", "* You can now pick any value of **phase** or **frequency** up to the precision allowed by the slider.\n", "* What you see in the cell above will not be exported in any HTML snapshot of the notebook." @@ -233,6 +233,126 @@ "* The argument order in the callable signature must match the order of the declared key dimensions.\n", "* All key dimensions are defined with a bounded ``range`` or ``values`` parameter (for categorical dimensions)." ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### The ``DynamicMap`` cache" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Above we mentioned that ``DynamicMap`` is an instance of ``HoloMap``. Does this means it has a ``.data`` attribute?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "dmap.data" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This is exactly the same sort of ``.data`` as the equivalent ``HoloMap`` except this value will vary according to how much you explored the parameter space of ``dmap`` using the sliders above. In a ``HoloMap``, ``.data`` contains a defined sampling along the different dimensions whereas in a ``DynamicMap``, the ``.data`` is the the *cache*.\n", + "\n", + "The cache serves two purposes:\n", + "\n", + "* Avoids recomputation of an element in the event that we revisit a particular point in the parameter space. This is fairly unlikely as sliders can now take any continuous value.\n", + "* Keeps a record of the space that has been explored with the ``DynamicMap`` by conversion to a ``HoloMap``.\n", + "\n", + "We can convert a ``DynamicMap`` directly to a ``HoloMap`` as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "hv.HoloMap(dmap)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This is in fact equivalent to declaring a HoloMap with the same parameters using ``dmap.data`` as input.\n", + "\n", + "Although creating a HoloMap this way is easy, the result is poorly controlled as the keys in the HoloMap are defined by how you moved the sliders around! This can be easily rectified using the same key selection semantics of ``HoloMap`` to define the elements in the cache:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "hv.HoloMap(dmap[{(0,0.01), (0,0.5), (0.5,0.01), (0.5,0.5)}])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As this sort of cartesian product is a very common way to sample a ``DynamicMap`` there is another convenient syntax which also works with ``HoloMaps``. Here is an equivalent way of defining the same set of keys:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "hv.HoloMap(dmap[{0,0.5},{0.01,0.5}])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note that you can index a ``DynamicMap`` with a literal key in exactly the same way as a ``HoloMap`` so long as you use an exact key value that exists in the cache:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "dmap[dmap.keys()[-1]]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Slicing bounded ``DynamicMaps``" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Slicing is currently not supported in bounded ``DynamicMaps`` (in future this will set the dimension ranges for continuous dimensions). The only slices that is allowed is a single unbounded slice which creates a clone:" + ] } ], "metadata": { From a1b778529e7bfe78a0ffa1ac6d8456857416ef7e Mon Sep 17 00:00:00 2001 From: jlstevens Date: Tue, 9 Feb 2016 21:00:52 +0000 Subject: [PATCH 05/19] Described sampling methods and DynamicMap open mode --- doc/Tutorials/Dynamic_Map.ipynb | 440 +++++++++++++++++++++++++++++++- 1 file changed, 428 insertions(+), 12 deletions(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index 9a05b910ee..75fd88856e 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -31,14 +31,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "
To use visualize and use a **DynamicMap** you need to be running a live Jupyter server.
This tutorial assumes that it will be run in a live notebook environment.
" + "
To use visualize and use a DynamicMap you need to be running a live Jupyter server.
This tutorial assumes that it will be run in a live notebook environment.
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "# ``DynamicMap`` " + "## ``DynamicMap`` " ] }, { @@ -116,7 +116,10 @@ "outputs": [], "source": [ "def sine_image(phase, freq):\n", - " return hv.Image(np.sin(phase + (freq*x**2+freq*y**2)))" + " return hv.Image(np.sin(phase + (freq*x**2+freq*y**2)))\n", + "\n", + "def cosine_image(phase, freq):\n", + " return hv.Image(np.cos(phase + (freq*x**2+freq*y**2)))" ] }, { @@ -127,7 +130,7 @@ }, "outputs": [], "source": [ - "sine_image(0,1)" + "sine_image(0,1) + cosine_image(0,1)" ] }, { @@ -141,7 +144,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Bounded mode" + "### Bounded mode" ] }, { @@ -150,7 +153,7 @@ "source": [ "A 'closed' mode ``DynamicMap`` is simply one where all the key dimensions have finite bounds. Bounded mode has the following properties:\n", "\n", - "* The limits of the space that can be explored must be declared for all the key dimensions.\n", + "* The limits of the space and/or the allowable values must be declared for all the key dimensions (unless [sampled mode](#SampledMode) is enabled).\n", "* You can explore within the declared bounds at any resolution.\n", "* The ``DynamicMap`` is defined using a callable that *must* be a function of its arguments (i.e the output is strictly determined by the input arguments).\n", "\n", @@ -206,6 +209,32 @@ "dmap" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As with every other component in HoloViews, you can generate ``Layouts`` using the ``+`` operator:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "dmap + hv.DynamicMap(cosine_image, kdims=[hv.Dimension('phase',range=(0, np.pi)),\n", + " hv.Dimension('frequency', range=(0.01,np.pi))])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As both elements are ``DynamicMaps`` with the same dimension ranges, the continuous sliders are retained. If a ``HoloMap`` is used the sliders will snap to the available samples across the ``HoloMaps`` in the layout. For bounded ``DynamicMaps`` that do not require ranges to be declared, see [sampled mode](#SampledMode)" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -220,7 +249,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Using your own callable" + "#### Using your own callable" ] }, { @@ -238,7 +267,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### The ``DynamicMap`` cache" + "#### The ``DynamicMap`` cache" ] }, { @@ -267,8 +296,9 @@ "\n", "The cache serves two purposes:\n", "\n", - "* Avoids recomputation of an element in the event that we revisit a particular point in the parameter space. This is fairly unlikely as sliders can now take any continuous value.\n", - "* Keeps a record of the space that has been explored with the ``DynamicMap`` by conversion to a ``HoloMap``.\n", + "* Avoids recomputation of an element should we revisit a particular point in the parameter space. This works well for categorical data but doesn't help much when using continuous sliders.\n", + "* Records the space that has been explored with the ``DynamicMap`` when converting to a ``HoloMap``.\n", + "* Ensures a finite history of generator output when using [open mode](#OpenMode) together with infinite generators.\n", "\n", "We can convert a ``DynamicMap`` directly to a ``HoloMap`` as follows:" ] @@ -290,7 +320,7 @@ "source": [ "This is in fact equivalent to declaring a HoloMap with the same parameters using ``dmap.data`` as input.\n", "\n", - "Although creating a HoloMap this way is easy, the result is poorly controlled as the keys in the HoloMap are defined by how you moved the sliders around! This can be easily rectified using the same key selection semantics of ``HoloMap`` to define the elements in the cache:" + "Although creating a HoloMap this way is easy, the result is poorly controlled as the keys in the HoloMap are defined by how you moved the sliders around! This can be easily rectified using the same key selection semantics of ``HoloMap`` to define exactly which elements are sampled in the cache:" ] }, { @@ -344,7 +374,131 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Slicing bounded ``DynamicMaps``" + "The default cache size is the reasonably high value of 500 elements. You can set the cache size using the ``cache_size`` parameter if you find you are running into issues with memory consumption. A bounded ``DynamicMap`` with ``cache_size`` of one requires the least memory but will recompute a new element every time the sliders are moved." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Sampling DynamicMaps" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We have now seen one way of sampling a DynamicMap which is to populate the cache with a set of keys. This approach is designed to make conversion of a ``DynamicMap`` into a ``HoloMap`` easy. One disadvantage of this type of sampling is that populating the cache consumes memory resulting in many of the same limitations as ``HoloMap``. To avoid this, there are two other ways of sampling a bounded ``DynamicMap``" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Dimension values" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "If you want a fixed sampling instead of continuous sliders but wish to retain the online generation of elements as the sliders are moved, you can simply set the dimension values. Here is an example that matches the cached sampled HoloMaps above but which generates elements as they are requested:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "hv.DynamicMap(sine_image, kdims=[hv.Dimension('phase',values=[0,0.5]),\n", + " hv.Dimension('frequency', values=[0.01,0.5])])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Sampled mode " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A bounded ``DynamicMap`` in sampled mode is the least restricted type of ``DynamicMap`` as it can be declared without any information about the allowable dimension ranges or values:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "dmap = hv.DynamicMap(sine_image, kdims=['phase', 'frequency'], sampled=True)\n", + "dmap" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As you can see, this type of ``DynamicMap`` cannot be visualized in isolation. As before, you can sample the cache and cast it to a HoloMap as necessary:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "dmap[{0,0.5},{0.01,0.5}]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The real value of a sampled ``DynamicMap`` is how it interacts with ``HoloMaps`` in a layout. As a sampled ``DynamicMap`` doesn't have explicitly declared dimension ranges, it can always adopt set of sample values from ``HoloMaps`` in the layout." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "dmap + hv.HoloMap({(p,f):sine_image(p, f) \n", + " for p in [0,0.5,1,1.5] \n", + " for f in [0.5,0.75]}, kdims=['phase', 'frequency'])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The additional convenience of sampled ``DynamicMaps`` is subject to three particular restrictions:\n", + "\n", + "* Sampled ``DynamicMaps`` do not visualize themselves in isolation (as we have already seen).\n", + "* You cannot build a layout consisting of ``DynamicMaps`` only as at least one HoloMap is needed to define the samples.\n", + "* There cannot be more dimensions declared in the sampled ``DynamicMap`` than across the rest of the layout." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Slicing bounded ``DynamicMaps``" ] }, { @@ -353,6 +507,268 @@ "source": [ "Slicing is currently not supported in bounded ``DynamicMaps`` (in future this will set the dimension ranges for continuous dimensions). The only slices that is allowed is a single unbounded slice which creates a clone:" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Open mode " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "``DynamicMap`` also allows unbounded exploration with unbounded dimensions in 'open' mode. There are two key differences with bounded mode:\n", + "\n", + "* Instead of a callable, the input to an open ``DynamicMap`` is a generator. Once created, the generator is only used via ``next()``.\n", + "* At least one of the declared key dimensions have an unbounded range. (upper range only? check!)\n", + "* An open mode ``DynamicMap`` can run forever or until a ``StopIteration`` exception is raised.\n", + "* Open mode ``DynamicMaps`` can be stateful with an irreversible direction of time.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Infinite generators" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Our first example will be using an infinite generator, using the fact that phase is an infinite cyclic quantity:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def sine_gen(freq=0.5):\n", + " phase=0\n", + " while True:\n", + " yield hv.Image(np.sin(phase + (freq*x**2+freq*y**2)))\n", + " phase+=0.2" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Just to confirm this is a generator:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "sine_gen()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And now for our infinite ``DynamicMap``:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "dmap = hv.DynamicMap(sine_gen(), kdims=['phase'])\n", + "dmap" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note that phase is shown as an integer. This is the default behavior and corresponds to the call count (i.e the number of times ``next()`` has been called on the generator. If we want to show the phase value properly, we need our generator to return a (key, element) pair:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def sine_kv_gen(freq=0.5):\n", + " phase=0\n", + " while True:\n", + " yield (phase, hv.Image(np.sin(phase + (freq*x**2+freq*y**2))))\n", + " phase+=0.2\n", + " \n", + "hv.DynamicMap(sine_kv_gen(), kdims=['phase'])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note that if you pause the dynamicmap, you can scrub back to previous frames in the cache. In other words you can view a limited history of elements output by the generator which does *not* re-execute the generator in any way as there is impossible to rewind generator state. If you have a stateful generator that say, depends on the current wind speed in Scotland, this history may be misleading in which case you can simply set the ``cache_size`` parameter to 1." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Multi-dimensional generators" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In open mode, elements are naturally serialized by a linear sequence of ``next()`` calls. This doesn't mean that they can't have multiple key dimensions:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def sine_kv_gen_2D(freq=0.5):\n", + " phase=0\n", + " while True:\n", + " yield ((phase, freq), hv.Image(np.sin(phase + (freq*x**2+freq*y**2))))\n", + " phase+=0.2\n", + " freq += 0.01\n", + " \n", + "hv.DynamicMap(sine_kv_gen_2D(), kdims=['phase', 'frequency'])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Finite generators" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Open mode ``DynamicMaps`` are finite and terminate if ``StopIteration`` is raised. This example uses a generator expression that is identical to the first open ``DynamicMap`` example above except it terminates after 20 phases:\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "dmap=hv.DynamicMap((hv.Image(np.sin(0.2*phase + (0.5*x**2+0.5*y**2))) for phase in range(21)), kdims=['phase'])\n", + "dmap" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It is trivial to adapt this generator expression to return (key, tuple) pairs where the key may have one or more dimensions.\n", + "\n", + "Now if you are familiar with generators in Python, you might be wondering what happens when a finite generator is exhausted. First we should mention that casting a ``DynamicMap`` to a list is always finite because ``__iter__`` returns the cache instead of a potentially infinite generator:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "list(dmap) # The cache" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As we know this ``DynamicMap`` is finite, we can exhaust it as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "while True:\n", + " try:\n", + " next(dmap) # Returns Image elements\n", + " except StopIteration:\n", + " print(\"The dynamic map is exhausted.\")\n", + " break" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's have a look at the dynamic map:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "dmap" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We are given only the repr to reflect that the generator is exhausted. However, as the process of iteration has populated the cache, we can still view the output as a ``HoloMap``:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "hv.HoloMap(dmap)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [] } ], "metadata": { From 22cbf18bbad650d4b2e413452f64f6dfc64c233c Mon Sep 17 00:00:00 2001 From: jlstevens Date: Tue, 9 Feb 2016 21:05:31 +0000 Subject: [PATCH 06/19] Added Dynamic Map tutorial to the sidebar --- doc/Tutorials/index.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/Tutorials/index.rst b/doc/Tutorials/index.rst index 37f78a89ac..ba45544240 100644 --- a/doc/Tutorials/index.rst +++ b/doc/Tutorials/index.rst @@ -102,6 +102,7 @@ extend HoloViews and apply it to real world data see the `Examples Exploring Data Sampling Data Columnar Data + Dynamic Map Options Exporting Continuous Coordinates From 48155ecc071c55ae0325761ad1b7359e197178b7 Mon Sep 17 00:00:00 2001 From: jlstevens Date: Tue, 9 Feb 2016 22:49:59 +0000 Subject: [PATCH 07/19] Complete draft of the Dynamic Map tutorial --- doc/Tutorials/Dynamic_Map.ipynb | 205 ++++++++++++++++++++++++++++++-- 1 file changed, 196 insertions(+), 9 deletions(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index 75fd88856e..500e7f6151 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -549,8 +549,7 @@ }, "outputs": [], "source": [ - "def sine_gen(freq=0.5):\n", - " phase=0\n", + "def sine_gen(phase=0, freq=0.5):\n", " while True:\n", " yield hv.Image(np.sin(phase + (freq*x**2+freq*y**2)))\n", " phase+=0.2" @@ -608,8 +607,7 @@ }, "outputs": [], "source": [ - "def sine_kv_gen(freq=0.5):\n", - " phase=0\n", + "def sine_kv_gen(phase=0, freq=0.5):\n", " while True:\n", " yield (phase, hv.Image(np.sin(phase + (freq*x**2+freq*y**2))))\n", " phase+=0.2\n", @@ -646,8 +644,7 @@ }, "outputs": [], "source": [ - "def sine_kv_gen_2D(freq=0.5):\n", - " phase=0\n", + "def sine_kv_gen_2D(phase=0, freq=0.5):\n", " while True:\n", " yield ((phase, freq), hv.Image(np.sin(phase + (freq*x**2+freq*y**2))))\n", " phase+=0.2\n", @@ -747,7 +744,23 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We are given only the repr to reflect that the generator is exhausted. However, as the process of iteration has populated the cache, we can still view the output as a ``HoloMap``:" + "We are given only the repr to reflect that the generator is exhausted. However, as the process of iteration has populated the cache, we can still view the output as a ``HoloMap`` using ``hv.HoloMap(dmap)`` as before." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Counter mode and temporal state" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Open mode is intended to interface live data streams or simulations with HoloViews. The ``DynamicMap`` to generate live visualizations for as long as new data is requested. Although this works for simple cases, Python generators have problematic limitations that can be resolved using 'counter' mode.\n", + "\n", + "In this example, lets say we have a simulation or data recording where time increases in integer steps:" ] }, { @@ -758,7 +771,91 @@ }, "outputs": [], "source": [ - "hv.HoloMap(dmap)" + "def time_gen(time=1):\n", + " while True:\n", + " yield time\n", + " time += 1\n", + " \n", + "time = time_gen()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we have two generators that return Images that are a function of the simulation time. Here, they have identical output except one of the outputs includes additive noise:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "ls = np.linspace(0, 10, 200)\n", + "xx, yy = np.meshgrid(ls, ls)\n", + "\n", + "def cells():\n", + " while True:\n", + " t = time.next()\n", + " arr = np.sin(xx+t)*np.cos(yy+t)\n", + " yield hv.Image(arr)\n", + "\n", + "def cells_noisy():\n", + " while True:\n", + " t = time.next()\n", + " arr = np.sin(xx+t)*np.cos(yy+t)\n", + " yield hv.Image(arr + 0.2*np.random.rand(200,200))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's create a Layout using these two generators:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "hv.DynamicMap(cells(), kdims=['time']) + hv.DynamicMap(cells_noisy(), kdims=['time'])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you pause the animation, you'll see that these two outputs are *not* in phase, despite the fact that the generators are defined identically (modulo the additive noise)!\n", + "\n", + "The issue is that generators are used via the ``next()`` interface and when either generator is called, the simultation time is increased. In other words, the noisy version in subfigure **B** is actually displayed at a later time than in subfigure **A**.\n", + "\n", + "This is a fundamental issue as the ``next`` method does not take arguments. What we want is for all the ``DynamicMaps`` presented in a Layout to share a common simulation time that is only incremented by interaction with the scrubber widget. This is exactly the sort of situation where you want to use counter mode." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Handling time-dependent state" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To define a ``DynamicMap`` in counter mode:\n", + "\n", + "* Leave one or more dimensions *unbounded* (as in open mode)\n", + "* Supply a callable (as in bounded mode) that accepts *one* argument\n", + "\n", + "This callable should act in the same way as the generators of open mode except the output in controlled by the single counter argument." ] }, { @@ -768,7 +865,97 @@ "collapsed": true }, "outputs": [], - "source": [] + "source": [ + "ls = np.linspace(0, 10, 200)\n", + "xx, yy = np.meshgrid(ls, ls)\n", + "\n", + "def cells_counter(t):\n", + " while True:\n", + " arr = np.sin(xx+t)*np.cos(yy+t)\n", + " return hv.Image(arr)\n", + "\n", + "def cells_noisy_counter(t):\n", + " while True:\n", + " arr = np.sin(xx+t)*np.cos(yy+t)\n", + " return hv.Image(arr + 0.2*np.random.rand(200,200))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now if we supply these functions instead of generators:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "hv.DynamicMap(cells_counter, kdims=['time']) + hv.DynamicMap(cells_noisy_counter, kdims=['time'])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now **A** and **B** are correctly in phase.\n", + "\n", + "Unfortunately, a counter is too simple to describe simultation time which is typically a float with real world units. To address this, we can simply return the actual key values we want along the time dimension, just as was demonstrated in open mode using generators:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "ls = np.linspace(0, 10, 200)\n", + "xx, yy = np.meshgrid(ls, ls)\n", + "\n", + "# Example of a global simulation time\n", + "# typical in many applications\n", + "t = 0 \n", + " \n", + "def cells_counter_kv(c):\n", + " global t\n", + " t = 0.1 * c\n", + " while True:\n", + " arr = np.sin(xx+t)*np.cos(yy+t)\n", + " return (t, hv.Image(arr))\n", + "\n", + "def cells_noisy_counter_kv(c):\n", + " global t\n", + " t = 0.1 * c\n", + " while True:\n", + " arr = np.sin(xx+t)*np.cos(yy+t)\n", + " return (t, hv.Image(arr + 0.2*np.random.rand(200,200)))\n", + " \n", + "hv.DynamicMap(cells_counter_kv, kdims=['time']) + hv.DynamicMap(cells_noisy_counter_kv, kdims=['time'])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "print(\"The global simulation time is now t=%f\" % t)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Ensuring that the HoloViews counter maps to a suitable simulation time is the responsibility of the user. However, once a consistent scheme is configured, the callable in each ``DynamicMap`` can specify the desired simulation time. If the requested simulation time is the same as the current simulation time, nothing needs to happen. Otherwise, the simulator can be run forward by the requested amount." + ] } ], "metadata": { From 6dccc6c03f60c95551236cd3cf88b87536994acd Mon Sep 17 00:00:00 2001 From: jlstevens Date: Tue, 9 Feb 2016 23:24:26 +0000 Subject: [PATCH 08/19] Removed the unused cosine_image function --- doc/Tutorials/Dynamic_Map.ipynb | 16 ++-------------- 1 file changed, 2 insertions(+), 14 deletions(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index 500e7f6151..9745a6eb6e 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -118,19 +118,7 @@ "def sine_image(phase, freq):\n", " return hv.Image(np.sin(phase + (freq*x**2+freq*y**2)))\n", "\n", - "def cosine_image(phase, freq):\n", - " return hv.Image(np.cos(phase + (freq*x**2+freq*y**2)))" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "sine_image(0,1) + cosine_image(0,1)" + "sine_image(0,1) + sine_image(0.5,2)" ] }, { @@ -224,7 +212,7 @@ }, "outputs": [], "source": [ - "dmap + hv.DynamicMap(cosine_image, kdims=[hv.Dimension('phase',range=(0, np.pi)),\n", + "dmap + hv.DynamicMap(sine_image, kdims=[hv.Dimension('phase',range=(0, np.pi)),\n", " hv.Dimension('frequency', range=(0.01,np.pi))])" ] }, From ef7e8c7a71a110f81604cac295e54a4e3df509de Mon Sep 17 00:00:00 2001 From: jlstevens Date: Tue, 9 Feb 2016 23:34:08 +0000 Subject: [PATCH 09/19] Added Dynamic Map link and description to the main tutorial page --- doc/Tutorials/index.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/doc/Tutorials/index.rst b/doc/Tutorials/index.rst index ba45544240..337b2d9d71 100644 --- a/doc/Tutorials/index.rst +++ b/doc/Tutorials/index.rst @@ -34,6 +34,11 @@ in this order: data, and how to apply operations to transform the data into complex visualizations easily. +* `Dynamic Map: `_ + How to work with datasets larger than the available memory by + computing elements on-the-fly. Using DynamicMap you can immediately + begin exploring huge volumes of data while keeping interaction + responsive and without running out of memory. Supplementary Tutorials ------------------------ From 5779a2e867bde141093d0efbe8d53d28645fd204 Mon Sep 17 00:00:00 2001 From: jlstevens Date: Wed, 10 Feb 2016 02:50:53 +0000 Subject: [PATCH 10/19] Added sections explaining slicing of DynamicMaps --- doc/Tutorials/Dynamic_Map.ipynb | 156 ++++++++++++++++++++++++++++---- 1 file changed, 136 insertions(+), 20 deletions(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index 9745a6eb6e..75936e3039 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -23,6 +23,7 @@ "\n", "* DynamicMaps require a live notebook server and cannot be directly exported to static HTML.\n", "* DynamicMaps do not hold data which reduces the utility of a pickling DynamicMap.\n", + "* DynamicMaps are often stateful as are their element caches.\n", "\n", "Now we have enumerated the pros and cons of DynamicMaps, let's create one!" ] @@ -288,7 +289,7 @@ "* Records the space that has been explored with the ``DynamicMap`` when converting to a ``HoloMap``.\n", "* Ensures a finite history of generator output when using [open mode](#OpenMode) together with infinite generators.\n", "\n", - "We can convert a ``DynamicMap`` directly to a ``HoloMap`` as follows:" + "We can always convert *any* ``DynamicMap`` directly to a ``HoloMap`` as follows:" ] }, { @@ -308,7 +309,7 @@ "source": [ "This is in fact equivalent to declaring a HoloMap with the same parameters using ``dmap.data`` as input.\n", "\n", - "Although creating a HoloMap this way is easy, the result is poorly controlled as the keys in the HoloMap are defined by how you moved the sliders around! This can be easily rectified using the same key selection semantics of ``HoloMap`` to define exactly which elements are sampled in the cache:" + "Although creating a HoloMap this way is easy, the result is poorly controlled as the keys in the HoloMap are defined by how you moved the sliders around. This can be easily rectified using the same key selection semantics of ``HoloMap`` to define exactly which elements are sampled in the cache:" ] }, { @@ -319,14 +320,18 @@ }, "outputs": [], "source": [ - "hv.HoloMap(dmap[{(0,0.01), (0,0.5), (0.5,0.01), (0.5,0.5)}])" + "dmap[{(0,0.01), (0,0.5), (0.5,0.01), (0.5,0.5)}] # Returns a *new* DynamicMap with the sampled keys in it's cache" ] }, { "cell_type": "markdown", - "metadata": {}, + "metadata": { + "collapsed": false + }, "source": [ - "As this sort of cartesian product is a very common way to sample a ``DynamicMap`` there is another convenient syntax which also works with ``HoloMaps``. Here is an equivalent way of defining the same set of keys:" + "This object behaves the same way as before it was sampled but now this ``DynamicMap`` can now be exported to static HTML with the allowed slider positions as specified in the cache (without even having to cast to a ``HoloMap``). Nonetheless, casting to a ``HoloMap`` makes the intention to generate a statically exportable, sampled visualization obvious.\n", + "\n", + "As the key selection above defines a cartesian product which is one of the most common way to sample across dimensions, there is another convenient syntax which also works with ``HoloMaps``. Here is an equivalent way of defining the same set of keys:" ] }, { @@ -337,7 +342,18 @@ }, "outputs": [], "source": [ - "hv.HoloMap(dmap[{0,0.5},{0.01,0.5}])" + "dmap[{0,0.5},{0.01,0.5}]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "dmap.data" ] }, { @@ -365,6 +381,57 @@ "The default cache size is the reasonably high value of 500 elements. You can set the cache size using the ``cache_size`` parameter if you find you are running into issues with memory consumption. A bounded ``DynamicMap`` with ``cache_size`` of one requires the least memory but will recompute a new element every time the sliders are moved." ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Slicing bounded ``DynamicMaps``" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The declared dimension ranges define the absolute limits allowed for exploration in a bounded ``DynamicMap``. That said, you can use the ``soft_range`` parameter to view subregions within that range. This setting of the ``soft_range`` parameter on dimensions defines the semantics of slicing on bounded ``DynamicMaps``:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "sliced = dmap[0.2:0.8, :]\n", + "sliced" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Open slices are used to release any ``soft_range`` values which resets the limits back to those defined by range:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "sliced[:, 0.05:0.1]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The ``[:]`` slice leaves the soft_range values alone and can be used as a convenient way to clone a ``DynamicMap``. Note that you are not allowed to mix slices when any other object type. In other words, once you use a single slice, you have to only uses slices in that indexing operation." + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -482,20 +549,6 @@ "* There cannot be more dimensions declared in the sampled ``DynamicMap`` than across the rest of the layout." ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Slicing bounded ``DynamicMaps``" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Slicing is currently not supported in bounded ``DynamicMaps`` (in future this will set the dimension ranges for continuous dimensions). The only slices that is allowed is a single unbounded slice which creates a clone:" - ] - }, { "cell_type": "markdown", "metadata": {}, @@ -944,6 +997,69 @@ "source": [ "Ensuring that the HoloViews counter maps to a suitable simulation time is the responsibility of the user. However, once a consistent scheme is configured, the callable in each ``DynamicMap`` can specify the desired simulation time. If the requested simulation time is the same as the current simulation time, nothing needs to happen. Otherwise, the simulator can be run forward by the requested amount." ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Slicing in open and counter mode" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Slicing open and counter mode ``DynamicMaps`` has the exact same semantics as normal ``HoloMap`` slicing except now the ``.data`` attribute corresponds to the cache. For instance:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def sine_kv_gen(phase=0, freq=0.5):\n", + " while True:\n", + " yield (phase, hv.Image(np.sin(phase + (freq*x**2+freq*y**2))))\n", + " phase+=0.2\n", + " \n", + "dmap = hv.DynamicMap(sine_kv_gen(), kdims=['phase'])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's fill the cache with some elements:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "for i in range(21):\n", + " dmap.next()\n", + " \n", + "print(\"Min key value in cache:%s\\nMax key value in cache:%s\" % (min(dmap.keys()), max(dmap.keys())))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "sliced = dmap[1:3.1]\n", + "print(\"Min key value in cache:%s\\nMax key value in cache:%s\" % (min(sliced.keys()), max(sliced.keys())))" + ] } ], "metadata": { From 3cf0c047fde2eeff05c65cd00fc406fd62dc0d48 Mon Sep 17 00:00:00 2001 From: jlstevens Date: Wed, 10 Feb 2016 03:09:46 +0000 Subject: [PATCH 11/19] Clarified statement regarding sampled DynamicMap restriction --- doc/Tutorials/Dynamic_Map.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index 75936e3039..21111ac71d 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -546,7 +546,7 @@ "\n", "* Sampled ``DynamicMaps`` do not visualize themselves in isolation (as we have already seen).\n", "* You cannot build a layout consisting of ``DynamicMaps`` only as at least one HoloMap is needed to define the samples.\n", - "* There cannot be more dimensions declared in the sampled ``DynamicMap`` than across the rest of the layout." + "* There cannot be more dimensions declared in the sampled ``DynamicMap`` than across the rest of the layout. We hope to relax this restriction in future." ] }, { From 8da023401c4935f609c1a19ae9a808390557ce74 Mon Sep 17 00:00:00 2001 From: "James A. Bednar" Date: Tue, 9 Feb 2016 22:34:23 -0600 Subject: [PATCH 12/19] Cleaned up tutorial somewhat --- doc/Tutorials/Dynamic_Map.ipynb | 72 +++++++++++++++++++-------------- 1 file changed, 41 insertions(+), 31 deletions(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index 21111ac71d..228143db31 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -4,9 +4,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The [Containers](Containers.ipynb) Tutorial introduced the [HoloMap](Containers.ipynb#HoloMap), a core HoloViews datastructure that allows easy exploration of parameter spaces. The essence of a HoloMap is that it contains a collection of [Elements](Elements.ipynb) that you can easily visualize and slice.\n", + "The [Containers](Containers.ipynb) Tutorial introduced the [HoloMap](Containers.ipynb#HoloMap), a core HoloViews data structure that allows easy exploration of parameter spaces. The essence of a HoloMap is that it contains a collection of [Elements](Elements.ipynb) (e.g. Images and Curves) that you can easily select and visualize.\n", "\n", - "HoloMaps are containers that hold elements at sampled points in a multidimensional space. Although this makes them useful for exploring high-dimensional parameter spaces, they can very quickly consume huge amounts of memory as a result. For instance, a hundred samples along four orthogonal dimensions would need a HoloMap containing a hundred *million* elements. This highlights some of the limitations of ``HoloMaps``:\n", + "HoloMaps hold Elements at sampled points in a multidimensional space. Although this property makes them useful for exploring high-dimensional parameter spaces, HoloMaps can very quickly consume huge amounts of memory for such spaces. For instance, a hundred samples along four orthogonal dimensions would need a HoloMap containing a hundred *million* Elements, each of which could be a substantial object. Thus ``HoloMaps`` have some clear limitations:\n", "\n", "* HoloMaps may require the generation of millions of elements before they are fully defined.\n", "* HoloMaps can easily exhaust all the memory available to Python.\n", @@ -32,7 +32,8 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "
To use visualize and use a DynamicMap you need to be running a live Jupyter server.
This tutorial assumes that it will be run in a live notebook environment.
" + "
To use visualize and use a DynamicMap you need to be running a live Jupyter server.
This tutorial assumes that it will be run in a live notebook environment.
\n", + "When viewed statically, DynamicMaps will only show the first available Element.
" ] }, { @@ -66,7 +67,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We will now create the ``DynamicMap`` equivalent of the ``HoloMap`` introduced in the [Containers Tutorial](Containers.ipynb#HoloMap). The ``HoloMap`` in that tutorial consisted of ``Image`` elements containing sine ring arrays s defined by the ``sine_array`` function:" + "We will now create the ``DynamicMap`` equivalent of the ``HoloMap`` introduced in the [Containers Tutorial](Containers.ipynb#HoloMap). The ``HoloMap`` in that tutorial consisted of ``Image`` elements containing sine ring arrays as defined by the ``sine_array`` function:" ] }, { @@ -140,11 +141,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "A 'closed' mode ``DynamicMap`` is simply one where all the key dimensions have finite bounds. Bounded mode has the following properties:\n", + "A 'bounded' mode ``DynamicMap`` is simply one where all the key dimensions have finite bounds. Bounded mode has the following properties:\n", "\n", "* The limits of the space and/or the allowable values must be declared for all the key dimensions (unless [sampled mode](#SampledMode) is enabled).\n", "* You can explore within the declared bounds at any resolution.\n", - "* The ``DynamicMap`` is defined using a callable that *must* be a function of its arguments (i.e the output is strictly determined by the input arguments).\n", + "* The ``DynamicMap`` is defined using a callable that *must* be a function of its arguments (i.e., the output is strictly determined by the input arguments).\n", "\n", "We can now create a DynamicMap by simply declaring the ranges of the two dimensions and passing the ``sine_image`` function as the ``.data``:\n", "\n" @@ -166,7 +167,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This object is created instantly as no data has been generated yet. We can now look at the repr of this object: " + "This object is created instantly as no data has been generated yet. We can now look at the ``repr`` of this object: " ] }, { @@ -202,7 +203,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As with every other component in HoloViews, you can generate ``Layouts`` using the ``+`` operator:" + "You can combine ``DynamicMaps`` with each other (and any other HoloViews Element) to create a ``Layout`` using the ``+`` operator:" ] }, { @@ -221,7 +222,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As both elements are ``DynamicMaps`` with the same dimension ranges, the continuous sliders are retained. If a ``HoloMap`` is used the sliders will snap to the available samples across the ``HoloMaps`` in the layout. For bounded ``DynamicMaps`` that do not require ranges to be declared, see [sampled mode](#SampledMode)" + "As both elements are ``DynamicMaps`` with the same dimension ranges, the continuous sliders are retained. If one or more ``HoloMaps`` is used with a ``DynamicMap``, the sliders will snap to the samples available in any ``HoloMap`` in the layout. For bounded ``DynamicMaps`` that do not require ranges to be declared, see [sampled mode](#SampledMode)." ] }, { @@ -281,11 +282,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This is exactly the same sort of ``.data`` as the equivalent ``HoloMap`` except this value will vary according to how much you explored the parameter space of ``dmap`` using the sliders above. In a ``HoloMap``, ``.data`` contains a defined sampling along the different dimensions whereas in a ``DynamicMap``, the ``.data`` is the the *cache*.\n", + "This is exactly the same sort of ``.data`` as the equivalent ``HoloMap``, except that this value will vary according to how much you explored the parameter space of ``dmap`` using the sliders above. In a ``HoloMap``, ``.data`` contains a defined sampling along the different dimensions, whereas in a ``DynamicMap``, the ``.data`` is simply the *cache*.\n", "\n", "The cache serves two purposes:\n", "\n", - "* Avoids recomputation of an element should we revisit a particular point in the parameter space. This works well for categorical data but doesn't help much when using continuous sliders.\n", + "* Avoids recomputation of an element should we revisit a particular point in the parameter space. This works well for categorical data, but doesn't help much when using continuous sliders.\n", "* Records the space that has been explored with the ``DynamicMap`` when converting to a ``HoloMap``.\n", "* Ensures a finite history of generator output when using [open mode](#OpenMode) together with infinite generators.\n", "\n", @@ -360,7 +361,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Note that you can index a ``DynamicMap`` with a literal key in exactly the same way as a ``HoloMap`` so long as you use an exact key value that exists in the cache:" + "Note that you can index a ``DynamicMap`` with a literal key in exactly the same way as a ``HoloMap``, so long as you use an exact key value that already exists in the cache:" ] }, { @@ -378,7 +379,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The default cache size is the reasonably high value of 500 elements. You can set the cache size using the ``cache_size`` parameter if you find you are running into issues with memory consumption. A bounded ``DynamicMap`` with ``cache_size`` of one requires the least memory but will recompute a new element every time the sliders are moved." + "The default cache size is the reasonably high value of 500 Elements. You can set the size using the ``cache_size`` parameter if you find you are running into issues with memory consumption. A bounded ``DynamicMap`` with ``cache_size=1`` requires the least memory, but will recompute a new Element every time the sliders are moved." ] }, { @@ -443,7 +444,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We have now seen one way of sampling a DynamicMap which is to populate the cache with a set of keys. This approach is designed to make conversion of a ``DynamicMap`` into a ``HoloMap`` easy. One disadvantage of this type of sampling is that populating the cache consumes memory resulting in many of the same limitations as ``HoloMap``. To avoid this, there are two other ways of sampling a bounded ``DynamicMap``" + "We have now seen one way of sampling a ``DynamicMap``, which is to populate the cache with a set of keys. This approach is designed to make conversion of a ``DynamicMap`` into a ``HoloMap`` easy. One disadvantage of this type of sampling is that populating the cache consumes memory, resulting in many of the same limitations as ``HoloMap``. To avoid this, there are two other ways of sampling a bounded ``DynamicMap``:" ] }, { @@ -459,7 +460,7 @@ "collapsed": true }, "source": [ - "If you want a fixed sampling instead of continuous sliders but wish to retain the online generation of elements as the sliders are moved, you can simply set the dimension values. Here is an example that matches the cached sampled HoloMaps above but which generates elements as they are requested:" + "If you want a fixed sampling instead of continuous sliders, yet still wish to retain the online generation of elements as the sliders are moved, you can simply set the dimension values. The result appears to the user just like a ``HoloMap``, and has the same data as the pre-cached version above, but now generates the data dynamically:" ] }, { @@ -485,7 +486,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "A bounded ``DynamicMap`` in sampled mode is the least restricted type of ``DynamicMap`` as it can be declared without any information about the allowable dimension ranges or values:" + "A bounded ``DynamicMap`` in sampled mode is the least restricted type of ``DynamicMap``, as it can be declared without any information about the allowable dimension ranges or values:" ] }, { @@ -504,7 +505,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As you can see, this type of ``DynamicMap`` cannot be visualized in isolation. As before, you can sample the cache and cast it to a HoloMap as necessary:" + "As you can see, this type of ``DynamicMap`` cannot be visualized in isolation. As before, you can sample the cache and thereby cast it to a HoloMap as necessary:" ] }, { @@ -522,7 +523,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The real value of a sampled ``DynamicMap`` is how it interacts with ``HoloMaps`` in a layout. As a sampled ``DynamicMap`` doesn't have explicitly declared dimension ranges, it can always adopt set of sample values from ``HoloMaps`` in the layout." + "The important usage of a sampled ``DynamicMap`` is how it interacts with ``HoloMaps`` in a Layout. As a sampled ``DynamicMap`` doesn't have explicitly declared dimension ranges, it can always adopt the set of sample values from ``HoloMaps`` in the layout." ] }, { @@ -560,12 +561,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "``DynamicMap`` also allows unbounded exploration with unbounded dimensions in 'open' mode. There are two key differences with bounded mode:\n", + "``DynamicMap`` also allows unbounded exploration with unbounded dimensions in 'open' mode. There are two key differences of this mode with bounded mode:\n", "\n", "* Instead of a callable, the input to an open ``DynamicMap`` is a generator. Once created, the generator is only used via ``next()``.\n", - "* At least one of the declared key dimensions have an unbounded range. (upper range only? check!)\n", - "* An open mode ``DynamicMap`` can run forever or until a ``StopIteration`` exception is raised.\n", - "* Open mode ``DynamicMaps`` can be stateful with an irreversible direction of time.\n" + "* At least one of the declared key dimensions must have an unbounded range. (upper range only? need to check!)\n", + "* An open mode ``DynamicMap`` can run forever, or until a ``StopIteration`` exception is raised.\n", + "* Open mode ``DynamicMaps`` can be stateful, with an irreversible direction of time.\n" ] }, { @@ -660,7 +661,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Note that if you pause the dynamicmap, you can scrub back to previous frames in the cache. In other words you can view a limited history of elements output by the generator which does *not* re-execute the generator in any way as there is impossible to rewind generator state. If you have a stateful generator that say, depends on the current wind speed in Scotland, this history may be misleading in which case you can simply set the ``cache_size`` parameter to 1." + "Note that if you pause the ``DynamicMap``, you can scrub back to previous frames in the cache. In other words, you can view a limited history of elements already output by the generator, which does *not* re-execute the generator in any way (as it is indeed impossible to rewind generator state). If you have a stateful generator that, say, depends on the current wind speed in Scotland, this history may be misleading, in which case you can simply set the ``cache_size`` parameter to 1." ] }, { @@ -705,7 +706,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Open mode ``DynamicMaps`` are finite and terminate if ``StopIteration`` is raised. This example uses a generator expression that is identical to the first open ``DynamicMap`` example above except it terminates after 20 phases:\n" + "Open mode ``DynamicMaps`` are finite and terminate if ``StopIteration`` is raised. This example uses a generator expression that is identical to the first open ``DynamicMap`` example above, except that it terminates after 20 phases:\n" ] }, { @@ -724,9 +725,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "It is trivial to adapt this generator expression to return (key, tuple) pairs where the key may have one or more dimensions.\n", + "It is trivial to adapt this generator expression to return (key, tuple) pairs, where the key may have one or more dimensions.\n", "\n", - "Now if you are familiar with generators in Python, you might be wondering what happens when a finite generator is exhausted. First we should mention that casting a ``DynamicMap`` to a list is always finite because ``__iter__`` returns the cache instead of a potentially infinite generator:" + "Now if you are familiar with generators in Python, you might be wondering what happens when a finite generator is exhausted. First we should mention that casting a ``DynamicMap`` to a list is always finite, because ``__iter__`` returns the cache instead of a potentially infinite generator:" ] }, { @@ -799,7 +800,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Open mode is intended to interface live data streams or simulations with HoloViews. The ``DynamicMap`` to generate live visualizations for as long as new data is requested. Although this works for simple cases, Python generators have problematic limitations that can be resolved using 'counter' mode.\n", + "Open mode is intended to interface live data streams or ongoing simulations with HoloViews. The ``DynamicMap`` will generate live visualizations for as long as new data is requested. Although this works for simple cases, Python generators have problematic limitations that can be resolved using 'counter' mode.\n", "\n", "In this example, lets say we have a simulation or data recording where time increases in integer steps:" ] @@ -875,9 +876,9 @@ "source": [ "If you pause the animation, you'll see that these two outputs are *not* in phase, despite the fact that the generators are defined identically (modulo the additive noise)!\n", "\n", - "The issue is that generators are used via the ``next()`` interface and when either generator is called, the simultation time is increased. In other words, the noisy version in subfigure **B** is actually displayed at a later time than in subfigure **A**.\n", + "The issue is that generators are used via the ``next()`` interface, and so when either generator is called, the simulation time is increased. In other words, the noisy version in subfigure **B** actually corresponds to a later time than in subfigure **A**.\n", "\n", - "This is a fundamental issue as the ``next`` method does not take arguments. What we want is for all the ``DynamicMaps`` presented in a Layout to share a common simulation time that is only incremented by interaction with the scrubber widget. This is exactly the sort of situation where you want to use counter mode." + "This is a fundamental issue, as the ``next`` method does not take arguments. What we want is for all the ``DynamicMaps`` presented in a Layout to share a common simulation time, which is only incremented by interaction with the scrubber widget. This is exactly the sort of situation where you want to use counter mode." ] }, { @@ -945,7 +946,7 @@ "source": [ "Now **A** and **B** are correctly in phase.\n", "\n", - "Unfortunately, a counter is too simple to describe simultation time which is typically a float with real world units. To address this, we can simply return the actual key values we want along the time dimension, just as was demonstrated in open mode using generators:" + "Unfortunately, an integer counter is often too simple to describe simulation time, which is more often a float with real-world units. To address this, we can simply return the actual key values we want along the time dimension, just as was demonstrated in open mode using generators:" ] }, { @@ -1009,7 +1010,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Slicing open and counter mode ``DynamicMaps`` has the exact same semantics as normal ``HoloMap`` slicing except now the ``.data`` attribute corresponds to the cache. For instance:" + "Slicing open and counter mode ``DynamicMaps`` has the exact same semantics as normal ``HoloMap`` slicing, except now the ``.data`` attribute corresponds to the cache. For instance:" ] }, { @@ -1060,6 +1061,15 @@ "sliced = dmap[1:3.1]\n", "print(\"Min key value in cache:%s\\nMax key value in cache:%s\" % (min(sliced.keys()), max(sliced.keys())))" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Using DynamicMaps in your code\n", + "\n", + "As you can see, ``DynamicMaps`` let you use HoloViews with a very wide range of dynamic data formats and sources, making it simple to visualize ongoing processes or very large data spaces." + ] } ], "metadata": { From ca28d298f6a5e1f119ce22130f7fd6fae80d938f Mon Sep 17 00:00:00 2001 From: jlstevens Date: Wed, 10 Feb 2016 11:41:39 +0000 Subject: [PATCH 13/19] Fixed incorrect statement regarding DynamicMap indexing --- doc/Tutorials/Dynamic_Map.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index 228143db31..9f5fe5a168 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -361,7 +361,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Note that you can index a ``DynamicMap`` with a literal key in exactly the same way as a ``HoloMap``, so long as you use an exact key value that already exists in the cache:" + "Note that you can index a ``DynamicMap`` with a literal key in exactly the same way as a ``HoloMap``. If the key exists in the cache, it is returned otherwise a suitable element will be generated. Here is an example of how you can access the last key in the cache as well as create a new element:" ] }, { From 97fe2096490845191502723297fbc371d7e54782 Mon Sep 17 00:00:00 2001 From: jlstevens Date: Wed, 10 Feb 2016 12:56:20 +0000 Subject: [PATCH 14/19] Updated tutorial with more interesting main example --- doc/Tutorials/Dynamic_Map.ipynb | 72 +++++++++++++++++++++++++++------ 1 file changed, 60 insertions(+), 12 deletions(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index 9f5fe5a168..0b870cf978 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -250,7 +250,56 @@ "\n", "* There must be as many positional arguments in the callable signature as key dimensions.\n", "* The argument order in the callable signature must match the order of the declared key dimensions.\n", - "* All key dimensions are defined with a bounded ``range`` or ``values`` parameter (for categorical dimensions)." + "* All key dimensions are defined with a bounded ``range`` or ``values`` parameter (for categorical dimensions).\n", + "\n", + "Here is another example of a bounded ``DynamicMap``:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def shapes(N, radius=0.5): # Positional keyword arguments are fine\n", + " paths = [hv.Path([[(radius*np.sin(a), radius*np.cos(a)) \n", + " for a in np.linspace(-np.pi, np.pi, n+2)]], \n", + " extents=(-1,-1,1,1)) \n", + " for n in range(N,N+3)]\n", + " return hv.Overlay(paths)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "%%opts Path (linewidth=1.5)\n", + "dmap = hv.DynamicMap(shapes, kdims=[hv.Dimension('N', range=(2,20)), hv.Dimension('radius', range=(0.5,1))])\n", + "dmap" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As you can see, you can return ``Overlays`` from ``DynamicMaps`` and ``DynamicMaps`` can be styled in exactly the same way as ``HoloMaps``. Note that currently, ``Overlay`` objects should be returned from the callable and the ``*`` operator is not yet supported at the ``DynamicMap`` level. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "%opts Path (linewidth=1.5)" ] }, { @@ -321,7 +370,7 @@ }, "outputs": [], "source": [ - "dmap[{(0,0.01), (0,0.5), (0.5,0.01), (0.5,0.5)}] # Returns a *new* DynamicMap with the sampled keys in it's cache" + "dmap[{(2,0.5), (2,1.0), (3,0.5), (3,1.0)}] # Returns a *new* DynamicMap with the specified keys in it's cache" ] }, { @@ -343,7 +392,7 @@ }, "outputs": [], "source": [ - "dmap[{0,0.5},{0.01,0.5}]" + "dmap[{2,3},{0.5,1.0}]" ] }, { @@ -404,7 +453,7 @@ }, "outputs": [], "source": [ - "sliced = dmap[0.2:0.8, :]\n", + "sliced = dmap[4:8, :]\n", "sliced" ] }, @@ -423,7 +472,7 @@ }, "outputs": [], "source": [ - "sliced[:, 0.05:0.1]" + "sliced[:, 0.8:1.0]" ] }, { @@ -471,8 +520,8 @@ }, "outputs": [], "source": [ - "hv.DynamicMap(sine_image, kdims=[hv.Dimension('phase',values=[0,0.5]),\n", - " hv.Dimension('frequency', values=[0.01,0.5])])" + "hv.DynamicMap(shapes, kdims=[hv.Dimension('N', values=[2,3,4,5]), \n", + " hv.Dimension('radius', values=[0.7,0.8,0.9,1])])\n" ] }, { @@ -497,7 +546,7 @@ }, "outputs": [], "source": [ - "dmap = hv.DynamicMap(sine_image, kdims=['phase', 'frequency'], sampled=True)\n", + "dmap = hv.DynamicMap(shapes, kdims=['N', 'radius'], sampled=True)\n", "dmap" ] }, @@ -516,7 +565,7 @@ }, "outputs": [], "source": [ - "dmap[{0,0.5},{0.01,0.5}]" + "dmap[{2,3},{0.5,1.0}]" ] }, { @@ -534,9 +583,8 @@ }, "outputs": [], "source": [ - "dmap + hv.HoloMap({(p,f):sine_image(p, f) \n", - " for p in [0,0.5,1,1.5] \n", - " for f in [0.5,0.75]}, kdims=['phase', 'frequency'])" + "(dmap + hv.HoloMap({(N,r):shapes(N, r) for N in [3,4,5] \n", + " for r in [0.5,0.75]}, kdims=['N', 'radius']))" ] }, { From 21e00d833fe7a8eb6e27e91c2fefdc35f33e5ace Mon Sep 17 00:00:00 2001 From: jlstevens Date: Wed, 10 Feb 2016 14:16:06 +0000 Subject: [PATCH 15/19] Updated tutorial with a better example of an infinite DynamicMap --- doc/Tutorials/Dynamic_Map.ipynb | 73 +++++++++++++++++++++++---------- 1 file changed, 51 insertions(+), 22 deletions(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index 0b870cf978..f5aca60cfa 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -583,8 +583,8 @@ }, "outputs": [], "source": [ - "(dmap + hv.HoloMap({(N,r):shapes(N, r) for N in [3,4,5] \n", - " for r in [0.5,0.75]}, kdims=['N', 'radius']))" + "(dmap.relabel('HMap') + hv.HoloMap({(N,r):shapes(N, r) for N in [3,4,5] \n", + " for r in [0.5,0.75]}, kdims=['N', 'radius']).relabel('DMAP'))" ] }, { @@ -628,7 +628,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Our first example will be using an infinite generator, using the fact that phase is an infinite cyclic quantity:" + "Our first example will be using an infinite generator which plots the histogram for a given number of random samples drawn from a Gaussian distribution:" ] }, { @@ -639,17 +639,19 @@ }, "outputs": [], "source": [ - "def sine_gen(phase=0, freq=0.5):\n", - " while True:\n", - " yield hv.Image(np.sin(phase + (freq*x**2+freq*y**2)))\n", - " phase+=0.2" + "def gaussian_histogram(samples, scale):\n", + " frequencies, edges = np.histogram([np.random.normal(scale=scale) \n", + " for i in range(samples)], 20)\n", + " return hv.Histogram(frequencies, edges).relabel('Gaussian distribution')\n", + "\n", + "gaussian_histogram(100,1) + gaussian_histogram(150,1.5) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Just to confirm this is a generator:" + "Lets now use this in the following generator:" ] }, { @@ -660,14 +662,20 @@ }, "outputs": [], "source": [ - "sine_gen()" + "def gaussian_sampler(samples=10, delta=10, scale=1.0):\n", + " np.random.seed(1)\n", + " while True:\n", + " yield gaussian_histogram(samples, scale)\n", + " samples+=delta\n", + " \n", + "gaussian_sampler()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "And now for our infinite ``DynamicMap``:" + "Which allows us to define the following infinite ``DynamicMap``:" ] }, { @@ -678,7 +686,7 @@ }, "outputs": [], "source": [ - "dmap = hv.DynamicMap(sine_gen(), kdims=['phase'])\n", + "dmap = hv.DynamicMap(sample_gaussian(), kdims=['step'])\n", "dmap" ] }, @@ -686,7 +694,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Note that phase is shown as an integer. This is the default behavior and corresponds to the call count (i.e the number of times ``next()`` has been called on the generator. If we want to show the phase value properly, we need our generator to return a (key, element) pair:" + "Note that step is shown as an integer. This is the default behavior and corresponds to the call count (i.e the number of times ``next()`` has been called on the generator. If we want to show the actual number of samples properly, we need our generator to return a (key, element) pair:" ] }, { @@ -697,12 +705,13 @@ }, "outputs": [], "source": [ - "def sine_kv_gen(phase=0, freq=0.5):\n", + "def gaussian_sampler_kv(samples=10, delta=10, scale=1.0):\n", + " np.random.seed(1)\n", " while True:\n", - " yield (phase, hv.Image(np.sin(phase + (freq*x**2+freq*y**2))))\n", - " phase+=0.2\n", + " yield (samples, gaussian_histogram(samples, scale))\n", + " samples+=delta\n", " \n", - "hv.DynamicMap(sine_kv_gen(), kdims=['phase'])" + "hv.DynamicMap(gaussian_sampler_kv(), kdims=['samples'])" ] }, { @@ -723,7 +732,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In open mode, elements are naturally serialized by a linear sequence of ``next()`` calls. This doesn't mean that they can't have multiple key dimensions:" + "In open mode, elements are naturally serialized by a linear sequence of ``next()`` calls but that doesn't mean multiple key dimensions cannot be defined:" ] }, { @@ -734,13 +743,33 @@ }, "outputs": [], "source": [ - "def sine_kv_gen_2D(phase=0, freq=0.5):\n", + "def gaussian_sampler_2D(samples=10, scale=1.0, delta=10):\n", + " np.random.seed(1)\n", " while True:\n", - " yield ((phase, freq), hv.Image(np.sin(phase + (freq*x**2+freq*y**2))))\n", - " phase+=0.2\n", - " freq += 0.01\n", + " yield ((samples, scale), gaussian_histogram(samples, scale))\n", + " samples=(samples + delta) if scale==2 else samples\n", + " scale = 2 if scale == 1 else 1\n", " \n", - "hv.DynamicMap(sine_kv_gen_2D(), kdims=['phase', 'frequency'])" + "dmap = hv.DynamicMap(gaussian_sampler_2D(), kdims=['samples', 'scale'])\n", + "dmap" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here we bin the histogram for two different scale values. Above we can visualize this linear sequence of ``next()`` calls but by casting this open map to a ``HoloMap``, we can obtain a multi-dimensional parameter space that we can freely explore:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "hv.HoloMap(dmap)" ] }, { From 2ea84d05ad99b95ad2e3c9522ac9d2a59596a1dc Mon Sep 17 00:00:00 2001 From: jlstevens Date: Wed, 10 Feb 2016 15:29:27 +0000 Subject: [PATCH 16/19] Added a more interesting example of a finite open DynamicMap --- doc/Tutorials/Dynamic_Map.ipynb | 37 ++++++++++++++++++++++++++------- 1 file changed, 29 insertions(+), 8 deletions(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index f5aca60cfa..97c90fddae 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -583,8 +583,9 @@ }, "outputs": [], "source": [ - "(dmap.relabel('HMap') + hv.HoloMap({(N,r):shapes(N, r) for N in [3,4,5] \n", - " for r in [0.5,0.75]}, kdims=['N', 'radius']).relabel('DMAP'))" + "(dmap.relabel('HoloMap') \n", + " + hv.HoloMap({(N,r):shapes(N, r) for N in [3,4,5] for r in [0.5,0.75]}, \n", + " kdims=['N', 'radius']).relabel('DynamicMap'))" ] }, { @@ -686,7 +687,7 @@ }, "outputs": [], "source": [ - "dmap = hv.DynamicMap(sample_gaussian(), kdims=['step'])\n", + "dmap = hv.DynamicMap(gaussian_sampler(), kdims=['step'])\n", "dmap" ] }, @@ -783,7 +784,29 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Open mode ``DynamicMaps`` are finite and terminate if ``StopIteration`` is raised. This example uses a generator expression that is identical to the first open ``DynamicMap`` example above, except that it terminates after 20 phases:\n" + "Open mode ``DynamicMaps`` are finite and terminate if ``StopIteration`` is raised. This example terminates when two the mean of two sets of gaussian samples fall within a certain distance of each other:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def sample_distributions(samples=10, delta=50, tol=0.04):\n", + " np.random.seed(42)\n", + " while True:\n", + " gauss1 = np.random.normal(size=samples)\n", + " gauss2 = np.random.normal(size=samples)\n", + " data = (['A']*samples + ['B']*samples, np.hstack([gauss1, gauss2]))\n", + " diff = abs(gauss1.mean() - gauss2.mean())\n", + " if abs(gauss1.mean() - gauss2.mean()) > tol:\n", + " yield ((samples, diff), hv.BoxWhisker(data, kdims=['Group'], vdims=['Value']))\n", + " else:\n", + " raise StopIteration\n", + " samples+=delta" ] }, { @@ -794,7 +817,7 @@ }, "outputs": [], "source": [ - "dmap=hv.DynamicMap((hv.Image(np.sin(0.2*phase + (0.5*x**2+0.5*y**2))) for phase in range(21)), kdims=['phase'])\n", + "dmap = hv.DynamicMap(sample_distributions(), kdims=['samples', '$\\delta$'])\n", "dmap" ] }, @@ -802,8 +825,6 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "It is trivial to adapt this generator expression to return (key, tuple) pairs, where the key may have one or more dimensions.\n", - "\n", "Now if you are familiar with generators in Python, you might be wondering what happens when a finite generator is exhausted. First we should mention that casting a ``DynamicMap`` to a list is always finite, because ``__iter__`` returns the cache instead of a potentially infinite generator:" ] }, @@ -822,7 +843,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As we know this ``DynamicMap`` is finite, we can exhaust it as follows:" + "As we know this ``DynamicMap`` is finite, we can make sure it is exhausted it as follows:" ] }, { From 792fbcdba57aebc49cd2b125f5dc02c803ed4bc0 Mon Sep 17 00:00:00 2001 From: jlstevens Date: Wed, 10 Feb 2016 19:02:31 +0000 Subject: [PATCH 17/19] Simplified example of DynamicMap + HoloMap layout --- doc/Tutorials/Dynamic_Map.ipynb | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index 97c90fddae..44af1e491d 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -583,9 +583,7 @@ }, "outputs": [], "source": [ - "(dmap.relabel('HoloMap') \n", - " + hv.HoloMap({(N,r):shapes(N, r) for N in [3,4,5] for r in [0.5,0.75]}, \n", - " kdims=['N', 'radius']).relabel('DynamicMap'))" + "dmap + hv.HoloMap({(N,r):shapes(N, r) for N in [3,4,5] for r in [0.5,0.75]}, kdims=['N', 'radius'])" ] }, { From 239b2cde43d402ae9ba081c0394527cf09cdc1b7 Mon Sep 17 00:00:00 2001 From: jlstevens Date: Wed, 10 Feb 2016 21:04:38 +0000 Subject: [PATCH 18/19] Added section describing normalization of DynamicMaps --- doc/Tutorials/Dynamic_Map.ipynb | 49 ++++++++++++++++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index 44af1e491d..046988d090 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -1162,7 +1162,54 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Using DynamicMaps in your code\n", + "## DynamicMaps and normalization" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "By default, a ``HoloMap`` normalizes the display of elements according the minimum and maximum values found across the ``HoloMap``. This automatic behavior is not possible in a ``DynamicMap`` where arbitrary new elements are being generated on the fly. Consider the following examples where the arrays contained within the returned ``Image`` objects are scaled with time:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "%%opts Image {+axiswise}\n", + "ls = np.linspace(0, 10, 200)\n", + "xx, yy = np.meshgrid(ls, ls)\n", + "\n", + "def cells(vrange=False):\n", + " \"The range is set on the value dimension when vrange is True \"\n", + " time = time_gen()\n", + " while True:\n", + " t = time.next()\n", + " arr = t*np.sin(xx+t)*np.cos(yy+t)\n", + " vdims=[hv.Dimension('Intensity', range=(0,10))] if vrange else ['Intensity']\n", + " yield hv.Image(arr, vdims=vdims)\n", + "\n", + "hv.DynamicMap(cells(vrange=False), kdims=['time']) + hv.DynamicMap(cells(vrange=True), kdims=['time'])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here we use ``+axiswise`` to see the behavior of the two cases independently. We see in **A** that when ``vrange=False`` and no range is set on the value dimension, no automatic normalization occurs (as it would for a ``HoloMap``). In **B** we see that normalization is applied but only when the value dimension range has been specified. \n", + "\n", + "In other words ``DynamicMaps`` do not support automatic normalization across their elements but do support the same explicit normalization behavior as ``HoloMaps``. Values that are generated outside this range are simply clipped according the usual semantics of explicit value dimension ranges." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using DynamicMaps in your code\n", "\n", "As you can see, ``DynamicMaps`` let you use HoloViews with a very wide range of dynamic data formats and sources, making it simple to visualize ongoing processes or very large data spaces." ] From 88a080f5d1bf3e5008de4e98c62bf4f2a9a31f8c Mon Sep 17 00:00:00 2001 From: jlstevens Date: Wed, 10 Feb 2016 21:17:45 +0000 Subject: [PATCH 19/19] Added comment about normalizing by casting to HoloMap --- doc/Tutorials/Dynamic_Map.ipynb | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/doc/Tutorials/Dynamic_Map.ipynb b/doc/Tutorials/Dynamic_Map.ipynb index 046988d090..dda175a158 100644 --- a/doc/Tutorials/Dynamic_Map.ipynb +++ b/doc/Tutorials/Dynamic_Map.ipynb @@ -521,7 +521,7 @@ "outputs": [], "source": [ "hv.DynamicMap(shapes, kdims=[hv.Dimension('N', values=[2,3,4,5]), \n", - " hv.Dimension('radius', values=[0.7,0.8,0.9,1])])\n" + " hv.Dimension('radius', values=[0.7,0.8,0.9,1])])" ] }, { @@ -1202,7 +1202,9 @@ "source": [ "Here we use ``+axiswise`` to see the behavior of the two cases independently. We see in **A** that when ``vrange=False`` and no range is set on the value dimension, no automatic normalization occurs (as it would for a ``HoloMap``). In **B** we see that normalization is applied but only when the value dimension range has been specified. \n", "\n", - "In other words ``DynamicMaps`` do not support automatic normalization across their elements but do support the same explicit normalization behavior as ``HoloMaps``. Values that are generated outside this range are simply clipped according the usual semantics of explicit value dimension ranges." + "In other words ``DynamicMaps`` do not support automatic normalization across their elements but do support the same explicit normalization behavior as ``HoloMaps``. Values that are generated outside this range are simply clipped according the usual semantics of explicit value dimension ranges. \n", + "\n", + "Note that we can always have the option of casting a ``DynamicMap`` to a ``HoloMap`` in order to automatically normalize across the cached values without needing explicit value dimension ranges." ] }, {