-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Renamed to pyviz_comms and added Python package #5
Merged
+523
−6
Merged
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
b2f4c4e
Added __pycache__ to .gitignore
jlstevens 2ddcb79
Added pyviz_comms/__init__.py
jlstevens 338b5ef
Added setup.py for python package
jlstevens ab5f0f5
Added conda.recipe directory
jlstevens b3d899b
Updated the README.md
jlstevens aa79580
Renamed HoloViews global object to PyViz in the code
jlstevens 27c27f0
Made PYVIZ_PROXY JavaScript block available for import
jlstevens 3f3e5a2
Updated docstring
jlstevens 7276bcd
Added temporary alias till next jupyterlab extension release
jlstevens b4cc4c9
Improved definition of PyViz proxy
jlstevens 6825f95
Fixed issue with PyViz object being undefined
jlstevens 7e1ef97
Updated meta.yaml homepage URL and build number
jlstevens 5f7100c
Re-indented JS_CALLBACK block
jlstevens e9f2157
Made Javascript indentation consistent
jlstevens 508a343
Set the pyviz_comms package version to 0.1.0
jlstevens File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,3 +2,4 @@ | |
lib/ | ||
node_modules/ | ||
*.egg-info/ | ||
__pycache__ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
{% set sdata = load_setup_py_data() %} | ||
|
||
package: | ||
name: pyviz_comms | ||
version: {{ sdata['version'] }} | ||
|
||
source: | ||
path: .. | ||
|
||
build: | ||
noarch: python | ||
number: 1 | ||
script: python setup.py --quiet install --single-version-externally-managed --record record.txt | ||
|
||
|
||
requirements: | ||
build: | ||
- python | ||
- setuptools | ||
run: | ||
- python | ||
|
||
test: | ||
imports: | ||
- pyviz_comms | ||
|
||
about: | ||
home: www.pyviz.org | ||
summary: Bidirectional communication for PyViz | ||
license: BSD 3-Clause |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,436 @@ | ||
import json | ||
import uuid | ||
import sys | ||
import traceback | ||
|
||
try: | ||
from StringIO import StringIO | ||
except: | ||
from io import StringIO | ||
|
||
|
||
__version__ = '0.1.0' | ||
|
||
PYVIZ_PROXY = """ | ||
if (window.PyViz === undefined) { | ||
if (window.HoloViews === undefined) { | ||
var PyViz = {comms: {}, comm_status:{}, kernels:{}, receivers: {}, plot_index: []} | ||
} else { | ||
var PyViz = window.HoloViews; | ||
} | ||
window.PyViz = PyViz; | ||
window.HoloViews = PyViz; // TEMPORARY HACK TILL NEXT NPM RELEASE | ||
} | ||
""" | ||
|
||
|
||
# Following JS block becomes body of the message handler callback | ||
bokeh_msg_handler = """ | ||
var plot_id = "{plot_id}"; | ||
if (plot_id in window.PyViz.plot_index) {{ | ||
var plot = window.PyViz.plot_index[plot_id]; | ||
}} else {{ | ||
var plot = Bokeh.index[plot_id]; | ||
}} | ||
if (plot_id in window.PyViz.receivers) {{ | ||
var receiver = window.PyViz.receivers[plot_id]; | ||
}} else if (Bokeh.protocol === undefined) {{ | ||
return; | ||
}} else {{ | ||
var receiver = new Bokeh.protocol.Receiver(); | ||
window.PyViz.receivers[plot_id] = receiver; | ||
}} | ||
if (buffers.length > 0) {{ | ||
receiver.consume(buffers[0].buffer) | ||
}} else {{ | ||
receiver.consume(msg) | ||
}} | ||
const comm_msg = receiver.message; | ||
if (comm_msg != null) {{ | ||
plot.model.document.apply_json_patch(comm_msg.content, comm_msg.buffers) | ||
}} | ||
""" | ||
|
||
|
||
JS_CALLBACK = """ | ||
function unique_events(events) {{ | ||
// Processes the event queue ignoring duplicate events | ||
// of the same type | ||
var unique = []; | ||
var unique_events = []; | ||
for (var i=0; i<events.length; i++) {{ | ||
var _tmpevent = events[i]; | ||
event = _tmpevent[0]; | ||
data = _tmpevent[1]; | ||
if (unique_events.indexOf(event)===-1) {{ | ||
unique.unshift(data); | ||
unique_events.push(event); | ||
}} | ||
}} | ||
return unique; | ||
}} | ||
function process_events(comm_status) {{ | ||
// Iterates over event queue and sends events via Comm | ||
var events = unique_events(comm_status.event_buffer); | ||
for (var i=0; i<events.length; i++) {{ | ||
var data = events[i]; | ||
var comm = window.PyViz.comms[data["comm_id"]]; | ||
comm.send(data); | ||
}} | ||
comm_status.event_buffer = []; | ||
}} | ||
function on_msg(msg) {{ | ||
// Receives acknowledgement from Python, processing event | ||
// and unblocking Comm if event queue empty | ||
msg = JSON.parse(msg.content.data); | ||
var comm_id = msg["comm_id"] | ||
var comm_status = window.PyViz.comm_status[comm_id]; | ||
if (comm_status.event_buffer.length) {{ | ||
process_events(comm_status); | ||
comm_status.blocked = true; | ||
comm_status.time = Date.now()+{debounce}; | ||
}} else {{ | ||
comm_status.blocked = false; | ||
}} | ||
comm_status.event_buffer = []; | ||
if ((msg.msg_type == "Ready") && msg.content) {{ | ||
console.log("Python callback returned following output:", msg.content); | ||
}} else if (msg.msg_type == "Error") {{ | ||
console.log("Python failed with the following traceback:", msg['traceback']) | ||
}} | ||
}} | ||
// Initialize Comm | ||
comm = window.PyViz.comm_manager.get_client_comm("{plot_id}", "{comm_id}", on_msg); | ||
if (!comm) {{ | ||
return | ||
}} | ||
// Initialize event queue and timeouts for Comm | ||
var comm_status = window.PyViz.comm_status["{comm_id}"]; | ||
if (comm_status === undefined) {{ | ||
comm_status = {{event_buffer: [], blocked: false, time: Date.now()}} | ||
window.PyViz.comm_status["{comm_id}"] = comm_status | ||
}} | ||
// Add current event to queue and process queue if not blocked | ||
event_name = cb_obj.event_name | ||
data['comm_id'] = "{comm_id}"; | ||
timeout = comm_status.time + {timeout}; | ||
if ((comm_status.blocked && (Date.now() < timeout))) {{ | ||
comm_status.event_buffer.unshift([event_name, data]); | ||
}} else {{ | ||
comm_status.event_buffer.unshift([event_name, data]); | ||
setTimeout(function() {{ process_events(comm_status); }}, {debounce}); | ||
comm_status.blocked = true; | ||
comm_status.time = Date.now()+{debounce}; | ||
}} | ||
""" | ||
|
||
|
||
class StandardOutput(list): | ||
""" | ||
Context manager to capture standard output for any code it | ||
is wrapping and make it available as a list, e.g.: | ||
>>> with StandardOutput() as stdout: | ||
... print('This gets captured') | ||
>>> print(stdout[0]) | ||
This gets captured | ||
""" | ||
|
||
def __enter__(self): | ||
self._stdout = sys.stdout | ||
sys.stdout = self._stringio = StringIO() | ||
return self | ||
|
||
def __exit__(self, *args): | ||
self.extend(self._stringio.getvalue().splitlines()) | ||
sys.stdout = self._stdout | ||
|
||
|
||
class Comm(object): | ||
""" | ||
Comm encompasses any uni- or bi-directional connection between | ||
a python process and a frontend allowing passing of messages | ||
between the two. A Comms class must implement methods | ||
send data and handle received message events. | ||
If the Comm has to be set up on the frontend a template to | ||
handle the creation of the comms channel along with a message | ||
handler to process incoming messages must be supplied. | ||
The template must accept three arguments: | ||
* id - A unique id to register to register the comm under. | ||
* msg_handler - JS code which has the msg variable in scope and | ||
performs appropriate action for the supplied message. | ||
* init_frame - The initial frame to render on the frontend. | ||
""" | ||
|
||
html_template = """ | ||
<div id="fig_{plot_id}"> | ||
{init_frame} | ||
</div> | ||
""" | ||
|
||
js_template = '' | ||
|
||
def __init__(self, id=None, on_msg=None): | ||
""" | ||
Initializes a Comms object | ||
""" | ||
self.id = id if id else uuid.uuid4().hex | ||
self._on_msg = on_msg | ||
self._comm = None | ||
|
||
|
||
def init(self, on_msg=None): | ||
""" | ||
Initializes comms channel. | ||
""" | ||
|
||
|
||
def send(self, data=None, buffers=[]): | ||
""" | ||
Sends data to the frontend | ||
""" | ||
|
||
|
||
@classmethod | ||
def decode(cls, msg): | ||
""" | ||
Decode incoming message, e.g. by parsing json. | ||
""" | ||
return msg | ||
|
||
|
||
@property | ||
def comm(self): | ||
if not self._comm: | ||
raise ValueError('Comm has not been initialized') | ||
return self._comm | ||
|
||
|
||
def _handle_msg(self, msg): | ||
""" | ||
Decode received message before passing it to on_msg callback | ||
if it has been defined. | ||
""" | ||
comm_id = None | ||
try: | ||
stdout = [] | ||
msg = self.decode(msg) | ||
comm_id = msg.pop('comm_id', None) | ||
if self._on_msg: | ||
# Comm swallows standard output so we need to capture | ||
# it and then send it to the frontend | ||
with StandardOutput() as stdout: | ||
self._on_msg(msg) | ||
except Exception as e: | ||
frame =traceback.extract_tb(sys.exc_info()[2])[-2] | ||
fname,lineno,fn,text = frame | ||
error_kwargs = dict(type=type(e).__name__, fn=fn, fname=fname, | ||
line=lineno, error=str(e)) | ||
error = '{fname} {fn} L{line}\n\t{type}: {error}'.format(**error_kwargs) | ||
if stdout: | ||
stdout = '\n\t'+'\n\t'.join(stdout) | ||
error = '\n'.join([stdout, error]) | ||
reply = {'msg_type': "Error", 'traceback': error} | ||
else: | ||
stdout = '\n\t'+'\n\t'.join(stdout) if stdout else '' | ||
reply = {'msg_type': "Ready", 'content': stdout} | ||
|
||
# Returning the comm_id in an ACK message ensures that | ||
# the correct comms handle is unblocked | ||
if comm_id: | ||
reply['comm_id'] = comm_id | ||
self.send(json.dumps(reply)) | ||
|
||
|
||
class JupyterComm(Comm): | ||
""" | ||
JupyterComm provides a Comm for the notebook which is initialized | ||
the first time data is pushed to the frontend. | ||
""" | ||
|
||
js_template = """ | ||
function msg_handler(msg) {{ | ||
var buffers = msg.buffers; | ||
var msg = msg.content.data; | ||
{msg_handler} | ||
}} | ||
window.PyViz.comm_manager.register_target('{plot_id}', '{comm_id}', msg_handler); | ||
""" | ||
|
||
def init(self): | ||
from ipykernel.comm import Comm as IPyComm | ||
if self._comm: | ||
return | ||
self._comm = IPyComm(target_name=self.id, data={}) | ||
self._comm.on_msg(self._handle_msg) | ||
|
||
|
||
@classmethod | ||
def decode(cls, msg): | ||
""" | ||
Decodes messages following Jupyter messaging protocol. | ||
If JSON decoding fails data is assumed to be a regular string. | ||
""" | ||
return msg['content']['data'] | ||
|
||
|
||
def send(self, data=None, buffers=[]): | ||
""" | ||
Pushes data across comm socket. | ||
""" | ||
if not self._comm: | ||
self.init() | ||
self.comm.send(data, buffers=buffers) | ||
|
||
|
||
|
||
class JupyterCommJS(JupyterComm): | ||
""" | ||
JupyterCommJS provides a comms channel for the Jupyter notebook, | ||
which is initialized on the frontend. This allows sending events | ||
initiated on the frontend to python. | ||
""" | ||
|
||
js_template = """ | ||
<script> | ||
function msg_handler(msg) {{ | ||
var msg = msg.content.data; | ||
var buffers = msg.buffers | ||
{msg_handler} | ||
}} | ||
comm = window.PyViz.comm_manager.get_client_comm("{comm_id}"); | ||
comm.on_msg(msg_handler); | ||
</script> | ||
""" | ||
|
||
def __init__(self, id=None, on_msg=None): | ||
""" | ||
Initializes a Comms object | ||
""" | ||
from IPython import get_ipython | ||
super(JupyterCommJS, self).__init__(id, on_msg) | ||
self.manager = get_ipython().kernel.comm_manager | ||
self.manager.register_target(self.id, self._handle_open) | ||
|
||
|
||
def _handle_open(self, comm, msg): | ||
self._comm = comm | ||
self._comm.on_msg(self._handle_msg) | ||
|
||
|
||
def send(self, data=None, buffers=[]): | ||
""" | ||
Pushes data across comm socket. | ||
""" | ||
self.comm.send(data, buffers=buffers) | ||
|
||
|
||
|
||
class CommManager(object): | ||
""" | ||
The CommManager is an abstract baseclass for establishing | ||
websocket comms on the client and the server. | ||
""" | ||
|
||
js_manager = """ | ||
function CommManager() { | ||
} | ||
CommManager.prototype.register_target = function() { | ||
} | ||
CommManager.prototype.get_client_comm = function() { | ||
} | ||
window.PyViz.comm_manager = CommManager() | ||
""" | ||
|
||
_comms = {} | ||
|
||
server_comm = Comm | ||
|
||
client_comm = Comm | ||
|
||
@classmethod | ||
def get_server_comm(cls, on_msg=None, id=None): | ||
comm = cls.server_comm(id, on_msg) | ||
cls._comms[comm.id] = comm | ||
return comm | ||
|
||
@classmethod | ||
def get_client_comm(cls, on_msg=None, id=None): | ||
comm = cls.client_comm(id, on_msg) | ||
cls._comms[comm.id] = comm | ||
return comm | ||
|
||
|
||
|
||
class JupyterCommManager(CommManager): | ||
""" | ||
The JupyterCommManager is used to establishing websocket comms on | ||
the client and the server via the Jupyter comms interface. | ||
There are two cases for both the register_target and get_client_comm | ||
methods: one to handle the classic notebook frontend and one to | ||
handle JupyterLab. The latter case uses the globally available PyViz | ||
object which is made available by each PyViz project requiring the | ||
use of comms. This object is handled in turn by the JupyterLab | ||
extension which keeps track of the kernels associated with each | ||
plot, ensuring the corresponding comms can be accessed. | ||
""" | ||
|
||
js_manager = """ | ||
function JupyterCommManager() { | ||
} | ||
JupyterCommManager.prototype.register_target = function(plot_id, comm_id, msg_handler) { | ||
if (window.comm_manager || ((window.Jupyter !== undefined) && (Jupyter.notebook.kernel != null))) { | ||
var comm_manager = window.comm_manager || Jupyter.notebook.kernel.comm_manager; | ||
comm_manager.register_target(comm_id, function(comm) { | ||
comm.on_msg(msg_handler); | ||
}); | ||
} else if ((plot_id in window.PyViz.kernels) && (window.PyViz.kernels[plot_id])) { | ||
window.PyViz.kernels[plot_id].registerCommTarget(comm_id, function(comm) { | ||
comm.onMsg = msg_handler; | ||
}); | ||
} | ||
} | ||
JupyterCommManager.prototype.get_client_comm = function(plot_id, comm_id, msg_handler) { | ||
if (comm_id in window.PyViz.comms) { | ||
return window.PyViz.comms[comm_id]; | ||
} else if (window.comm_manager || ((window.Jupyter !== undefined) && (Jupyter.notebook.kernel != null))) { | ||
var comm_manager = window.comm_manager || Jupyter.notebook.kernel.comm_manager; | ||
var comm = comm_manager.new_comm(comm_id, {}, {}, {}, comm_id); | ||
if (msg_handler) { | ||
comm.on_msg(msg_handler); | ||
} | ||
} else if ((plot_id in window.PyViz.kernels) && (window.PyViz.kernels[plot_id])) { | ||
var comm = window.PyViz.kernels[plot_id].connectToComm(comm_id); | ||
comm.open(); | ||
if (msg_handler) { | ||
comm.onMsg = msg_handler; | ||
} | ||
} | ||
window.PyViz.comms[comm_id] = comm; | ||
return comm; | ||
} | ||
window.PyViz.comm_manager = new JupyterCommManager(); | ||
""" | ||
|
||
server_comm = JupyterComm | ||
|
||
client_comm = JupyterCommJS |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
#!/usr/bin/env python | ||
|
||
import sys, os | ||
try: | ||
from setuptools import setup | ||
except ImportError: | ||
from distutils.core import setup | ||
|
||
install_requires = [] | ||
setup_args = {} | ||
setup_args.update(dict( | ||
name='pyviz_comms', | ||
version="0.1.0", | ||
install_requires = install_requires, | ||
description='Launch jobs, organize the output, and dissect the results.', | ||
long_description=open('README.md').read() if os.path.isfile('README.md') else 'Consult README.md', | ||
author= "PyViz developers", | ||
author_email= "", | ||
maintainer= "PyViz", | ||
maintainer_email= "holoviews@gmail.com", | ||
platforms=['Windows', 'Mac OS X', 'Linux'], | ||
license='BSD', | ||
url='http://pyviz.org', | ||
packages = ["pyviz_comms"], | ||
classifiers = [ | ||
"License :: OSI Approved :: BSD License", | ||
"Development Status :: 5 - Production/Stable", | ||
"Programming Language :: Python :: 2.7", | ||
"Programming Language :: Python :: 3.3", | ||
"Programming Language :: Python :: 3.4", | ||
"Operating System :: OS Independent", | ||
"Intended Audience :: Science/Research", | ||
"Intended Audience :: Developers", | ||
"Natural Language :: English", | ||
"Topic :: Scientific/Engineering", | ||
"Topic :: Software Development :: Libraries"] | ||
)) | ||
|
||
|
||
if __name__=="__main__": | ||
setup(**setup_args) |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consistent spacing would be nice if you're already making the change above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in e9f2157