Skip to content

Commit

Permalink
support write_only flag for external processes
Browse files Browse the repository at this point in the history
  • Loading branch information
miguelgrinberg committed Jan 10, 2016
1 parent f9f54f9 commit 7d3c8ff
Show file tree
Hide file tree
Showing 3 changed files with 66 additions and 36 deletions.
66 changes: 44 additions & 22 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -604,42 +604,64 @@ There are two requirements to use multiple Flask-SocketIO workers:

- Since each of the servers owns only a subset of the client connections, a
message queue such as Redis or RabbitMQ is used by the servers to coordinate
complex operations such as broadcasting or rooms. The Kombu package is used
to access the message queue, so it must be installed (i.e. ``pip install
kombu``). Depending on the selected message queue, Kombu may require
additional packages to be installed.
complex operations such as broadcasting or rooms.

To start multiple Flask-SocketIO servers, you must first ensure you have a
compatible message queue running. The supported list of message queues can be
seen in the documentation for `Kombu
<http://docs.celeryproject.org/projects/kombu/en/latest/introduction.html#transport-comparison>`_.
Note that if you are using Redis, Kombu needs its Python package to also be
installed (i.e. ``pip install redis``).
When working with a message queue, there are additional dependencies that need to
be installed:

To start a Socket.IO server and have it connect to the message queue, add
the ``message_queue`` argument to the ``SocketIO`` constructor::
- For Redis, the package ``redis`` must be installed (``pip install redis``).
- For RabbitMQ, the package ``kombu`` must be installed (``pip install kombu``).
- For ther message queues supported by Kombu, see the `Kombu documentation
<http://docs.celeryproject.org/projects/kombu/en/latest/introduction.html#transport-comparison>`_
to find out what dependencies are needed.

To start multiple Flask-SocketIO servers, you must first ensure you have the
message queue service running. To start a Socket.IO server and have it connect to
the message queue, add the ``message_queue`` argument to the ``SocketIO``
constructor::

socketio = SocketIO(app, message_queue='redis://')

The value of the ``message_queue`` argument is the connection URL of the
queue service that is used. The Kombu package has a `documentation section
<http://docs.celeryproject.org/projects/kombu/en/latest/userguide/connections.html?highlight=urls#urls>`_
queue service that is used. For a redis queue running on the same host as the
server, the ``'redis://`` URL can be used. Likewise, for a default RabbitMQ
queue the ``'amqp://'`` URL can be used. The Kombu package has a `documentation
section <http://docs.celeryproject.org/projects/kombu/en/latest/userguide/connections.html?highlight=urls#urls>`_
that describes the format of the URLs for all the supported queues.

Emitting from an External Process
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For many types of applications, it is necessary to emit events from a process
that is not the SocketIO server, for a example a Celery worker. To do this,
the external process must create a ``SocketIO`` instances similar to the one
used by the main process. For example, if the application is using a Redis
message queue, the following Python script can broadcast an event to all
clients::

app = create_app()
socketio = SocketIO(app, message_queue='redis://')
that is not the SocketIO server, for a example a Celery worker. If the
SocketIO server or servers are configured to listen on a message queue as
shown in the previous section, then any other process can create its own
``SocketIO`` instance and use it to emit events in the same way the server
does.

For example, for an application that runs on an eventlet web server and uses
a Redis message queue, the following Python script can broadcast an event to
all clients::

import eventlet
eventlet.monkey_patch()
socketio = SocketIO(message_queue='redis://')
socketio.emit('my event', {'data': 'foo'}, namespace='/test')

Note that a Flask application instance is not required when initializing a
``SocketIO`` object that is only going to be used to emit events and not as a
server.

The ``channel`` argument to ``SocketIO`` can be used to select a specific
channel of communication through the message queue. Using a custom channel
name is necessary when there are multiple independent SocketIO services
sharing the same queue.

Note that Flask-SocketIO does not apply monkey patching when eventlet or
gevent are used. When working with a message queue, it is very likely that
the Python package that talks to the message queue service will hang if the
Python standard library is not monkey patched.

API Reference
-------------

Expand Down
34 changes: 21 additions & 13 deletions flask_socketio/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,11 @@ class SocketIO(object):
server can use for multi-process communication. A
message queue is not required when using a single
server process.
:param channel: The channel name, when using a message queue. Normally this
does not need to be set, but if multiple clusters of
processes need to use the same message queue, then each
group should use a different channel so that they do not
interfere.
:param channel: The channel name, when using a message queue. If a channel
isn't specified, a default channel will be used. If
multiple clusters of SocketIO processes need to use the
same message queue without interfering with each other, then
each cluster should use a different channel.
:param resource: The SocketIO resource name. Defaults to ``'socket.io'``.
Leave this as is unless you know what you are doing.
:param kwargs: Socket.IO and Engine.IO server options.
Expand Down Expand Up @@ -115,20 +115,27 @@ def __init__(self, app=None, **kwargs):
self.handlers = []
self.exception_handlers = {}
self.default_exception_handler = None
if app is not None:
if app is not None or len(kwargs) > 0:
self.init_app(app, **kwargs)

def init_app(self, app, **kwargs):
if not hasattr(app, 'extensions'):
app.extensions = {} # pragma: no cover
app.extensions['socketio'] = self
if app is not None:
if not hasattr(app, 'extensions'):
app.extensions = {} # pragma: no cover
app.extensions['socketio'] = self
self.server_options = kwargs

if 'client_manager' not in self.server_options:
url = kwargs.pop('message_queue', None)
channel = kwargs.pop('channel', None)
channel = kwargs.pop('channel', 'flask-socketio')
write_only = app is None
if url:
queue = socketio.KombuManager(url, channel=channel)
if url.startswith('redis://'):
queue_class = socketio.RedisManager
else:
queue_class = socketio.KombuManager
queue = queue_class(url, channel=channel,
write_only=write_only)
self.server_options['client_manager'] = queue

resource = kwargs.pop('resource', 'socket.io')
Expand All @@ -137,8 +144,9 @@ def init_app(self, app, **kwargs):
self.server = socketio.Server(**self.server_options)
for handler in self.handlers:
self.server.on(handler[0], handler[1], namespace=handler[2])
app.wsgi_app = _SocketIOMiddleware(self.server, app,
socketio_path=resource)
if app is not None:
app.wsgi_app = _SocketIOMiddleware(self.server, app,
socketio_path=resource)

def on(self, message, namespace=None):
"""Decorator to register a SocketIO event handler.
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
platforms='any',
install_requires=[
'Flask>=0.9',
'python-socketio>=0.8.2',
'python-socketio>=0.9.0',
'python-engineio>=0.8.2'
],
tests_require=[
Expand Down

0 comments on commit 7d3c8ff

Please sign in to comment.