run with SubdomainDispatcher

[j-walker23]

Hey, wondering if there is a way to run flask-socketio with the flask SubdomainDispatcher. Code from http://flask.pocoo.org/docs/0.11/patterns/appdispatch/#dispatch-by-subdomain

The way i am running it now

def make_app(sub):
  if sub == 'admin':
     return admin.create_app()
  return web.create_app()

app = SubdomainDispatcher('domain.com', make_app)  # type: Flask

if __name__ == '__main__':
  run_simple('0.0.0.0', 8070, app,
    use_reloader=True, use_debugger=True, use_evalex=False)

I can't figure out how to make socketio.run(app) work since i don't new up the app like the normal app.run() method.

[miguelgrinberg]

I think you should be able to use a subdomain dispatcher. You will not be able to start the app using socketio.run() though, as that does not have a way to insert a custom middleware. But as long as you start the web server appropriately after you set up the subdomain middleware in front of the app everything should work.

The socket.run() method doesn't really do magic, it just makes it easier to start the web server, which can be Werkzeug, eventlet or gevent, depending on your choices. You can take the code that start the web server from this method and use it to start the app with the added middleware.

[j-walker23]

Thanks for the quick reply! Could you please explain your last sentence a little bit?

You can take the code that start the web server from this method and use it to start the app with the added middleware.

Are you talking about taking the code from the socketio.run method?

I tried doing a couple different things. But i wasn't sure when to call init_app since it seems to be needed before running.

[miguelgrinberg]

So you have a create_app() function that you are using with your subdomain middleware, correct?

Inside this factory function you create your app, and call init_app() on all your extensions. This is where you will also initialize socket.io, like all other extensions.

Then you will create a subdomain dispatcher, exactly as in the code you posted above. But instead of running run_simple(), you have to start the web server that you are using, in the way that is appropriate for socket.io to work. You did not say if you are using eventlet, gevent or Werkzeug. The socketio.run() function has the code to start all three, so you just need to find the block of code that applies to the webserver that you are using, and copy that code into your if __name__ == '__main__': block.

[j-walker23]

Nice, thank you for explaining that. I think i understand what you are saying, i hope i can explain my issue.

I am using eventlet. And i thought that if section of the socketio.run method needed self.server setup, which happens in the init_app.

But the create_app of the subdomain middleware doesn't get called until the first request comes through.

I think that was my ultimate problem after digging the other day. I couldn't figure out how to run the socketio server, without first calling create_app/init_app. And since i don't call create_app during startup i was in a infinite circular logic problem lol

[miguelgrinberg]

I don't see any references to self.server in the eventlet start up code. All you need to start the eventlet server is the "app", which can be any WSGI app or middleware. In this case the app referenced by eventlet is your subdomain middleware instance, which you conveniently also call app.

To make sure we are talking about the same thing, this is the chunk of code that you need: https://github.com/miguelgrinberg/Flask-SocketIO/blob/master/flask_socketio/__init__.py#L436-L459. The middle part that deals with SSL can probably be removed, unless you care about SSL, of course.

[j-walker23]

Oh sorry, you were talking about the eventlet code.
I was talking about self.server = socketio.Server(**self.server_options) in the init_app method.

Thanks for bearing with me. One last question. Are you saying to just run this section (eventlet part) of the run method?

            def run_server():
                import eventlet
                eventlet_socket = eventlet.listen((host, port))

                # If provided an SSL argument, use an SSL socket
                ssl_args = ['keyfile', 'certfile', 'server_side', 'cert_reqs',
                            'ssl_version', 'ca_certs',
                            'do_handshake_on_connect', 'suppress_ragged_eofs',
                            'ciphers']
                ssl_params = {k: kwargs[k] for k in kwargs if k in ssl_args}
                if len(ssl_params) > 0:
                    for k in ssl_params:
                        kwargs.pop(k)
                    ssl_params['server_side'] = True  # Listening requires true
                    eventlet_socket = eventlet.wrap_ssl(eventlet_socket,
                                                        **ssl_params)

                eventlet.wsgi.server(eventlet_socket, app,
                                     log_output=log_output, **kwargs)

            if use_reloader:
                run_with_reloader(run_server, extra_files=extra_files)
            else:
                run_server()

[miguelgrinberg]

The self.server part will run when you call init_app from your application factory function. That definitely needs to happen, but it is not something you need to worry about outside of just making sure you initialize the extension along with your other extensions.

I think all you need to start the eventlet server is a subset of that code I linked. Assuming you don't care about SSL or the reloader, you just need this:

import eventlet
eventlet_socket = eventlet.listen((host, port))
eventlet.wsgi.server(eventlet_socket, app)

The app in this example is your subdomain middleware instance.

[j-walker23]

Perfect, I get it now. Really appreciate your help. Thank you!
On Wed, Aug 17, 2016 at 7:29 PM Miguel Grinberg [email protected]
wrote:

The self.server part will run when you call init_app from your
application factory function. That definitely needs to happen, but it is
not something you need to worry about outside of just making sure you
initialize the extension along with your other extensions.

I think all you need to start the eventlet server is a subset of that code
I linked. Assuming you don't care about SSL or the reloader, you just need
this:

import eventlet
eventlet_socket = eventlet.listen((host, port))
eventlet.wsgi.server(eventlet_socket, app)

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#309 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB7ORdOQkS5fTXyZhIvJ1TsRWCaKWlVNks5qg7V3gaJpZM4Jm9n7
.

[j-walker23]

@miguelgrinberg Hey, sorry one more question please.

I am getting a AttributeError: module 'eventlet' has no attribute 'wsgi'. I can't figure out why the socketio.run method works and eventlet.wsgi is valid but mine isn't.

def dev_server(host: str, port: int, app: SubdomainDispatcher, use_reloader=False, **kwargs):
  from werkzeug.debug import DebuggedApplication
  app = DebuggedApplication(app, evalex=True)

  def run_server():
    import eventlet
    eventlet_socket = eventlet.listen((host, port))

    eventlet.wsgi.server(eventlet_socket, app,
      log_output=False, **kwargs)

  if use_reloader:
    run_with_reloader(run_server, extra_files=None)
  else:
    run_server()

app = SubdomainDispatcher('domain.com', make_app)  # type: Flask

if __name__ == '__main__':
  dev_server('0.0.0.0', 8070, app,
    use_reloader=True)

[miguelgrinberg]

This is a weird one, yes. I did not realize this, but when you say import evenlet that does not import eventlet.wsgi automatically. You have to say import eventlet.wsgi to get the wsgi subpackage. This is due to how the eventlet package is structured.

The odd part is that in my examples I create a server, which makes the socket.io code import a bunch of things, including eventlet.wsgi, so then it is available without having to import it explicitly in the application layer. In your case this implicit import occurs later, when the create_app() function is invoked.

So the proper start up sequence is then:

import eventlet
import eventlet.wsgi
eventlet_socket = eventlet.listen((host, port))
eventlet.wsgi.server(eventlet_socket, app,
  log_output=False, **kwargs)

And I need to make the same change in my examples, so that I don't take advantage of an import that occurs as a side effect.

[j-walker23]

You are the man. Thanks again. That was something i probably should have been able to figure out. I was thinking the wsgi attribute was getting dynamically set by your code instead of it just being part of the eventlet module. Noob.

Hopefully last one. Everything started and seemed great. My connection events fired on both server and client. But for some reason when i do a flask_socketio.emit the client doesn't see it. I retried the same code with the normal run method with no Subdomain and it all works.

Very small code, but it looks something like this. As always, thanks for any help. I am super excited about this integration.

# I have logging in here and it does join the room successfully
@sio.on('join account')
def account_connect(js):
  join_room(js['account_id'])

# later on a non context aware emit to room
# This is received by the listener below when not using the new custom run method
sio.emit('event', {'msg': 'Cool msg'}, room=account_id)

// The client
this.sio = io.connect()
this.sio.on('connect', () => this.sio.emit('join account', { account_id: acc.id }))

this.sio.on('event', data => {
  console.log('data', data)
})

Because it does work the old way, i was thinking maybe i am missing a important piece in the code i skipped. Flask/sockets are new to me so i am not even sure where to start debugging or i wouldn't be bothering you.

[miguelgrinberg]

Add logger=True, engineio_logger=True to your SocketIO constructor. That will dump some information to your console. Then do a run with the subdomain and capture the output, and another with the working version based on socketio.run() and again capture the output. Then we can compare and see where's the difference.

[j-walker23]

Hey, you mean this right?

sio = SocketIO(
  logger=True,
  engineio_logger=True
)

This was super weird, and it took me a while to figure it out. I thought i was my other code.
With those kwargs added, i do get more logs. But none of my socket messages work on the server. connect, custom event, nothing. I tried both kwargs separately too.
That is weird huh?

All tested with socketio.run() which works if i use sio = SocketIO() with no kwargs.

[miguelgrinberg]

Are you saying that just by adding these two logging arguments your application breaks? That would be very odd.

[j-walker23]

Lol i know. I spent an hour making sure because i know how stupid it sounds.
And thats because it was stupid, i forgot about init_app. For whatever reason the logging kwargs in SocketIO() prevents the server from working in my demo. But doing init_app(app, logger=True, engineio_logger=True) does work. So i can finally give you the logs.

Also i have some more information that might help. The last log emitting event "finish pubsub" that doesn't fire on the subdomain version is in a webhook callback. Specifically a Google PubSub task. (I am running Flask on a Google Compute Engine box)

The flow for this demo is -

1. Server starts and client is put in a room.
2. User clicks button in UI, server emits start pubsub and then creates the pubsub task.
3. When this offline task completes the server emits finish pubsub.

Both versions work for all steps, except the subdomain version does not send the message finish pubsub.

You are a saint man. Thank you again. Sure this has probably been frustrating.
Here are the logs.

socketio.run

Server initialized for eventlet.
(5511) wsgi starting up on http://0.0.0.0:8070
(5511) accepted ('127.0.0.1', 50405)
be352aa3ba66460788a505982565e4e8: Sending packet OPEN data {'pingTimeout': 60000, 'sid': 'be352aa3ba66460788a505982565e4e8', 'upgrades': ['websocket'], 'pingInterval': 25000}
be352aa3ba66460788a505982565e4e8: Sending packet MESSAGE data 0
(5511) accepted ('127.0.0.1', 50407)
be352aa3ba66460788a505982565e4e8: Received request to upgrade to websocket
(5511) accepted ('127.0.0.1', 50408)
be352aa3ba66460788a505982565e4e8: Sending packet NOOP data None
be352aa3ba66460788a505982565e4e8: Received packet MESSAGE data 2["join account",{"account_id":17}]
(5511) accepted ('127.0.0.1', 50409)
received event "join account" from be352aa3ba66460788a505982565e4e8 [/]
be352aa3ba66460788a505982565e4e8 is entering room 17 [/]
be352aa3ba66460788a505982565e4e8: Upgrade to websocket successful
(5511) accepted ('127.0.0.1', 50412)
emitting event "start pubsub" to 17 [/]
fe32d4dee9ad4144ad348d2e788561d9: Sending packet MESSAGE data 2["start pubsub",{"msg":"Task Started"}]
(5511) accepted ('127.0.0.1', 50421)
emitting event "finish pubsub" to 17 [/]
be352aa3ba66460788a505982565e4e8: Sending packet MESSAGE data 2["finish pubsub",{"msg":"Task Finished"}]
be352aa3ba66460788a505982565e4e8: Received packet PING data None
be352aa3ba66460788a505982565e4e8: Sending packet PONG data None

devserver() with subdomain middleware

(5590) wsgi starting up on http://0.0.0.0:8070
(5590) accepted ('127.0.0.1', 50433)
Server initialized for eventlet.
fe32d4dee9ad4144ad348d2e788561d9: Sending packet OPEN data {'sid': 'fe32d4dee9ad4144ad348d2e788561d9', 'upgrades': ['websocket'], 'pingInterval': 25000, 'pingTimeout': 60000}
fe32d4dee9ad4144ad348d2e788561d9: Sending packet MESSAGE data 0
(5590) accepted ('127.0.0.1', 50435)
fe32d4dee9ad4144ad348d2e788561d9: Received request to upgrade to websocket
fe32d4dee9ad4144ad348d2e788561d9: Sending packet NOOP data None
(5590) accepted ('127.0.0.1', 50436)
(5590) accepted ('127.0.0.1', 50437)
fe32d4dee9ad4144ad348d2e788561d9: Received packet MESSAGE data 2["join account",{"account_id":17}]
received event "join account" from fe32d4dee9ad4144ad348d2e788561d9 [/]
fe32d4dee9ad4144ad348d2e788561d9 is entering room 17 [/]
fe32d4dee9ad4144ad348d2e788561d9: Upgrade to websocket successful
(5590) accepted ('127.0.0.1', 50440)
emitting event "start pubsub" to 17 [/]
fe32d4dee9ad4144ad348d2e788561d9: Sending packet MESSAGE data 2["start pubsub",{"msg":"Task Started"}]
(5590) accepted ('127.0.0.1', 50447)
Server initialized for eventlet.
emitting event "finish pubsub" to 17 [/]
fe32d4dee9ad4144ad348d2e788561d9: Received packet PING data None
fe32d4dee9ad4144ad348d2e788561d9: Sending packet PONG data None

[miguelgrinberg]

There's something strange going on with the subdomain test. Did you notice the server was restarted? This line specifically, near the end:

Server initialized for eventlet.

Does that happen every time you run the test? Any idea why the server is restarting?

[j-walker23]

Ahh, nice job. I was thinking that it was part of the flask_reloader but i tested without that and it is consistent.
Server initialized for eventlet. happens every time before it fails.

Would Server initialized for eventlet. be logged during a create_app/init_app? Wondering if its happening when a different app gets created in the SubdomainDispatcher

[miguelgrinberg]

That is logged when a server is created, so you have two servers. Are you using the correct one? Sounds like in the socketio.run() case you have only one, why do you have two in this situation?

[j-walker23]

I am thinking its a bug in my SubdomainDispatcher that i have probably had for months and didn't know it. And trying to use your tool is surfacing my true noob-ness. Thanks for going down this rabbit hole with me. I appreciate it.