Skip to content

Commit

Permalink
fixes #28: example app did not start background thread when running u…
Browse files Browse the repository at this point in the history 
…nder a production server
  • Loading branch information
@miguelgrinberg
miguelgrinberg committed May 21, 2014
1 parent e86f31d commit 6c995a5
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion example/app.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
app.debug = True
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)
thread = None


def background_thread():
Expand All @@ -25,6 +26,10 @@ def background_thread():

@app.route('/')
def index():
global thread
if thread is None:
thread = Thread(target=background_thread)
thread.start()
return render_template('index.html')


Expand Down Expand Up @@ -80,5 +85,4 @@ def test_disconnect():


if __name__ == '__main__':
Thread(target=background_thread).start()
socketio.run(app)

10 comments on commit 6c995a5

@espretto
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi miguel,
i'm trying to setup your flask-socket.io with sqlalchemy greened with psyopg2 and embersockets on the clientside.
i was wondering since you're using gevent and call monkey.patch_all() if the above equals

# [...]
def background_thread():
    # [...]

from gevent import Greenlet
@app.route('/')
def index():
    Greenlet.spawn(background_thread)
    return render_template('index.html')
# [...]

@miguelgrinberg
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, you don't have to use the monkey patched threading, it's all the same in the end.

@boots-shirts
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @miguelgrinberg ,
I still have a question about having a background thread running in a flask app.
In my case, the background thread does a job of updating a list, which is a global variable shared by a socketio function. However, there seems to be a coordinating problem, for which I've tried to use threading.Condition(), only getting nowhere.

my code (following your example):

@app.route('/home')
def home():
    global thread
    if thread==None:
        thread=background_thread
        thread = threading.Timer(time, background_function, ())
        thread.start()
    return render_template('home.html')

background_thread=threading.Thread()
condition=threading.Condition()
my_list=[]

def background_function():
     condition.acquire()
     global my_list
     #get new_data#
     my_list.extend(new_data)
     socketio.emit('my response',
                      {'data': 'list updated'})
     condition.notify()
     condition.release()

@socketio.on('my event', namespace)
def my_event_handler:
     condition.acquire()
     global my_list
     if len(my_list)==0:
            while True:
                 emit('response',{'data':'Loading,please wait'})
                 condition.wait()
    else:
         while True:
                  do something
          condition.release()

Now , I get stuck forever with the message 'Loading,please wait' on the front. It just seems the background function doesn't run at all, because I never get the 'list updated' message.
Really grateful if you could enlighten me on this issue.

@miguelgrinberg
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@boots-shirts what is the behavior when you don't use the condition? If you are appending to the list on the bg thread and only checking the list length in the socket handler I think you do not need to worry about bad interactions.

@boots-shirts
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@miguelgrinberg Thanks for your reply. What I want to do is that while bg thread keeps updating a list, the socket handler get data from that list, then do something with it.
I've changed to use Queue.Queue instead of a list to avoid using threading.Condition, and wrap the main part of my event handler into another thread as well. However the code below still doesn't work, because I never get message from bg.

myqueue=Queue.Queue(0)

@app.route('/home')
def home():
    t1= threading.Timer(time, bg)
    t1.start()
    t1.daemon=True                                #same with or without this line
    return render_template('home.html')

def bg():
    data=get_data()
    myqueue.put(data)
    socketio.emit('my response',
                      {'data': 'data updated'},
                      namespace)

@socketio.on('my event', namespace)              
def my_event_handler():                                                                                  
    while True:
        d=myqueue.get()
        myqueue.task_done()                            # same with or without this line
        t2=threading.Thread(consume_data,(d))
        t2.start()
        time.sleep(4)

@miguelgrinberg
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first thing I notice in your example is that your bg thread does not have a loop, so it will run once and then end. On the other side, your socketio handler does have a loop and it shouldn't, this function will be called each time the client sends something, and it is supposed to return after the event is processed. I do not think your problem is with the list or the queue.

@boots-shirts
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@miguelgrinberg , thanks a lot for the feedback. After examining everything, I have finally narrowed down my problem to be the bg task--get data from a third party--'working outside of request context, RuntimeError'.

I made a mistake earlier, moving the thread.start() statement from @app.route('/') to @app.route('/home') like shown in the code above, trying to avoid this RuntimeError, and it turned out the reason why that error didn't occur again, was that the background didn't even get started.

The almost working code:
@app.route('/')
def index():
t1=bg() # I create a bg class
t1.daemon=True
t1.start()
return render_template('home.html')

So the final question is, how can I sort out the request context problem? The flask documentation on this is beyond me. Really appreciate your help here!

@miguelgrinberg
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's hard to tell without seeing the code. Your background thread apparently relies on one of the Flask context variables, which do not exist in your thread. Read the Application and Request context documentation for Flask, that may help.

@boots-shirts
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @miguelgrinberg, I've read the documents and made a few changes, but still having no luck. Would you please take a look at my code here: https://github.com/boots-shirts/Flask-SocketIO-threading? A deadline is looming, really appreciate your help! THANKS!

@miguelgrinberg
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@boots-shirts, my recommendation is that you try to make the different parts of your application work separately first, then once you have working code for each part you attempt the integration. Just from a quick review I see several problems, some really basic, like you calling app.text_request_context instead of app.test_request_context. And really, test_request_context is not supposed to be used in a production app anyway. Another issue is that every time someone connects to /home you will be spawing another bg thread, while I think you want just one bg thread for the whole process.

Please sign in to comment.