AttributeError: 'Response' object has no attribute 'status_code'

[bclark8923]

I continually get the error listed below. Nothing seems to break but I'd like to clean it up. Any ideas what causes this?

2016-02-03 21:26:02 [33724] [ERROR] Error handling request
Traceback (most recent call last):
  File "/Users/brianclark/Desktop/Projects/HDP/application/api/venv/lib/python2.7/site-packages/gunicorn/workers/async.py", line 45, in handle
    self.handle_request(listener, req, client, addr)
  File "/Users/brianclark/Desktop/Projects/HDP/application/api/venv/lib/python2.7/site-packages/gunicorn/workers/async.py", line 102, in handle_request
    resp.close()
  File "/Users/brianclark/Desktop/Projects/HDP/application/api/venv/lib/python2.7/site-packages/gunicorn/http/wsgi.py", line 369, in close
    self.send_headers()
  File "/Users/brianclark/Desktop/Projects/HDP/application/api/venv/lib/python2.7/site-packages/gunicorn/http/wsgi.py", line 284, in send_headers
    tosend = self.default_headers()
  File "/Users/brianclark/Desktop/Projects/HDP/application/api/venv/lib/python2.7/site-packages/gunicorn/http/wsgi.py", line 265, in default_headers
    elif self.should_close():
  File "/Users/brianclark/Desktop/Projects/HDP/application/api/venv/lib/python2.7/site-packages/gunicorn/http/wsgi.py", line 198, in should_close
    if self.status_code < 200 or self.status_code in (204, 304):
AttributeError: 'Response' object has no attribute 'status_code'

[miguelgrinberg]

How are you starting the server? If you are using gunicorn, please make sure you run with just one worker (--workers 1).

[bclark8923]

Here's my startup command

gunicorn --worker-class eventlet -w 1 server:app --bind="127.0.0.1:5000"

[miguelgrinberg]

Thanks. I'd seen this before when I was investigating a different gunicorn issue. I actually reported it to the gunicorn guys: benoitc/gunicorn#1147.

This error is benign I think, in the sense that it does not prevent the socket communication from happening. It occurs because gunicorn does not recognize the request that is ending is a WebSocket one, and tries to apply the HTTP logic to it.

Sounds like I need to write a separate bug for it.

[bclark8923]

Oh! I was on gunicorn 18, i see in 19 you have to sepcify the -w 1, I switched to 19.4.5 to see if it keeps happening

[miguelgrinberg]

Yeah, this error is in both R18 and R19.

[bclark8923]

Ah you're right, I switched and it's still happening.

[bclark8923]

@miguelgrinberg did you open up an issue w/ Gunicorn? Was hoping to subscribe to that as well

[miguelgrinberg]

I did report it, but as something I discovered while investigating another issue. benoitc/gunicorn#1147 (comment). I believe there is no open issue specific to this problem.

[bclark8923]

Submitted one myself, hopefully can get this resolved

benoitc/gunicorn#1210

Thanks,
Brian Clark
(248) 990 5616
www.hdphealth.com

Get to know me on Facebook https://facebook.com/bclark8923 & Twitter
https://twitter.com/blaurenceclark!

On Sun, Feb 14, 2016 at 11:16 PM, Miguel Grinberg [email protected]
wrote:

I did report it, but as something I discovered while investigating another
issue. benoitc/gunicorn#1147 (comment)
benoitc/gunicorn#1147 (comment).
I believe there is no open issue specific to this problem.

—
Reply to this email directly or view it on GitHub
#223 (comment)
.

[jpmolla]

I noticed the same problem with the same gunicorn config but this error appears only when the client (web navigator or cordova application in my case) is closed suddenly (so the pipe connection is broken).

As an example, after the websocket connection my application loops on a request and everything is fine for hours. But as soon as the client is closed abruptly or the page is reloaded, the error appears after 45 to 60 seconds.

It looks like an attempt for a reconnection or data exchange from the server to the client and after a timeout. But as the pipe is broken gunicorn is not able to send the request.

Does it makes sens ? Can't this come from the Flask-SocketIO module which could try to send something to the client while ignoring the broken hearbeat ?

[sykp241095]

Same issue, mark.

[miguelgrinberg]

This is a gunicorn issue. Closing.

[erm0l0v]

I have the same problem.
Environment:
AWS server (EC2 AWS Linux)
Docker 1.9
Python 2.7
gunicorn 18.0
Flask 0.10.1
flask-socketio 2.1
eventlet 0.18.3
werkzeug 0.11.3

I start only one socket.io worker via command:
/usr/local/bin/gunicorn socket_io_app:app -w 1 -b 127.0.0.1:5001 --worker-class eventlet --log-level=debug --chdir=/dus

And I always get error:

Traceback (most recent call last):
  File "gunicorn/workers/async.py", line 45, in handle
                          self.handle_request(listener, req, client, addr)
  File "gunicorn/workers/async.py", line 102, in handle_request
                  resp.close()
  File "gunicorn/http/wsgi.py", line 369, in close
              self.send_headers()
  File "gunicorn/http/wsgi.py", line 284, in send_headers
          tosend = self.default_headers()
  File "gunicorn/http/wsgi.py", line 265, in default_headers
          elif self.should_close():
  File "gunicorn/http/wsgi.py", line 198, in should_close
          if self.status_code < 200 or self.status_code in (204, 304):
AttributeError'Response' object has no attribute 'status_code'

What I missing?

[miguelgrinberg]

@erm0l0v Does it prevent the server from working? When I saw this (which is a bug in gunicorn, see benoitc/gunicorn#1210), it was benign, in the sense that it happened when a socket connection was ending, and did not stop the server from accepting new connections.

[erm0l0v]

Yes server works normally. I just get a mount of this kind errors. Should I ignore them?

[miguelgrinberg]

@erm0l0v I think there is no other choice at this time than to ignore them. As I said, there is an open issue for this problem with gunicorn, so hopefully it'll be soon fixed.

[erm0l0v]

Ok. Thank You!!

[bclark8923]

@miguelgrinberg There's an update on the issue from gunicorn

benoitc/gunicorn#1210

[lequangdzung]

@miguelgrinberg without this fix of gunicorn, do we have any way to handle this error on flask-socketio to make it work right? I'm running on production so don't want to lost data because of this issue.

[miguelgrinberg]

My understanding is that this error is cosmetic, there is no data loss, and the service does not go down because of it. Have you experienced any data loss from this error?

[lequangdzung]

@miguelgrinberg I got some error like this:

127.0.0.1 - - [26/May/2016:22:57:43 +0000] "GET /socket.io/?EIO=3&transport=polling&t=1464303409105-2&sid=f2cf23926430474ca17ac2b9d3f57da6 HTTP/1.1" 400 11 "http://localhost/client/139" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36"

Do you think socketio still works as well in this case without refresh browser?

[miguelgrinberg]

The client application has retry logic in it. If any errors occur on the server, the connection will drop and the client will establish a new one right away.

A 400 error indicates a potential issue on the server, but without detailed logs I can't really tell you what happened. You should run with logging enabled (add logger=True, engineio_logger=True to your SocketIO constructor call) and maybe that will provide some additional info when this happens again. You should also enable logging on the browser, as indicated in the Socket.IO client documentation.

[w-A-L-L-e]

Trying out flack as we need a similar websocket python solution that is scalable.
However it's not easy to get more workers. This works (with the error above but messages are correctly sent/received on ws): gunicorn -b 0.0.0.0:5000 -k eventlet -w 1 flack.wsgi
-> problems with it I can't do -w 2 or -w 8 to scale up and it resorts to long polling whereas doing this (for develpment not for production: python manage.py runserver does do a proper websocket and gets no errors. Any suggestions on how to scale this for production (get multiple workers with websockets running?). Right now I only see using nginx to sticky proxy to different servers as a possibility. For celery I can indeed use 4 workers, again it's not trivial to see how to make it 8 or 16 workers from commandline in the flack example. Thanks in advance should you take the time to answer...

[miguelgrinberg]

@w-A-L-L-e You need to read the documentation to learn how to run multiple workers. Gunicorn's -w flag needs to always be set to 1. Anything higher than 1 will give you problems. Instead, you need to start multiple gunicorn processes, all with -w 1, and put a load balancer in front, such as nginx. You may also want to watch my PyCon 2016 session Flask At Scale, where I explained how to scale this Flack application in more detail.

[w-A-L-L-e]

Thanks for quick reply. That's strange so you can only just use 1 worker -> why even bother with gunicorn ;) Ok so we can do nginx proxy_pass to multiple processes but I thought the whole reason you use something like gunicorn is to keep those workers properly running and kill/restart any that go out of line memory or cpu wise. Anyway it's how unicorn or puma is used in ruby on rails land where you use this to scale the frontend processes (or threads with puma) and afterwards for multiple servers go the nginx proxy_pass route...

[miguelgrinberg]

@w-A-L-L-e As I said, this is explained in the documentation...

Gunicorn does not support sticky sessions, and that is required for Socket.IO, which is a stateful protocol. The day gunicorn adds sticky sessions (similar to nginx's ip_hash) you will be able to use multiple workers the way you describe.

[w-A-L-L-e]

Ok, thanks for the info(back from holiday). Anyway the reason I tried gunicorn multiple workers was because when I configured to do the nginx sticky session proxy_pass way I noticed the client (browser inspect) shows long polling instead of true websocket. When it's running in develop mode using python manage.py runserver it does use websocket without long poll. Is this because I'm using python 2.7.x or does that also happen on 3.x or maybe it's due to that error we get:
When I run:
gunicorn -b 0.0.0.0:5000 -k eventlet -w 1 flack.wsgi

[96865] [ERROR] Error handling request /socket.io/?EIO=3&transport=websocket&sid=57a0b413bca64a1a903bcc953527ecce
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/gunicorn/workers/async.py", line 52, in handle
self.handle_request(listener_name, req, client, addr)
File "/usr/local/lib/python2.7/site-packages/gunicorn/workers/async.py", line 114, in handle_request
resp.close()
File "/usr/local/lib/python2.7/site-packages/gunicorn/http/wsgi.py", line 403, in close
self.send_headers()
File "/usr/local/lib/python2.7/site-packages/gunicorn/http/wsgi.py", line 319, in send_headers
tosend = self.default_headers()
File "/usr/local/lib/python2.7/site-packages/gunicorn/http/wsgi.py", line 300, in default_headers
elif self.should_close():
File "/usr/local/lib/python2.7/site-packages/gunicorn/http/wsgi.py", line 233, in should_close
if self.status_code < 200 or self.status_code in (204, 304):
AttributeError: 'Response' object has no attribute 'status_code'

Basically I see a 101 response (switching protocols) when it tries websocket when this happens then it goes to long polling and gets an ok status. When running in dev mode it does not do the protocol switch.

In other words running like this gives me true websockets (but is most likely not optimal for production):
python manage.py runserver

[w-A-L-L-e]

Anyways above question is not so much important as long as it scales and works with the websocket in long polling mode. A different question I'm extending this project and first thing was to split up the models.py in seperate files in a models dir. The problem is in init.py I need to do book keeping for every file added:

#from .models import * # noqa
#here we now need to import a line per model ugh... solve this later
from .models.user import User
from .models.user_notification import UserNotification
from .models.conversation import Conversation
from .models.conversation_contact import ConversationContact
from .models.message import Message
from .models.notification import Notification
from .models.notification_type import NotificationType

Is there a better way to do that?

Next up I tried to add migrations using flask_migrate package (much like in your tutorial here http://blog.miguelgrinberg.com/post/flask-migrate-alembic-database-migration-wrapper-for-flask/page/3).

So I tried doing this in the manage.py script:
import eventlet
eventlet.monkey_patch()

from flask_script import Manager, Command, Server as _Server, Option
from notifications_app import create_app, db, socketio
from flask_migrate import Migrate, MigrateCommand

app = create_app
migrate = Migrate(app,db)
manager = Manager(app)

manager.add_command( 'db', MigrateCommand )

But this does not work when you run:
python manage.py db init

you get following errors:
Traceback (most recent call last):
File "manage.py", line 120, in
manager.run()
File "/usr/local/lib/python2.7/site-packages/flask_script/init.py", line 412, in run
result = self.handle(sys.argv[0], sys.argv[1:])
File "/usr/local/lib/python2.7/site-packages/flask_script/init.py", line 383, in handle
res = handle(_args, *_config)
File "/usr/local/lib/python2.7/site-packages/flask_script/commands.py", line 216, in call
return self.run(_args, *_kwargs)
File "/usr/local/lib/python2.7/site-packages/flask_migrate/init.py", line 89, in init
directory = current_app.extensions['migrate'].directory
KeyError: 'migrate'

[w-A-L-L-e]

Ok I solved the migration issue like so. Seems to work ok:

working migrations in manage.py :

#!/usr/bin/env python
import os
import subprocess
import sys

import eventlet
eventlet.monkey_patch()

from flask_script import Manager, Command, Server as _Server, Option
from notifications_app import create_app, db, socketio
from flask_migrate import Migrate, MigrateCommand

app = create_app()
manager = Manager(app)

migrate = Migrate(app,db)
manager.add_command( 'db', MigrateCommand )

Then running
python manage.py db init
creates wanted dir and
python manage.py db migrate
creates the proper migration with all table info.
python manage.py db upgrade then migrates db correctly to current version

Now only to solve this import statements in init.py :
#Import models so that they are registered with SQLAlchemy
#when all models were in one file we could do:
#from .models import * # noqa
#here we now need to import a line per model ugh... solve this later
from .models.user import User
from .models.user_notification import UserNotification
from .models.conversation import Conversation
from .models.conversation_contact import ConversationContact
from .models.message import Message
from .models.notification import Notification
from .models.notification_type import NotificationType

Any tips on simplifying the from .models. lines into 1 line (like an import * for the whole dir?).

[miguelgrinberg]

Not sure I understand the problem. If you want to have all your models in the same module just use a single models.py, or if you want to use a package, just import all the models in the package __init__.py file so that from the outside you can import all the models from the same source.

[jackmullen2]

Hi Miquel :

I am attempting to start a general non client specific background process that updates song titles on website -- it should broadcast and ideally start before the first webpage is loaded because it monitors song titles and databases them -- but I am using the index page to start the process using the example you wrote here https://github.com/miguelgrinberg/Flask-SocketIO/blob/master/example/app.py .. I am using 11.1 and have not installed gevent or eventlet.. When I try thread = socketio.start_background_task(target=getSongTitles) I get the error message : "thread = socketio.start_background_task(target=getSongTitles)
AttributeError: 'SocketIO' object has no attribute 'start_background_task'" ... Can you think of what I might be doing wrong? thank you ..

[miguelgrinberg]

@jackmullen2 what version of Flask-SocketIO do you have? That method was added in version 2.5.

[jackmullen2]

I just used pip to update .. it went to 11. 1 .. I guess that is 0.11.1 ? i am not sure how to get the latest version .. flask.version
'0.11.1'

[jackmullen2]

wow .. sorry .. I was checking flask version .. I upgraded to version 2.2 for flask-socketio .. the results produced this message "
failed to get json data from stream server Working outside of request context." I saw this message when I tried to start a conventional thread with the def index() method passing the socketio context did not produce socket.emit()s .. I am sorry I was blinded by thinking flask version was flask-socketio version .. so I have not invested enough time into this new issue ..

[qwexvf]

i can't fix it. please help.

[w-A-L-L-e]

I'm using Flask-SocketIO (2.4) and it seems pretty stable without issues for me...
@miguelgrinberg why is it needed to fix an outdated 2.2 version?
I did however add some cleanup for stale sockets by keeping track of user_sockets in a seperate db etc by using a before_first_request

@main.before_app_first_request
def before_first_request():
    """Start a background thread that looks for users that leave."""
    def find_offline_users(app):
        with app.app_context():
            while True:
                user_sockets = UserSocket.find_closed_sockets()

...

  if not current_app.config['TESTING']:
        thread = threading.Thread(target=find_offline_users,
                                  args=(current_app._get_current_object(),))
        thread.start()

This works fine and has right app context...
I initially tried before_app_start or something but that did not work either due to same context issue. Workaround is doing a curl request right after deploy so that the before_first_request functions similarly...

[qwexvf]

@w-A-L-L-e hi! could talk about more about your code?

[w-A-L-L-e]

Thought it would help basically I had same context/thread issue in beginning so I posted it here. By using the before first request the part with app.app_context() seemed to work correctly. We also kept track of the sockets in database (postgres) that allowed to do cleanup/offline detection a bit better for our use case... Only thing to mention is we have jenkins build server always call a /stats route with curl after deploy so we know this cleanup thread is always running...

Ow and that import stuff from way back is not really an issue anymore (or rather I don't need to change the imports much so I can live with it...)

Anyway hope it helped. Keep up the good work.

[sushanth-krishnaswamy]

@miguelgrinberg : Still getting the same error with gunicorn and two worker threads. Would you suggest using uWSGI or any other wsgi supported servers other than gunicorn?

[miguelgrinberg]

You can't use multiple workers with Gunicorn when you have the Flask-SocketIO extension. Try one worker.

[sushanth-krishnaswamy]

@miguelgrinberg : Thank you! that worked

[luqijun]

I solved my problem by install flask-sockets and flask-sock :

pip install flask-sockets flask-sock