Update python-socketio & python-engineio to latest #2591

phillxnet · 2023-06-09T12:18:28Z

Post #2567 in testing branch we are no longer limited by Py2.7. It is proposed that we update this primary dependency accodingly. From https://pypi.org/project/python-socketio/ we have the latest version still supporting our now prior Py3.6 base. [EDIT: we now use Py3.9 as of #2692],

N.B. we use a js library as our client so we must co-ordinate an update accordingly in our https://github.com/rockstor/rockstor-jslibs repository.

Linking to the last update of this client js library for context:
#1503

Note also the interdependency within our Python dependency on

python-engineio 2.3.2 Engine.IO server
└── six >=1.9.0
python-socketio 1.6.0 Socket.IO server
├── python-engineio >=1.0.0
│   └── six >=1.9.0 
└── six >=1.9.0

Also noting the larger context of replacement of this mechanism with the Django build-in channels 4.0:
https://github.com/django/channels/
https://channels.readthedocs.io/en/latest/releases/4.0.0.html
Which in turn requires at least Py3.7 [EDIT: we use Py3.9 now] and our next planned Django LTS version of 3.2, see Milestone: https://github.com/rockstor/rockstor-core/milestone/24

As the original Channels (Channels 1) underwent a major breaking change re-write in Channels 2:
https://channels.readthedocs.io/en/latest/releases/2.0.0.html
we want to avoid Channels 1 (which is compatible with our current Django) entirely.

The text was updated successfully, but these errors were encountered:

phillxnet · 2023-08-02T16:13:10Z

Linking to stepping stone issue #2625

FroggyFlox · 2023-08-19T19:08:33Z

Just playing with dependencies here:
Unpinning python-socketio in pyproject.toml, followed by poetry update:

buildvm155:/opt/rockstor # poetry update
Updating dependencies
Resolving dependencies... (6.9s)

Writing lock file

Package operations: 0 installs, 1 update, 0 removals

  • Updating python-socketio (1.6.0 -> 2.1.1)

That gets us to 2.1.1... maybe due to six?:
EDIT: scratch that... it was simply due to python-engineio.

phillxnet · 2023-10-12T16:16:16Z

We have a requirement to update python-engineio upon which python-socketio depends: however we have the following:

python-socketio 1.6.0 Socket.IO server
├── python-engineio >=1.0.0
│   └── six >=1.9.0 
...

So we may be able to independantly update the engineio part.

python-engineio

Version run-down:

Notable changes:

Release 4.7.1 - 2023-09-12

Replace gevent-websocket with simple-websocket when using gevent (commit)

Release 4.7.0 - 2023-09-03

Upgrade to pypy-3.9 in unit tests (commit)
** So this matches our current Py3.9] **

Release 4.6.0 - 2023-08-21

Better handling of Gunicorn threaded worker (commit)
** We plan to move to gtheread in gunicorn.

Release 4.5.0 - 2023-07-05

Remove old Python 2 syntax in super() calls (commit)
** A prior barrier for our updating this dependency. **

Release 4.4.0 - 2023-03-16

Add Python 3.11 to builds (commit)

...

Release 4.3.2 - 2022-04-24

Remove 3.6 and pypy-3.6 builds, add 3.10 and pypy-3.8 (commit)

...

Release 4.2.0 - 2021-05-15

Added Open Collective funding option (commit)

...
Release 4.1.0 - 2021-04-15

Remove obsolete 'mock' dependency Remove obsolete 'mock' dependency miguelgrinberg/python-engineio#218 (commit) (thanks Michał Górny!)

From the above we can now likely unpin python-engineio entirely.

phillxnet · 2023-10-12T16:47:45Z

If we remove our python-engineio pinning we get:

lbuildvm:/opt/rockstor # poetry update
Updating dependencies
Resolving dependencies... (8.1s)

Writing lock file

Package operations: 3 installs, 1 update, 0 removals

  • Installing h11 (0.14.0)
  • Installing wsproto (1.2.0)
  • Installing simple-websocket (1.0.0)
  • Updating python-engineio (2.3.2 -> 4.7.1)
lbuildvm:/opt/rockstor #

Viewing our Dashboard post this one change and we have 404 on all socket request: No header 'live' info and no widget info.

Response for each (Web-UI network tools: "Response" tab on detail of failed call: Response Payload:

"The client is using an unsupported version of the Socket.IO or Engine.IO protocols"

phillxnet · 2023-10-12T17:11:00Z

Downgrading to say 3.14.2 and we get:

  • Removing h11 (0.14.0)
  • Removing simple-websocket (1.0.0)
  • Removing wsproto (1.2.0)
  • Updating python-engineio (4.7.1 -> 3.14.2)

But see a different failure re:
"https://buildhostname.lan is not an accepted origin."

So we have quite a few changes along the way here.

3.13.2 gives bad request !! And likewise with our minimal fix of 3.9.0.
So it looks like we may just have to re-do this whole set:

jslib client side.
python-socketio.
python-engineio.

Hooverdan96 · 2023-10-12T17:31:55Z

just found a seemingly "working combination":

python-engineio==3.13.2
python-socketio==4.6.0

phillxnet · 2023-10-12T17:32:12Z

python-socketio

Last rockstor-core changes: Issue#1503 migrating gevent socketio to python socketio #1510
Last rockstor-jslibs changes: rockstor/rockstor-jslibs#9

phillxnet · 2023-10-12T17:39:32Z

@Hooverdan96 Re:

• Updating python-engineio (3.9.0 -> 3.13.2)
• Updating python-socketio (1.6.0 -> 4.6.0)

For me is still a bad request. Have your disabled network cache in Browser and reloaded dashboard.

Bit of a pain all this as we are to throw out for Django Channels 4 so don't really want to mess with it too much. But we have to update our python-engineio. Hence dabbling rather than looking at it properly.

Hooverdan96 · 2023-10-12T17:39:32Z

I know, this is in the context of Flask, but this might be useful too in the Django context:

https://flask-socketio.readthedocs.io/en/latest/intro.html#version-compatibility

And I would be remiss for not giving credit to this thread:

https://stackoverflow.com/questions/66069215/the-client-is-using-an-unsupported-version-of-the-socket-io-or-engine-io-protoco

Hooverdan96 · 2023-10-12T17:41:39Z

And yes, cleaning out all the caches on the browser made the above suggestion fail for me, too ...

FroggyFlox · 2023-10-12T17:43:37Z

Yes that's valid, @Hooverdan96 , it's the compatibility table from python-socketio directly:
https://python-socketio.readthedocs.io/en/latest/intro.html#version-compatibility

phillxnet · 2023-10-12T17:43:39Z

@Hooverdan96 Yes I saw this table. And was tempted to just up our entire stack end-to-end. Hence re-referencing the jslibs changes back-then. As I suspect we will run into protocol issues otherwise. And they seem to have thrown in some more orgin tests etc. Not motivated to re-write with Channels on this testing run so will likely just read-up more on what's going on here.

So maybe we just up everything end-to-end. But we don't have automated testing for jslibs modifications so a bit more tedious. I'll have a think and re-visit I think. Our versions are super old so maybe we just up everything and fix what's broken then.

phillxnet · 2023-10-13T09:36:06Z

Going all-in on newest available here will bring the following benefit:
miguelgrinberg/python-engineio@614f564
I.e. removal of monkey-patching requirement re

https://github.com/miguelgrinberg/python-engineio/blob/main/CHANGES.md#python-engineio-change-log
Release 4.7.1 - 2023-09-12

Replace gevent-websocket with simple-websocket when using gevent (commit)

phillxnet · 2023-10-13T16:53:42Z

With the latest debug version of our js client library (socket.io.js) in place, and the latest of both python-socketio and python-engineio installed we get, via Web dev tools in browser:

"Not an accepted origin." as Response.

Potential pointers: https://socket.io/docs/v4/handling-cors/

phillxnet · 2023-10-13T17:19:32Z

To review our server implementation we have:

Installation: https://python-socketio.readthedocs.io/en/latest/server.html

Deployment strategies: https://python-socketio.readthedocs.io/en/latest/server.html#deployment-strategies

N.B. our current deployment is via gevent :
https://python-socketio.readthedocs.io/en/latest/server.html#gevent

It is proposed that we make the minimum changes to re-enable functionality for the time being.

phillxnet · 2023-10-13T18:17:50Z

We have the initial call responding as expected:
Troubleshooting: https://socket.io/docs/v4/handling-cors/#troubleshooting

Python-socketio troubleshooting: https://python-socketio.readthedocs.io/en/latest/server.html#debugging-and-troubleshooting

curl --insecure "https://lbuildvm.lan/socket.io/?EIO=4&transport=polling"
0{"sid":"ArN1gylaXVu0BiA7AAMi","upgrades":["websocket"],"pingTimeout":20000,"pingInterval":25000}

but follow-ups all get "Not an accepted origin." as the response.

Useful info when right-clicking on the many 400 (BAD REQUEST) in browser dev tools and selecting:
Use as fetch in console:

Firefox can’t establish a connection to the server at wss://lbuildvm.lan/socket.io/?EIO=4&transport=websocket&sid=dGrWxXKDgnqyU9e3AAR2.

await fetch("https://lbuildvm.lan/socket.io/?EIO=4&transport=polling&t=OigJNV-&sid=tNq3sFkFzz8SrTdtAAR1", {
    "credentials": "include",
    "headers": {
        "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0",
        "Accept": "*/*",
        "Accept-Language": "en-GB,en;q=0.5",
        "Content-type": "text/plain;charset=UTF-8",
        "Sec-Fetch-Dest": "empty",
        "Sec-Fetch-Mode": "cors",
        "Sec-Fetch-Site": "same-origin",
        "Pragma": "no-cache",
        "Cache-Control": "no-cache"
    },
    "referrer": "https://lbuildvm.lan/",
    "body": "40/disk_widget,",
    "method": "POST",
    "mode": "cors"
});

Hooverdan96 · 2023-10-13T19:06:24Z

When looking at this issue:
miguelgrinberg/Flask-SocketIO#1501

though Rockstor is not using proxy_params (if I'm not mistaken), it seems the nginx.conf changes posted by the OP seem to work (for them, I have not tried this):

location /socket.io {
      proxy_pass http://localhost:5000/socket.io;
      proxy_http_version 1.1;
      proxy_buffering off;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "Upgrade";
      proxy_set_header Host $http_host;
      proxy_set_header X-Forwarded-Host $http_host;
      proxy_set_header X-Forwarded-Proto $scheme;
    }

compared to what is in Rockstor's nginx.conf:

	location /socket.io {
		proxy_pass http://127.0.0.1:8001;
		proxy_redirect off;
		proxy_http_version 1.1;
		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection "upgrade";
        }

I don't want to slow you down, if that's helpful, great, if not, just carry on, you will figure it out :)

phillxnet · 2023-10-13T19:37:12Z

@Hooverdan96 Thanks, I'll take a look on my next session. I had wonered about the proxy but not taken a look yet.

proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $http_host;

do look interesting, given the tighter controls on these that look to have been introduced in the newer versions of the client and server of socketio. All quite frustrating but getting there I think. However I did experiment with some server options that should have allowed any client access and still got the refusals.

phillxnet · 2023-10-14T11:12:31Z

Before proxy changes:

tail -f /opt/rockstor/var/log/supervisord_data-collector_stderr.log
...
Server initialized for gevent.
g_ItEsKnppvri72UAAAA: Sending packet OPEN data {'sid': 'g_ItEsKnppvri72UAAAA', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
https://lbuildvm.lan is not an accepted origin. (further occurrences of this error will be logged with level INFO)
https://lbuildvm.lan is not an accepted origin.
https://lbuildvm.lan is not an accepted origin.
YwMwKa27ufI3Sr8HAAAB: Sending packet OPEN data {'sid': 'YwMwKa27ufI3Sr8HAAAB', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
NBD3SSfs_v3WIYN3AAAC: Sending packet OPEN data {'sid': 'NBD3SSfs_v3WIYN3AAAC', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
49Z2CqhQegNfv-FAAAAD: Sending packet OPEN data {'sid': '49Z2CqhQegNfv-FAAAAD', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
0OVsvIgWmhBnuS1bAAAE: Sending packet OPEN data {'sid': '0OVsvIgWmhBnuS1bAAAE', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
https://lbuildvm.lan is not an accepted origin.
https://lbuildvm.lan is not an accepted origin.
https://lbuildvm.lan is not an accepted origin.
https://lbuildvm.lan is not an accepted origin.
https://lbuildvm.lan is not an accepted origin.

Post suggested changes and we still have the same behaviour:

Server initialized for gevent.
I45XB6Lo0aB6h5LKAAAA: Sending packet OPEN data {'sid': 'I45XB6Lo0aB6h5LKAAAA', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
https://lbuildvm.lan is not an accepted origin. (further occurrences of this error will be logged with level INFO)
https://lbuildvm.lan is not an accepted origin.
https://lbuildvm.lan is not an accepted origin.
mG0OkSsJrXFM0DIdAAAB: Sending packet OPEN data {'sid': 'mG0OkSsJrXFM0DIdAAAB', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
bJ3XRhJQlEIsuwTeAAAC: Sending packet OPEN data {'sid': 'bJ3XRhJQlEIsuwTeAAAC', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}

However we do have some communication here, and the initial response from the server giving the 'sid' is in response to a protocol 4 (?EIO=4) request. So we are stuck on establishing the websocket I think:

I.e. the initial GET works
GET > wss://lbuildvm.lan/socket.io/?EIO=4&transport=websocket&sid=ckj3sQjF9IWGCjB2AAA7

as intended to initiate a protocol switch to websocket and gets status 101
But there-after our

POST request: > https://lbuildvm.lan/socket.io/?EIO=4&transport=polling&t=OijquzC&sid=ckj3sQjF9IWGCjB2AAA7
Get the Bad request 400: "Not an accepted origin."

We also get successful polling events by the look of it.

So the key here is:
Firefox can’t establish a connection to the server at wss://lbuildvm.lan/socket.io/?EIO=4&transport=websocket&sid=uz6Ba3kthZto4IJ5AAEW.

I'll review our dependencies on the browser side here as this looks like we may be missing something that has emerged in the last 7 years !!

phillxnet · 2023-10-14T11:17:45Z

cors_allowed_origins='*'

Is already in play when the dev setup initiates our Python-serverio in data_collector.py.

    sio_server = socketio.Server(async_mode="gevent", cors_allowed_origins=['*'], logger=True, engineio_logger=True)

phillxnet · 2023-10-14T11:22:00Z

OK, here-it-comes:

sio_server = socketio.Server(async_mode="gevent", cors_allowed_origins='*', logger=True, engineio_logger=True)

Works much better !!!! I'll pretend this didn't happen and stagger on with what is now a tone of stuff working again :).

Raise pinned versions to latest available: ## Includes: - Revised nginx reverse proxy for data_collector app. - Replace gevent-websocket with simple-websocket. - Replace gevent for regular Python threads in socketio.Server.

…-python-engineio-to-latest Update python-socketio & python-engineio to latest #2591

phillxnet · 2023-10-15T11:58:40Z

Closing as:
Fixed by #2701

phillxnet self-assigned this Oct 12, 2023

phillxnet mentioned this issue Oct 13, 2023

Update socket.io client lib rockstor/rockstor-jslibs#36

Closed

phillxnet changed the title ~~Py3.6 update python-socketio~~ Update python-socketio Oct 13, 2023

phillxnet changed the title ~~Update python-socketio~~ Update python-socketio & python-engineio Oct 14, 2023

phillxnet changed the title ~~Update python-socketio & python-engineio~~ Update python-socketio & python-engineio to latest Oct 14, 2023

phillxnet mentioned this issue Oct 14, 2023

Update python-socketio & python-engineio to latest #2591 #2701

Merged

phillxnet added this to the Py3.9 milestone Oct 15, 2023

phillxnet added a commit that referenced this issue Oct 15, 2023

Merge pull request #2701 from phillxnet/2591-Update-python-socketio-&…

235628f

…-python-engineio-to-latest Update python-socketio & python-engineio to latest #2591

phillxnet closed this as completed Oct 15, 2023

phillxnet mentioned this issue Nov 11, 2023

Django Server Sent Events to provide live data requirements #2739

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update python-socketio & python-engineio to latest #2591

Update python-socketio & python-engineio to latest #2591

phillxnet commented Jun 9, 2023 •

edited

Loading

phillxnet commented Aug 2, 2023

FroggyFlox commented Aug 19, 2023 •

edited

Loading

phillxnet commented Oct 12, 2023 •

edited

Loading

phillxnet commented Oct 12, 2023 •

edited

Loading

phillxnet commented Oct 12, 2023 •

edited

Loading

Hooverdan96 commented Oct 12, 2023

phillxnet commented Oct 12, 2023

phillxnet commented Oct 12, 2023

Hooverdan96 commented Oct 12, 2023

Hooverdan96 commented Oct 12, 2023

FroggyFlox commented Oct 12, 2023

phillxnet commented Oct 12, 2023

phillxnet commented Oct 13, 2023

phillxnet commented Oct 13, 2023

phillxnet commented Oct 13, 2023 •

edited

Loading

phillxnet commented Oct 13, 2023 •

edited

Loading

Hooverdan96 commented Oct 13, 2023 •

edited

Loading

phillxnet commented Oct 13, 2023 •

edited

Loading

phillxnet commented Oct 14, 2023

phillxnet commented Oct 14, 2023

phillxnet commented Oct 14, 2023

phillxnet commented Oct 15, 2023

Update python-socketio & python-engineio to latest #2591

Update python-socketio & python-engineio to latest #2591

Comments

phillxnet commented Jun 9, 2023 • edited Loading

phillxnet commented Aug 2, 2023

FroggyFlox commented Aug 19, 2023 • edited Loading

phillxnet commented Oct 12, 2023 • edited Loading

python-engineio

Version run-down:

phillxnet commented Oct 12, 2023 • edited Loading

phillxnet commented Oct 12, 2023 • edited Loading

Hooverdan96 commented Oct 12, 2023

phillxnet commented Oct 12, 2023

python-socketio

phillxnet commented Oct 12, 2023

Hooverdan96 commented Oct 12, 2023

Hooverdan96 commented Oct 12, 2023

FroggyFlox commented Oct 12, 2023

phillxnet commented Oct 12, 2023

phillxnet commented Oct 13, 2023

phillxnet commented Oct 13, 2023

phillxnet commented Oct 13, 2023 • edited Loading

phillxnet commented Oct 13, 2023 • edited Loading

Hooverdan96 commented Oct 13, 2023 • edited Loading

phillxnet commented Oct 13, 2023 • edited Loading

phillxnet commented Oct 14, 2023

phillxnet commented Oct 14, 2023

phillxnet commented Oct 14, 2023

phillxnet commented Oct 15, 2023

phillxnet commented Jun 9, 2023 •

edited

Loading

FroggyFlox commented Aug 19, 2023 •

edited

Loading

phillxnet commented Oct 12, 2023 •

edited

Loading

phillxnet commented Oct 12, 2023 •

edited

Loading

phillxnet commented Oct 12, 2023 •

edited

Loading

phillxnet commented Oct 13, 2023 •

edited

Loading

phillxnet commented Oct 13, 2023 •

edited

Loading

Hooverdan96 commented Oct 13, 2023 •

edited

Loading

phillxnet commented Oct 13, 2023 •

edited

Loading