Websocket proxy and consoles next steps

[martinpovolny]

This issues is a place to collect inputs and designs for the next steps of the websocket proxy that we use for the HTML5 consoles and may use in the future for other console types, such as HTML5 ssh/terminal client or RDP console.

Ruby version
There's a ruby version of the python proxy that we have, but it lack's some feature of the python counterpart most notably SSL support:
https://github.com/kanaka/websockify/wiki/Feature_Matrix

Single proxy, multiple clients
There's a way to use one instance of the proxy for several clients:

--target-config=FILE  Configuration file containing valid targets in the
                        form 'token: host:port' or, alternatively, a directory
                        containing configuration files of this form

This is a mode of operations that is heavily pushed by the RHEVM people. The main argument being that this way there are 2 secrets that have to be presented by the client -- the token for this file verified by the proxy and then the one-time password that is verified by the RHEVM or VMware SPICE or VNC backend.

From our point of view, it would be convenient to open just one port for the proxy not a port for each client that connects.

Client address checking
It would be convenient if the file above had the structure token: host:port:client_addr meaning that the proxy would also verify the source address from where the browser/client is supposed to connect. When we ask for the terminal ticket, we know the address of the client so we could use it.

Actually it would make more sense to have the above in a database table, rather than a file/directory.

Management of connections

We need to implement management of entries in this file / database table to deal with closed or timed-out connections. We could also warn the user in the UI if he tries to connect to a VM where there's a recent connection from another client.

Websocket under apache
Another nice-to have feature would be the ability to pass an open connection from Apache to the websocket proxy in the appliance. That would allow us to have no extra port open for the websocket proxy and would allow the consoles to work seamlessly with self-signed certificates provided in the appliance.

• http://httpd.apache.org/docs/2.4/mod/mod_proxy_wstunnel.html
• https://github.com/disconnect/apache-websocket

SPICE (and/or VNC) w/o HTML5
As suggested here: http://talk.manageiq.org/t/spice-without-html5/660/ we could provide better user experience by supporting existing SPICE and VPN clients that are more capable than the HTML5 variants. Especially if such tools can be launched from a web browser.

Python version as a package

If we decide to keep the python version, we would like to distribute it in a standard way and not as part of our project. Probably an "egg"? RPM for the productized version?

Support for web balancers

Web balancers that I know about do not support any way of handling the connections that we do to the ports serving our ws:// wss://. To support any web balancers at all we need to implement "Websocket under apache" as suggested above.

If we implement the above then only the web balancers that support session affinity would work as the proxy is launched on the appliance that serves the request for the console UI.

To support balancers that do not support session affinity, we would need to be able to open the SPICE/VNC websocket proxy on any of the UI workers. To do that we would need to put the connection information into a database as suggested above under "Management of connections"

User feedback

• It would be great if the console would "calculate the console down" to the local resolution so that all of the screen is accesible all the time. If that helps, the browser zoom function could be used for that, since mouse positining even works well if you zoom the VNC console in our out using the browser's zooming.
• Otherwise, the browser windows should get scroll bars to move to the parts that are outside the window.
• A full screen mode would be great, removing the browser decoration and allowing full screen access to the remote machine, similar as in virt-manager, for example.
• Some more buttons for keyboard combinations could be useful, like switching Ctrl-Alt-F1..n to switch Linux consoles or an Escape button or to enter a VMware BIOS would be useful.

[Fryguy]

One thing I'd like is to not have the websockify code directly in our source...if it's Ruby, we can use a gem, but even with the python one, can we use an egg?

[martinpovolny]

@Fryguy : according to the feature matrix above, the Ruby version is quite limited. But yes, based on features that we will want to support in the next release, we can either switch to the Ruby version or pack the Python version as an egg.

[martinpovolny]

Consoles in the SSUI

• move launch of proxy to the backend
• create a model to track display tickets
• add an API action for the display tickets
• use single proxy process (python) and a single port for all clients
• implement client side in SSUI
• (optional) switch to port 443 (use some apache module)

Considerations

• just add a FW rule and show proxy params for use by a desktop client?

[skateman]

Just for an experiment, I checked out if we have any kind of apache modules for websockets available in our appliances.

httpd -M | grep ws
 proxy_wstunnel_module (shared)

yay 😄

[martinpovolny]

Cool, @skateman.

The next step is to figure out how to manage the configuration of the web socket proxy process once it's running as a daemon.

You need to

• add items to the list (from Ruby side),
• remove entries that are closed (from Python side)

If you do it in memcache, then you should consider the load-balancing option. You'd need session affinity on the load balancer probably.

[skateman]

So i took over this task and I experimented a bit with the faye-websocket gem. Currently I have a proxy which supports multiple clients connecting to multiple endpoints. It listens on just one TCP port and the multiplexing is done based on the URL. It is fully compatible with rack, but I would like to have it as a separate worker for better scalabillity.

• Client connects to the websocket server (proxy) with the URI: /ws/console/vm/123
• Proxy does an activerecord VM.find(123) and returns the host:port for VNC
• Proxy connects to the host:port with a regular socket
• Proxy forks a thread which will be responsible for the socket -> websocket dataflow
• The faye implementation is responsible for the websocket -> socket dataflow
• When the client disconnects from the websocket, the thread is killed and the socket is closed

The implementation is fully rack-compatible and it can be mounted as a rails route, but it would be better to have this as a separate worker in production. For development it can be in the config/routes.rb as:

mount WsProxy.server => '/ws/console' if Rails.env.development?

To continue with this I need a new worker, which will use a rackup file other than config.ru. Rumors say that @Fryguy can help me with this :)

cc: @martinpovolny @dclarizio @kbrock

[skateman]

As @matthewd said EventMachine is evil, so I reimplemented the proxy on a lower level without EM. Some exception handling and refactoring is still required but I think I can demo it this week.

[martinpovolny]

@skateman : come on he did not say that! But cool 👍

[skateman]

Probably found the solution for creating a new worker. If i replace the Rails.application here with a rack appliaction, it will run! 🍷
So if I inherit & redefine & copy some stuff, it should be okay. Can you confirm it @Fryguy ?

[martinpovolny]

model changes: #7224

[martinpovolny]

REST API skeleton: #7178

[Fryguy]

@skateman @martinpovolny Now that #7078 is merged, can we remove the changes to expose ports 5900-5999 and listed here? https://github.com/ManageIQ/manageiq-appliance-build/blob/master/kickstarts/partials/post/firewalld.ks.erb#L24

I believe those were open because of that weird VNC console thing we did years ago (with the browser plugins and the MiqServer acting as a VNC proxy server).

cc @bdunne @simaishi @jrafanie Can you comment on my thoughts here?

[Fryguy]

Also, I merged ManageIQ/manageiq-appliance#61 since it was related to #7078

[jrafanie]

@skateman @martinpovolny Now that #7078 is merged, can we remove the changes to expose ports 5900-5999 and listed here? https://github.com/ManageIQ/manageiq-appliance-build/blob/master/kickstarts/partials/post/firewalld.ks.erb#L24

@martinpovolny It would be awesome to remove that. Do we need something else open on the firewall?

[skateman]

@jrafanie nope, just close these ports

[martinpovolny]

@jrafanie: as @skateman wrote, we just close those. Everything is proxied through 443 now. Also all the SSL PITA is gone now. Have to update the docs.

[jrafanie]

That's awesome! 💖

----- Reply message -----
From: "Martin Povolny" [email protected]
To: "ManageIQ/manageiq" [email protected]
Cc: "Joe Rafaniello" [email protected]
Subject: [ManageIQ/manageiq] Websocket proxy and consoles next steps (#2451)
Date: Thu, Apr 7, 2016 4:52 PM

@jrafanie: as @skateman wrote, we just close those. Everything is proxied through 443 now. Also all the SSL PITA is gone now. Have to update the docs.

—You are receiving this because you were mentioned.Reply to this email directly or view it on GitHub

[kbrock]

This is very cool. Great job. Good write up AND good follow through removing the warts

[miq-bot]

This issue has been automatically marked as stale because it has not been updated for at least 6 months.

If you can still reproduce this issue on the current release or on master, please reply with all of the information you have about it in order to keep the issue open.

Thank you for all your contributions!

[skateman]

Ugh, just close this PR, it's done! 😉