Plans for implementing Zalenium Grid based on Selenium 4 Grid

[rvarum]

Hi Zalenium Team,

I was wondering if you have plans to implement a Zalenium grid based on the Selenium 4 grid which was re-written and should be much more robust and stable.

and kudos guys for this wonderful project!

[pearj]

I did investigate this December last year when I had a few days at work to work on whatever I wanted and @diemol told me this:

basically, the new grid is splitting the hub in three parts

• router -> it is the front, routes calls to nodes or to create a new session
• sessionMap -> stores associations of sessions and nodes
• distributor -> creates sessions and registers new nodes

Unfortunately, though, I spent most of that time learning the buck build tool (that's what selenium 4.x uses as the build tool), as I found it rather odd to work with (I'm used to maven and gradle).

I'm only interested in the Kubernetes side, so the way I thought of implementing it would be to have the sessionMap actually implemented directly as annotations on the selenium pods in kubernetes (essentially following my kubernetes HA suggestions in #484). This would obviously make parts of the implementation very specific to running in a kubernetes cluster though.

I suppose the vanilla docker implementation could use the reference implementation of the sessionMap I guess?

@rvarum how do you currently run zalenium?

Saying all this, I don't have any plans at present to implement this, as I use Selenium quite infrequently these days. Not saying I wouldn't implement it, because I do like the idea of making Zalenium a bit more resilient to changes in the kubernetes cluster, but it would end up as after hours work.

[diemol]

To be honest, the path isn't quite clear. My initial goal would be to put as much as possible from Zalenium's functionalities into the upstream Grid, and perhaps have the extra bits that don't fit there in an additional project.
With that to make a scenario where it is more simple to maintain both projects.

[pearj]

@diemol, so maybe put all the docker container creation code (docker, swarm and k8s) in selenium directly? And then maybe just some of the UI bits in a separate project?

Have you talked to Simon about it? Is he keen for that?

[diemol]

Yes, the alpha of the new Grid already starts a docker container itself. I am not sure how we will handle the Kubernetes stuff.

[rvarum]

@rvarum how do you currently run zalenium?

sorry for the late response @pearj .
I'm using Zalenium in some sort of an pre made AMI that is spawned from jenkins as a slave agent.
Both hub and nodes images are on the same machine.
Usually the agent with the Zalenium grid is started for each Test execution build in jenkins.
We run tests in parallel leveraging the grid for that (about 10 concurrent sessions on a pretty strong EC2 instance).
We are currently not using Kubernetes in our company.

to be honest, most of my motivation for using Selenium 4 grid is stability.
we are getting the well known FORWARDING_TO_NODE_FAILED errors from time to time and it's really annoying :)
my hope is that Selenium 4 grid handles the networking layer better specially in parallel sessions.

[pearj]

@rvarum fair enough, for what it's worth, I haven't seen the FORWARDING_TO_NODE_FAILED error when running in Kubernetes, my suspicion is simply that there are more resources available. Since you're using Amazon, you could try deploying it on EKS. But if you've never used Kubernetes before there will be a bit of a learning curve.

[diemol]

Just an update here, due to the changes that Grid 4.x will have, it becomes really hard to migrate Zalenium to use it. It will be easier to write it from scratch.
Therefore, I have decided to work in the upstream docker-selenium project and port as many functionalities from Zalenium to it. If anyone wants to give us a hand, just start checking the open issues in the Selenium project.
Additionally, I will be around the Zalenium project in case anyone wants to help with it, I will contribute by reviewing and merging PRs, but won't be around to implement new functionalities.

I will close this issue, based on my comment.