waitUntilReady leaves watches/websocket open upon timeout

[rnorth]

It appears that waitUntilReady for a Pod will leave a Watch, and therefore websocket connection, open if the Pod fails to become ready in time. This can lead to a buildup of open websocket connections to the k8s API, and eventually resource exhaustion.

We've not dug into every nuance, but in our usage we are deleting the pod after the timeout occurs. The watch seems to remain open despite both the timeout and the deletion.

One workaround seems to be to use a custom Watcher, and manually ensure that the ensuing Watch is closed when a failure occurs.

[shawkins]

What version of fabric8 is this on?

[rnorth]

We've observed this with 5.9.0 and 5.7.2.

[shawkins]

Under the covers the logic should terminate the informer / watch when the future completes -

kubernetes-client/kubernetes-client/src/main/java/io/fabric8/kubernetes/client/dsl/base/BaseOperation.java

Line 881 in 278ca23

future.whenComplete((r, t) -> informer.stop());

However I do see that the cancel is being applied to the wrong future. I've opened a pr for this.

[rnorth]

Aha, great, thank you @shawkins