(bug) better handle deleted cluster

[gianlucam76]

When cluster is still present but marked as deleted, Sveltos can avoid removing resources (helm charts and/or kubernetes resources) from such a cluster.
Anything else created in the management cluster for such managed cluster still needs to be removed.

Fixes #735
Fixes #732

[TheNilesh]

Thank you, @gianlucam76, for addressing this. Would it be possible to create a release that includes this change?

[gianlucam76]

Hi @TheNilesh yes. I was thinking to do release on Monday. Let me know if this is urgent. I can create a temporary tag with this fix

Though just to clarify, there was nothing wrong with previous Sveltos behaviour.

I added this simply because a bug in the AWS clusterAPI infrastructure provider was not removing a deleted cluster. And so another user ended up with deleted Profile also remaining in the management cluster.
If the deleted clusterAPI is removed correctly, then Sveltos has no problem.

With that I mean, that even with 0.39.0 and 0.40.0, if a ClusterAPI cluster is deleted, Sveltos already cleans resources in the management cluster correctly.

[TheNilesh]

Thank you! A Monday release should work.

We’re currently using version v0.39.0 of the Sveltos-addon controller and consistently notice that addons aren’t always removed after a CAPI cluster is deleted. From the logs, it appears Sveltos is still attempting to reach the deleted control plane. It seems that, after a certain number of attempts, it stops trying and leaves the addons in place indefinitely. While my observation could be incorrect, I built a new image of the addon-controller with the commits from this PR, and cleanup worked as expected.

Could you please confirm if this fix addresses the same issue or a different one?

[gianlucam76]

Thanks. For me to properly answer, can you tell me if your deleted CAPI cluster is still present in the management cluster (in deleted state)?

[TheNilesh]

No, it is gone.

[gianlucam76]

Thanks. That's strange. If you were deploying just helm charts, v0.39.0 should already have fix. Meaning Sveltos should have correctly removed all resources deployed in the management cluster.

In any case, this fix better handles that situation. Sveltos, when detects cluster is marked as deleted (even though still present) does not even try to access it and simply moves forwards removing resources in the management cluster (if any).

[TheNilesh]

I’m unable to reproduce the issue now. It may be a different problem. If it occurs again, should I create a new GH issue and post logs there?

[gianlucam76]

yes please. If you get to repro please file a GH issue with logs. Thanks

[gianlucam76]

@TheNilesh just released v0.41.0