🐛Autoscaling/Comp backend: drain retired nodes so that they can be re-used

[sanderegg]

What do these changes do?

Autoscaled Cluster was expanded to also keep retired_nodes.
These nodes were retired via dask scheduler mechanism, that removes idle workers and transfer their memory to other workers based on some internal mechanisms.

These "retired" workers are now properly recognized, and are forcefully drained. This way they can be:

• re-used, by undraining and restarting a dask-sidecar in case a new arrives in between,
• or terminated if there is nothing else to do

Related issue/s

• fixes Computational cluster: retired worker not terminated #6319

How to test

Dev-ops checklist

• No ENV changes or I properly updated ENV (read the instruction)

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 64.5%. Comparing base (cafbf96) to head (18cce86).
Report is 523 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (cafbf96) and HEAD (18cce86). Click for more details.

HEAD has 1 upload less than BASE

Flag	BASE (cafbf96)	HEAD (18cce86)
unittests	1	0

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #6345      +/-   ##
=========================================
- Coverage    84.5%   64.5%   -20.1%     
=========================================
  Files          10     591     +581     
  Lines         214   30224   +30010     
  Branches       25     260     +235     
=========================================
+ Hits          181   19511   +19330     
- Misses         23   10653   +10630     
- Partials       10      60      +50     

Flag	Coverage Δ
integrationtests	64.5% <ø> (?)
unittests	?

Flags with carried forward coverage won't be shown. Click here to find out more.

see 601 files with indirect coverage changes

[matusdrobuliak66]

👍