Comp backend/clusters creation functionalities

[sanderegg]

What do these changes do?

• added remote debugging via debugpy facilities
• moved dask-gateway fixtures into pytest-simcore package
• clusters-keeper now uses dask client to connect with created dask-gateway
• clusters-keeper defines RPC entrypoint to clusters-keeper.get_or_create a cluster for a user/wallet
• background task that auto-terminates external AWS clusters created by clusters-keeper based on:
  • terminate any cluster where *SERVICE_TRACKING_HEARTBEAT * CLUSTERS_KEEPER_MAX_MISSED_HEARTBEATS_BEFORE_CLUSTER_TERMINATION
  • set last heartbeat defined as a cluster exist in the dask-gateway, and there are tasks being processed or datasets saved in the cluster
• auto-create docker swarm on newly created primary cluster machine
• auto-start osparc-dask-gateway on newly created primary cluster machine

Next tasks

• potentially completely remove dask-gateway and replace with standard dask-scheduler/worker using tls protocol
• modify autoscaling service to understand dask tasks and create machines for the demand
• start autoscaling
• connect with director-v2

Related issue/s

• requires ✨RabbitMQ: Add RPC decorator for server side #4682
• requires Rabbitmq/separate clients #4685

How to test

DevOps Checklist

[codecov]

Codecov Report

Merging #4602 (dda0460) into master (df7114d) will decrease coverage by 2.0%.
The diff coverage is 94.7%.

@@           Coverage Diff            @@
##           master   #4602     +/-   ##
========================================
- Coverage    85.3%   83.3%   -2.0%     
========================================
  Files        1077    1086      +9     
  Lines       45240   45518    +278     
  Branches     1027    1027             
========================================
- Hits        38611   37938    -673     
- Misses       6409    7360    +951     
  Partials      220     220             

Flag	Coverage Δ
integrationtests	38.7% <ø> (-19.9%)	⬇️
unittests	83.1% <94.7%> (-1.3%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
...rc/simcore_service_clusters_keeper/rpc/clusters.py	46.4% <46.4%> (ø)
...rc/simcore_service_clusters_keeper/modules/dask.py	97.2% <97.2%> (ø)
...eeper/src/simcore_service_clusters_keeper/_meta.py	91.3% <100.0%> (+4.3%)	⬆️
.../src/simcore_service_clusters_keeper/api/health.py	100.0% <100.0%> (ø)
...imcore_service_clusters_keeper/core/application.py	100.0% <100.0%> (ø)
...src/simcore_service_clusters_keeper/core/errors.py	100.0% <100.0%> (ø)
...c/simcore_service_clusters_keeper/core/settings.py	100.0% <100.0%> (ø)
...eper/src/simcore_service_clusters_keeper/models.py	100.0% <100.0%> (ø)
...imcore_service_clusters_keeper/modules/clusters.py	100.0% <100.0%> (ø)
...lusters_keeper/modules/clusters_management_core.py	100.0% <100.0%> (ø)
... and 8 more

... and 76 files with indirect coverage changes

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
3.0% Duplication

[codeclimate]

Code Climate has analyzed commit dda0460 and detected 0 issues on this pull request.

View more on Code Climate.