External clusters: Enable selected AWS regions

[sanderegg]

Any EC2 instance started on AWS uses so called Service Quotas, that in effect limit the total amount of vCPUs one can start per account.
In order to increase that limit, one needs to deploy in different AWS regions (e.g. us-east-1, us-east-2, us-west-1, ...).

Tasks

Preview Give feedback

1. Create necessary infrastructure in AWS to allow usage of another region in AWS-master @YuryHrytsuk
2. Start all external clusters in a different AWS region than the dynamic services in AWS-master @sanderegg
3. Create necessary infrastructure in AWS to allow usage of another region in Sim4life.io @YuryHrytsuk
4. Start all external clusters in a different AWS region than the dynamic services in Sim4life.io @sanderegg

[sanderegg]

1st test included:

• AWS-master
• Modified clusters-keeper ENV variables such that clusters start in us-east-2
  • endpoint needed to be removed as it seems that setting https://ec2.amazonaws.com is incompatible with using us-east-2 region. to be further tested
• Needed to copy key-pair to new region, kept same name
• Needed to copy AMI to new region, new AMI ID --> check if there is another way maybe using the AMI Name field?
• Need to adapt the osparc-configs and management of ENV variables to allow to select different AMIIDs

[sanderegg]

AWS-master is now persistently running ext clusters in us-east-2.

• https://ec2.amazonaws.com is only for us-east-1 confirmed as explained https://docs.aws.amazon.com/general/latest/gr/ec2-service.html#ec2_region
• osparc-config adjusted to be able to have multiple regions, for multiple AMIs. Tracking could still be done by using a smart tagging system.
• osparc_clusters.py monitoring script needs to be adapted

[sanderegg]

this case is completed.