Performance Improvements for Large Projects

[SCA-ZMT]

User Story

GOAL

It is very important for the POs that the user experience of handling (potentially many and ) large projects is smooth.

This considers

• loading/startup time and user experience within the platform
• heavy projects, many smash/simulation projects (in the tree)
• submitting jobs to the comp backend, monitoring jobs
• receiving computed jobs and performing postprocessing

Singularity

Preview Give feedback

1. Filesystem - Large File Handling (Caching) osparc-simcore#6411
   3 of 7
   PO issue +1
   

   

Tasks

Preview Give feedback

1. Optimize EBS buffering osparc-simcore#6329
   1 of 8
   a:autoscaling a:clusters-keeper a:infra+ops +3
   
2. Use S3 transfer acceleration osparc-simcore#6247
   a:infra+ops +1
   
3. Check Ec2 networking tuning (ec2 type dependent?)

Event Horizon

Preview Give feedback

1. Startup of dynamic sidecar osparc-simcore#6859
   a:dynamic-sidecar a:infra+ops +2
   
2. RClone options tuning osparc-simcore#6858
   a:dynamic-sidecar +1
   
3. Autoscaling: When hot buffer are in use the warm buffer are not replacing them osparc-simcore#6929
   a:autoscaling bug +2
   

MartinKippenberger

Preview Give feedback

1. Use S3 VPC Gateway Endpoint osparc-simcore#5865
   0 of 1
   t:enhancement +1
   

   
2. Use deterministic zipping for inputs/outputs folders osparc-simcore#6244
   2 of 2
   a:dask-service a:simcore-sdk +2
   
3. api-server is aware of internal S3 upload links #1745
   +0
   

Eisbock

Preview Give feedback

1. pull docker images while pulling outputs and state osparc-simcore#5626
   a:director-v2 a:dynamic-sidecar t:enhancement +3
   
2. Preparing the performance check e2e #1674
   +0
   
3. Use initialized EBS storage instead of buffer machines osparc-simcore#5864
   17 of 17
   a:autoscaling a:infra+ops +2
   

[SCA-ZMT]

@drniiken @eofli Could you please edit the description on top with more concrete use case and expectations?

[SCA-ZMT]

@drniiken could you please update the description above? thanks! :)

[mguidon]

• Costs: S3: follow up with Dustin where this is enabled
• Performance: MaG: Need to check how this performs with the public API (we should get metrics for up/download after a fix in the code)
• Speed tests (for uploads)
• Optimization of EBS: cheap machines to save costs, double throughput with RAID0 w/o efs, look into metrics of usage
• Speed: Why takes the dynamic sidecar so long to start?

[sanderegg]

Summary for Event Horizon sprint

Done

• Autoscaling: When hot buffer are in use the warm buffer are not replacing them osparc-simcore#6929: Missing hot buffer machine was not replaced with an available warm buffer but from a machine started from scratch. Improvements in the order of 10 minutes on startup time when all hot buffers are used up.
• Startup of dynamic sidecar osparc-simcore#6859: typically takes 8 seconds to start on hot/warm buffers. Considered as done until further notice.
• RClone options tuning osparc-simcore#6858: Affects transfer of inputs/outputs/service data and allows to directly tune for better parameters. Allowed to improve massively on projects where many small files are present.
• ⚠️ In order to reduce costs the overall performance of the disks were slightly reduced, this should be barely visible for the users that get warm/hot machines, as the service images are already pulled. This might have an influence for zipping/unzipping or where IOPS are important. Please contact support if weird slow downs are detected.

Won't be done

• Use S3 transfer acceleration osparc-simcore#6247: after analysis this feature seems not very promising as it is mostly interesting for inter-region transfer which is not very useful at the moment. will not be implemented for now.

Planned for next sprint(s)

• Filesystem - Large File Handling (Caching) osparc-simcore#6411: cache service data on a separate disk that can be re-attached, this should bring noticeable improvements for large projects.