Maintenance

[elisabettai]

Descriptions:
Regular maintenance tasks such as updates, patches, and upgrades should be performed to ensure that the platform is up to date and running smoothly. This includes database and server maintenance, as well as updating dependencies and libraries.

Deliverable:
Regular platform maintenance (updates in each NIH SIM-Core update meeting)

Deadline:
Continuous task

wrike Y7–(Task) 9b: Maintenance
Maintenance details in https://github.com/orgs/ITISFoundation/projects/9/views/34

[pcrespov]

Maintenance details in https://github.com/orgs/ITISFoundation/projects/9/views/34

[elisabettai]

Thanks for the link @pcrespov! Let's not forget to write a summary at the end of the sprint, so we can easily present some highlights to POs/NIH.

[elisabettai]

Y7Q3 Status update (05/27/2024): Continuously Ongoing
Implementation: Presented to the NIH at the monthly update meeting on 03/27/2024 (see slides and minutes).

Summary of maintenance highlights for Q3:

• Upgrade to Docker engine version 25, which also helps with performance (start time of services)
• Improved logging infrastructure to track computational tasks
• Reorganization of email templates to send messages to users (e.g. account creation confirmations, 2 factor-authentication, etc..): a new library has been created to handle such templates and an automatized procedure ensures proper review and sign-off by product owners
• Fixed a bug detected by automatic code scanning (Sonarcloud) which risked to result in undetected Service output updates, and consequently failure to update the GUI and down-stream pipeline.
• Admin Center (only for developers for now) to manage pricing plans of different o²S²PARC deployments
• Dask library “fixes” (Dask is a 3rd-party library used in the computational backend): workarounds were developed to overcome newly introduced bugs in Dask 2023.4.x and newer versions
• Mitigation of known, but unaddressed, issues in Docker Swarm and IP addresses: the platform can become "full" (no service can start) due to docker not releasing temporary IP addresses after container shut down. This is a known docker issue, much discussed online. We created elaborate custom IP-ranges for docker to use in order to mitigate such issues
• Various fixes to improve performance of the API server (need identified during testing of the metamodeling framework): for example, task were occasionally mis-reported as “started” or “failed” after launching many tasks in parallel

[elisabettai]

@pcrespov, just a message for the future related to the Maintenance cases. This is an ongoing case we have to report on to NIH (both in the monthly meetings and in the quarterly reports). I didn't check yet what this looks like in Y8, but I suspect it is going to be similar.

I have been using the maintenance detail board as you suggested above. I suggest you and the others come up with a different approach, since there's not going to be anymore a patient oSPARC Application and Support engineer which spend time turning that huge list of PRs/issues into some "human-readable text" (see example in the Y7Q3 report). This is a bit the same story of the "user-friendly" release notes. 😉

Pinging also @mrnicegyu11, since he's the current scrum master.

As a side note the board was moved recently, and is not anymore the one you posted in this comment