Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚀 Pre-release master -> staging_sevenPeaks3 #5070

Closed
18 of 19 tasks
matusdrobuliak66 opened this issue Nov 22, 2023 · 1 comment
Closed
18 of 19 tasks

🚀 Pre-release master -> staging_sevenPeaks3 #5070

matusdrobuliak66 opened this issue Nov 22, 2023 · 1 comment
Assignees
Labels
release Preparation for pre-release/release t:maintenance Some planned maintenance work

Comments

@matusdrobuliak66
Copy link
Contributor

matusdrobuliak66 commented Nov 22, 2023

What kind of pre-release?

master branch

Sprint Name

sevenPeaks

Pre-release version

3

Commit SHA

5ad8c46

Did the commit CI suceeded?

  • The commit CI succeeded.

Motivation

  • Weekly release issue

What Changed

Devops check ⚠️ devops

  • Force deploy graylog and check that it is configured. If it goes good, remove configure_graylog pipeline stage
  • Check monitoring, registry and portainer services. Make sure introduced resource limits do not break anything
  • Run ansible for in-house deployments to introduce rsync copy backup cron job (AND to remove sshfs related cron jobs)
  • Restart PG-Backup (manually)
  • Check that dynamic-sidecar and computational backend sidecar image uses proper image tag @sanderegg

e2e testing check 🧪

No response

Summary 📝

  • make release-staging name=<sprint_name> version=<version> git_sha=5ad8c463893e2ddbd348d390e7e109450769ba82
    • https://github.com/ITISFoundation/osparc-simcore/releases/new?prerelease=1&target=<commit_sha>&tag=staging_<sprint_name><version>&title=Staging%20<sprint_name><version>
  • Draft pre-release
  • Announce (add redis key maintenance in every concerned deployment)
    {"start": "2023-02-01T12:30:00.000Z", "end": "2023-02-01T13:00:00.000Z", "reason": "Release ResistanceIsFutile9 "}
  • Announce release in Mattermost
    :loud_sound:  Maintenance scheduled for **NAMED_DAY DD. MM from START_TIME - END_TIME**.
    =========================================================================
    
    @all Be aware that you will automatically be logged out and your projects stopped and saved during the maintenance time. Affected:
    *   [https://staging.osparc.io](https://staging.osparc.io/)
    *   [https://https://staging.s4l-lite.io/](https://https://staging.s4l-lite.io//)
    
    and on premises:
    *   [https://osparc-staging.speag.com](https://osparc-staging.speag.com/)
    *   [https://tip-staging.speag.com](https://tip-staging.speag.com/)
    *   [https://s4l-staging.speag.com](https://s4l-staging.speag.com/)
    *   [https://s4l-lite-staging.speag.com](https://s4l-lite-staging.speag.com/)
    
    
    Reason: Scheduled staging-release of STAGING_NAME_AND_VERSION.
    
    Thanks for your understanding and sorry for the inconveniences,
    
    Your friendly oSparc Team
    
    

Releasing

  • Release (release draft)
  • Check Release CI
  • Check hanging sidecars. Helper command to run in director-v2 CLI simcore-service-director-v2 close-and-save-service <uuid>
  • Check deployed
    • aws deploy
    • dalco deploy
  • Delete announcement
  • Check e2e runs
  • Announce
https://github.com/ITISFoundation/osparc-simcore/releases/tag/staging_<sprint_name><version>
@matusdrobuliak66 matusdrobuliak66 added t:maintenance Some planned maintenance work release Preparation for pre-release/release labels Nov 22, 2023
@matusdrobuliak66 matusdrobuliak66 self-assigned this Nov 22, 2023
@YuryHrytsuk
Copy link
Contributor

YuryHrytsuk commented Nov 23, 2023

Issues:

  • On aws staging the following services lacked system resources (fixed by removing reservations)
    • clusters-keeper
    • dask-scheduler
  • aws stag: clusters_keeper ENV is wrong (manually updated with @sanderegg)
  • dalco staging: webserver failed to start itself with new tag. manual rescale + restart + retag helped

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release Preparation for pre-release/release t:maintenance Some planned maintenance work
Projects
None yet
Development

No branches or pull requests

2 participants