[Incident] US cluster was unstable again throughout the weekend

[kunxin-chor]

Bug description

Attempting to log into Gitpod via Github results in either 504 or the page stuck on loading. Tried clearing cache, different browsers etc.

Steps to reproduce

1. Clear cache, cookies and all site data
2. Go to https://www.gitpod.io
3. Click on Login
4. Click on Github
5. Log into Github
6. Redirect is stuck

Expected behavior

Able to launch workspaces

Example repository

No response

[kevinadhiguna]

I am facing an exactly same situation. Here are the logs and network tabs :

Nevertheless, what I wonder is status.gitpod.io says all services are operational :

Any help will be appreciated, thanks!

Edit : I tried logging in using Gitlab but still facing the same issue..

[amymaldonado]

also experiencing the same. EDIT: working now! EDIT AGAIN: stopped working again :(

[kunxin-chor]

It's still not working for myself. I cannot retrieve any workspaces, and attempt to login again results in no login buttons showing up:

[bigint]

Same, this is happening for me too!

[kunxin-chor]

And now it's back to yesterday's issues where returning to https://www.gitpod.io results in a blank page.

[amafjarkasi]

Still not working - none of the extensions or anything else are loading up

[jankeromnes]

Hi all. We are deeply sorry that, following #3499 and its mitigation, the US cluster relapsed over the weekend, offering on-and-off service until Monday morning western Europe time (where most of our small team is located). This is not acceptable, and we should do better.

The service is now fully operational again, and the team is focused on better understanding and permanently resolving several underlying issues that happened since Friday's deployment.

In parallel to that, we will step our our reliability and incident response processes, in particular by:

• reiterating the importance of communicating incidents & responses (in issues, the community forum, twitter, and status.gitpod.io)
• improving our monitoring and alerting in light of this weekend's issues
• accelerating our plans toward implementing a dedicated 24/7 on-call ops rotation, so as to more effectively catch incidents outside (western Europe) working hours

[kunxin-chor]

And it's down again.

[warriorBrian]

@jankeromnes
I think the problem remains, report:
Sorry, something went wrong 😓
Error: Connection got disposed.