Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RESOLVED] ⚠️Production outage: bot ceased receiving events from Discord after about 2022-10-13 15:20 UTC #199

Closed
kevinlul opened this issue Oct 14, 2022 · 1 comment
Closed

[RESOLVED] ⚠️Production outage: bot ceased receiving events from Discord after about 2022-10-13 15:20 UTC #199

kevinlul opened this issue Oct 14, 2022 · 1 comment
Labels
production Incidents from the running Bastion#1870 instance

Comments

@kevinlul
Copy link
Contributor

Resolved 2022-10-14T01:42:30.144Z by container reboot

After a search at 2022-10-13T15:20:03.012Z and a shard reconnect at 2022-10-13T15:24:38.261Z, Bastion's new instance ceased receiving events from the Discord gateway for over 10 hours before it was noticed by developers. The cause is unclear as the program remained running. Ping response, Slash Commands, and new <> search were down during this time.
@kevinlul kevinlul added the production Incidents from the running Bastion#1870 instance label Oct 14, 2022
This was referenced Oct 17, 2022
Closed
Merged
kevinlul added a commit that referenced this issue Oct 18, 2022
@kevinlul
Add early warning canary if no messageCreate events are received for …
19b6440 
…one minute (#208)

Given #199 and #204, this is an interim warning system before we can get a full healthcheck bot going with incident notifications to GitHub
@kevinlul kevinlul mentioned this issue Oct 23, 2022
Closed
@kevinlul kevinlul mentioned this issue Nov 22, 2022
Closed
@kevinlul kevinlul mentioned this issue Dec 20, 2022
Closed
@kevinlul
Copy link
Contributor Author

Fixed in a079568 (discordjs/discord.js#8989 is included in 14.8.0 and closed discordjs/discord.js#8486) unless proven otherwise

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
production Incidents from the running Bastion#1870 instance
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kevinlul