Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: prevent accidental removal of valid sector index announcements #1396

Merged
merged 1 commit into from
Apr 26, 2023

Conversation

jacobheun
Copy link
Contributor

@jacobheun jacobheun commented Apr 26, 2023

Summary

We discovered an issue where when calling the Lotus Storage API we were not properly handling the multiple states that can be returned for a sector (Cache status, Unsealed/Sealed status). This could result in a sector being considered as removed.

This update adds a state to the database to account for the various Cache states a sector may be in. In most instances the Sealed/Unsealed status should ultimately be recorded, but in the instance that is not returned for some reason, we will record the cache status so that the sector is not removed. Only if no status is returned will we consider the sector as removed.

  • Also fixes an issue where the database was not treating sector entries as unique, leading to numerous entries per sector instead of 1
  • Ensures the Database starts up and runs any migrations before the rest of the index provider starts

TODO:

  • Verify fix on SP
    • Ran multiple jobs to verify new index announcements were not being made, and we were no longer seeing incorrect removals (this previously was occurring on every run)
    • Deleted known entries from the DB and verified announcements were triggered

Note: This will result in a full index ingest for affected users.

fix: add cache tests and dont announce cache state
fix: add unique index to sector state db
fix: sealed and unsealed sector state conflict
fix: ensure index provider wrapper starts after db migration has completed
@jacobheun jacobheun marked this pull request as ready for review April 26, 2023 12:49
@jacobheun jacobheun merged commit 68e23f8 into main Apr 26, 2023
@jacobheun jacobheun deleted the fix/unseal-mgr branch April 26, 2023 13:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant