[Feature] Mark-and-sweep GC.

[ruseinov]

Summary of changes

Changes introduced in this pull request:

• Introduces a mark-and-sweep garbage collector.

Reference issue to close (if applicable)

Closes #3072

Other information and links

This will require a 0.13.1 release due to a database migration needed to deprecate RollingDB.
Perhaps a good idea is to incorporate the CHANGELOG.md changes within this PR as well to simplify things.

Change checklist

• I have performed a self-review of my own code,
• I have made corresponding changes to the documentation,
• I have added tests that prove my fix is effective or that my feature works (if possible),
• I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

[ruseinov]

@lemmih This design is not taking into account the AnyCar addition. I'm assuming we do want to traverse the reachable graph including the car storage, correct?

[lemmih]

@lemmih This design is not taking into account the AnyCar addition. I'm assuming we do want to traverse the reachable graph including the car storage, correct?

The CAR files contain read-only data. There's no need for the GC to traverse them. In the future, we do want to find unused CAR files but you can leave that for another PR.

[ruseinov]

The CAR files contain read-only data. There's no need for the GC to traverse them. In the future, we do want to find unused CAR files but you can leave that for another PR.

Right, that makes sense.

[ruseinov]

Let's merge this PR as-is and see if we get the bandwidth to improve the tests later.

Sounds good!

[elmattic]

What should depth be initialized to? In the associated issue we're speaking of a number of 2 x chain_finality, here we're taking max of chain_finality and recent_state_roots.

[ruseinov]

It's not garbage collector's concern, the decision is made by the caller. But it's a good catch, and I will comment and fix it to be 2x chain_finality.

[elmattic]

Should this new implementation of GC fix our long-standing send issue? (#3645)
I would really like to run my script, but it would not be possible without a way to trigger your GC.

Eventually, let's try to uncomment the test in calibnet_wallet_check.sh and see what happens.

[ruseinov]

Should this new implementation of GC fix our long-standing send issue? (#3645) I would really like to run my script, but it would not be possible without a way to trigger your GC.

Eventually, let's try to uncomment the test in calibnet_wallet_check.sh and see what happens.

Manual GC is deprecated, it makes no sense with the current approach.

[lemmih]

@ruseinov You probably have to update the branch before the CI checks will run.

[ruseinov]

@ruseinov You probably have to update the branch before the CI checks will run.

so it seems!