Decommit unused pages in the txn metadata table

[senderista]

This is the final PR in the "txn metadata GC" series. It implements the actual reclamation of unused memory in the txn table.

For anyone interested: in the comments on truncate_txn_table() I discuss the choice of flags to madvise(2). I originally used MADV_FREE because of its purported performance advantages (it allows the OS to lazily reclaim decommitted pages instead of immediately unmapping them), but because this has no effect on the RSS charge of the process, it's impossible to observe any effect in any memory management tools (including cgroups, which is extremely problematic). Apparently MacOS has a better solution, but on Linux we're stuck with this unpleasant tradeoff. I decided that for now it's more important to be able to reliably observe committed physical memory than to chase hypothetical performance gains, so I reverted to MADV_DONTNEED. The Golang runtime had a similar experience: golang/go#42330.

[LaurentiuCristofor]

Didn't you use a different type for memory addresses? I remember seeing uintptr_t or something like that in other changes. It would be nice to be consistent about the data type that we use with page addresses.

[senderista]

I normally use void* for memory addresses, but here I need char* because I need to do pointer arithmetic on the address (and that is UB on void*).

[yiwen-wong]

I think this will cause an issue (SIGBUS? SIGSEGV?) with UNIQUE indexes. We need to figure out a strategy for this before committing this change.

https://github.com/gaia-platform/GaiaPlatform/blob/master/production/db/core/src/index_builder.cpp#L167

[LaurentiuCristofor]

I think this will cause an issue (SIGBUS? SIGSEGV?) with UNIQUE indexes. We need to figure out a strategy for this before committing this change.

https://github.com/gaia-platform/GaiaPlatform/blob/master/production/db/core/src/index_builder.cpp#L167

But isn't the decommitting occurring after the index entries would have been garbage collected or is there no coordination of the sort?

[yiwen-wong]

I think this will cause an issue (SIGBUS? SIGSEGV?) with UNIQUE indexes. We need to figure out a strategy for this before committing this change.
https://github.com/gaia-platform/GaiaPlatform/blob/master/production/db/core/src/index_builder.cpp#L167

But isn't the decommitting occurring after the index entries would have been garbage collected or is there no coordination of the sort?

No, the issue is committed entries with an old begin_ts will still be around as those entries cannot be garbage collected since they are still pointing to valid data.
The solution that comes to mind is we have to mark the begin_ts as 'frozen' (Postgres reserves a special value, but c_invalid_txn_id might work but is confusing) before actually collecting the metadata.

[senderista]

@yiwen-wong and I had a long discussion, the conclusion of which was that we need to make some minor coordinated changes to ensure the safety of index entries before this PR can be merged. I will be publishing a PR to revise the watermark logic with this in mind, and @yiwen-wong will follow with her "frozen timestamp" proposal for index entries whose committing txn has been GC'ed.

[senderista]

I just merged #1076, which will unblock this PR when combined with @yiwen-wong 's upcoming change.

[yiwen-wong]

unblocked

[yiwen-wong]

LGTM

[senderista]

I wanted to test these combined changes with the palletbox workload, but I can't manage to build it, so I'm merging now and crossing my fingers there are no latent issues.