chore: run memory decommit after snapshot load/save #3828
Conversation
Signed-off-by: kostas <[email protected]>
kostasrim
changed the title
|
src/server/rdb_save.cc
Outdated
| // Decommit local memory. | ||
| // We create an RdbSaver for each thread, so each one will Decommit for itself. | ||
| auto* tlocal = ServerState::tlocal(); | ||
| if (tlocal) { |
There was a problem hiding this comment.
tests 😄
For example LoadEmpty in rdb_test.cc, does not run RdbLoad::Load in a shard thread so it crashes.
I do not know if a similar unit test exist for rdb_save but I assumed this is the case and I added this here as well to avoid crashes like that
There was a problem hiding this comment.
that odd because rdb_save calls service init. please check this again why it is not set
There was a problem hiding this comment.
I don't see why this is odd. The service is created but we just don't run the rdb loader on a proactor thread
src/server/rdb_load.cc
Outdated
| // full sync ends (since we explicitly reset the RdbLoader). | ||
| auto* tlocal = ServerState::tlocal(); | ||
| if (tlocal) { | ||
| tlocal->DecommitMemory(ServerState::kDataHeap | ServerState::kBackingHeap | |
There was a problem hiding this comment.
How about creating a ServerState::kAllMemory to contain all bits?
There was a problem hiding this comment.
I think this should mean good idea in Hebrew 🤣
src/server/server_state.h
Outdated
| kDataHeap = 1, | ||
| kBackingHeap = 2, | ||
| kGlibcmalloc = 4, | ||
| kAll = kDataHeap | kBackingHeap | kGlibcmalloc |
There was a problem hiding this comment.
Since this is an anonymous enum, I'd use a more verbose name. Maybe kAllHeaps or kAllMemory?
Alternatively we can name this enum (preferably use enum class) so that it will not be confusing.
Anyway, this is a nit so feel free to ignore.
Sometimes for large values during snapshot loading/saving we allocate a lot of extra memory. For that, we might need to manually run memory decommit for mimalloc to release memory pages back to the OS. This PR addresses that by manually running memory decommit after each shard finishes loading or saving a snapshot. --------- Signed-off-by: kostas <[email protected]>
Sometimes for large values during snapshot loading/saving we allocate a lot of extra memory. For that, we might need to manually run
memory decommitformimallocto release memory pages back to the OS. This PR addresses that by manually runningmemory decommitafter each shard finishes loading or saving a snapshot.