Skip to content

Commit

Permalink
Change the MergingRun credit tracking strategy
Browse files Browse the repository at this point in the history
The intention is to simplify things and make them more obviously correct
in the presence of concurrency. This should have the bonus of making it
easier to reliably determine leftover/excess supplied credits (which is
something we want for supplying credits to trees of merging runs).

The approach is to change the counters from three independent counters,
to just two, which are modified together as a pair atomically.
Previously we tracked the credits spent and unspent, and the steps
performed. We did not explicitly keep track of credits that were in the
process of being spent.

Now we track spent and unspent (and not steps performed), but the spent
credits includes those that are in the process of being spent. We keep
these together in a single atomic variable, and so all operations on the
pair are atomic. This makes the concurrency story much simpler because
all credit tracking changes are atomic. We avoid having to track steps
performed by accounting differently for the difference between credits
used for merging and steps performed: we simply borrow more credits from
the unspent pot, allowing the pot to become negative.
  • Loading branch information
dcoutts committed Feb 3, 2025
1 parent 492094a commit 5e55b46
Show file tree
Hide file tree
Showing 6 changed files with 392 additions and 278 deletions.
10 changes: 2 additions & 8 deletions src-extras/Database/LSMTree/Extras/NoThunks.hs
Original file line number Diff line number Diff line change
Expand Up @@ -321,14 +321,8 @@ deriving anyclass instance NoThunks MergePolicyForLevel
deriving stock instance Generic NumRuns
deriving anyclass instance NoThunks NumRuns

deriving stock instance Generic (UnspentCreditsVar s)
deriving anyclass instance Typeable s => NoThunks (UnspentCreditsVar s)

deriving stock instance Generic (StepsPerformedVar s)
deriving anyclass instance Typeable s => NoThunks (StepsPerformedVar s)

deriving stock instance Generic (SpentCreditsVar s)
deriving anyclass instance Typeable s => NoThunks (SpentCreditsVar s)
deriving stock instance Generic (CreditsVar s)
deriving anyclass instance Typeable s => NoThunks (CreditsVar s)

deriving stock instance Generic MergeKnownCompleted
deriving anyclass instance NoThunks MergeKnownCompleted
Expand Down
2 changes: 1 addition & 1 deletion src/Database/LSMTree/Internal/MergeSchedule.hs
Original file line number Diff line number Diff line change
Expand Up @@ -837,4 +837,4 @@ scaleCreditsForMerge LevelLevelling (DeRef mr) (Credits c) =
creditThresholdForLevel :: TableConfig -> LevelNo -> MR.CreditThreshold
creditThresholdForLevel conf (LevelNo _i) =
let AllocNumEntries (NumEntries x) = confWriteBufferAlloc conf
in MR.CreditThreshold x
in MR.CreditThreshold (MR.Credits x)
Loading

0 comments on commit 5e55b46

Please sign in to comment.