Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: lock keys for optimistic transactions #3865

Merged
merged 5 commits into from
Oct 8, 2024
Merged

chore: lock keys for optimistic transactions #3865

merged 5 commits into from
Oct 8, 2024

Conversation

kostasrim
Copy link
Contributor

We did not acquire any locks for transactions that are executing optimistically. However, this is problematic for callbacks that need to preempt (e.g. because a journal is active).

resolves #3841

@kostasrim kostasrim self-assigned this Oct 4, 2024
@kostasrim kostasrim requested a review from dranikpg October 4, 2024 12:32
sd.local_mask |= OPTIMISTIC_EXECUTION;
shard->stats().tx_optimistic_total++;

RunCallback(shard);

// Check state again, it could've been updated if the callback returned AVOID_CONCLUDING flag.
// Only possible for single shard.
if (coordinator_state_ & COORD_CONCLUDING)
if (coordinator_state_ & COORD_CONCLUDING) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dranikpg At first I was confused by this because for execute_optimistic to be true we need coordinator_state_ & COORD_CONCLUDING. However, I found out by looking at the stream family, that there are transactions that start with this state and then they might AVOID_CONCLUDING depending on the callback result. I almost removed the if here 😄

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, I found out by looking at the stream family,

You could've just looked at the comment above 😂 I don't write them just for fun

@@ -1060,26 +1077,25 @@ bool Transaction::ScheduleInShard(EngineShard* shard, bool execute_optimistic) {
// Check if we can run immediately
if (shard_unlocked && execute_optimistic &&
CheckLocks(GetDbSlice(shard->shard_id()), mode, lock_args)) {
// We need to acquire the fp locks because the executing callback
// within RunCallback might preempt.
acquire_fp_locks(shard_unlocked);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also thought adding a check here and make acquire_fp_locks return a bool. But there is no really a need since CheckLocks gurantees that we hold none.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like if we acquire locks, we can remove CheckLocks. Instead the condition will use lock_granted

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the code here should be rewritten. We acquire locks in any case, so CheckLocks is not needed

@kostasrim
Copy link
Contributor Author

@dranikpg you are our transaction guy so plz let me know if I forgot anything -- highly likely 😄

dranikpg
dranikpg previously approved these changes Oct 4, 2024
Copy link
Contributor

@dranikpg dranikpg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks valid. But maybe we can now remove the use of CheckLocks and instead use the success status of lock acquisition to decide on optimistic execution. This should also simplify the flow as we acquire locks unconditionally

sd.local_mask |= OPTIMISTIC_EXECUTION;
shard->stats().tx_optimistic_total++;

RunCallback(shard);

// Check state again, it could've been updated if the callback returned AVOID_CONCLUDING flag.
// Only possible for single shard.
if (coordinator_state_ & COORD_CONCLUDING)
if (coordinator_state_ & COORD_CONCLUDING) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, I found out by looking at the stream family,

You could've just looked at the comment above 😂 I don't write them just for fun

@@ -1060,26 +1077,25 @@ bool Transaction::ScheduleInShard(EngineShard* shard, bool execute_optimistic) {
// Check if we can run immediately
if (shard_unlocked && execute_optimistic &&
CheckLocks(GetDbSlice(shard->shard_id()), mode, lock_args)) {
// We need to acquire the fp locks because the executing callback
// within RunCallback might preempt.
acquire_fp_locks(shard_unlocked);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like if we acquire locks, we can remove CheckLocks. Instead the condition will use lock_granted

Signed-off-by: kostas <[email protected]>
@kostasrim kostasrim requested review from dranikpg and romange October 7, 2024 06:49
dranikpg
dranikpg previously approved these changes Oct 7, 2024
Comment on lines 1069 to 1071
// We need to acquire the fp locks because the executing callback
// within RunCallback below might preempt.
acquire_fp_locks(shard_unlocked);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nits:

  1. Maybe we don't need acquire_fp_locks as a separate function
  2. If you decide to keep it, maybe return lock_granted, otherwise the flow might be somewhat confusing, looking at the if below (i.e. not clear where lock_granted changed)

Copy link
Contributor

@dranikpg dranikpg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻

@kostasrim kostasrim merged commit c1e9d51 into main Oct 8, 2024
12 checks passed
@kostasrim kostasrim deleted the kpr7 branch October 8, 2024 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants