Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use partitioned lock to optimize disk cache #914

Closed
jiacai2050 opened this issue May 23, 2023 · 1 comment · Fixed by #974
Closed

Use partitioned lock to optimize disk cache #914

jiacai2050 opened this issue May 23, 2023 · 1 comment · Fixed by #974
Assignees
@tanruixiang
Labels
feature New feature or request

Comments

@jiacai2050
Copy link
Contributor

Describe This Problem

Partitioned lock is a common trick to reduce lock contention, disk cache could use it to improve perf.

https://github.com/CeresDB/ceresdb/blob/da6899c4c97089d4fbd8aa8a01db98ab8366d5bc/components/object_store/src/disk_cache.rs#L114

Proposal

Use PartitionedRwLock to replace cache in DiskCache

https://github.com/CeresDB/ceresdb/blob/da6899c4c97089d4fbd8aa8a01db98ab8366d5bc/common_util/src/partitioned_lock.rs#L17

Additional Context

No response
@jiacai2050 jiacai2050 added the feature New feature or request label May 23, 2023
@tanruixiang
Copy link
Member

please assign to me.

@tanruixiang tanruixiang mentioned this issue May 23, 2023
Merged
jiacai2050 pushed a commit that referenced this issue May 24, 2023
@tanruixiang
refactor: use binary bits to determine the number of partitions (#919)
0317948 
## Related Issues
Prepare for #914 

## Detailed Changes
- Modify the type of `partitions` in `PartitionedMutex` and
`PartitionedRwLock`.
- Fix the bug that multiple partitions use the same lock.
## Test Plan 
Unit tests under the same file.
@tanruixiang tanruixiang mentioned this issue May 26, 2023
Merged
jiacai2050 pushed a commit that referenced this issue May 29, 2023
@tanruixiang
refactor: use partition lock for memory cache (#936)
005eb26 
## Related Issues
Ralated #914 

## Detailed Changes
Add `build_fixed_seed_ahasher` to build fixed  seeds ahasher.
Use PartitionedMutex in `MemCache`.

## Test Plan 
UT.
@tanruixiang tanruixiang mentioned this issue Jun 7, 2023
Merged
jiacai2050 pushed a commit that referenced this issue Jun 7, 2023
@tanruixiang
refactor: disk cache use partition lock (#974)
25899eb 
## Rationale
Close #914

## Detailed Changes
Use `partition lock` in `disk cache`

## Test Plan
add ut.
dust1 pushed a commit to dust1/ceresdb that referenced this issue Aug 9, 2023
@tanruixiang @dust1
refactor: disk cache use partition lock (apache#974)
a1bfe42 
## Rationale
Close apache#914

## Detailed Changes
Use `partition lock` in `disk cache`

## Test Plan
add ut.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tanruixiang tanruixiang

Labels
feature New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

refactor: disk cache use partition lock tanruixiang/incubator-horaedb
2 participants
@jiacai2050 @tanruixiang