Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

statcache: add bench for LRU/mapCache #45277

Merged
merged 11 commits into from
Jul 12, 2023
Merged

statcache: add bench for LRU/mapCache #45277

merged 11 commits into from
Jul 12, 2023

Conversation

hawkingrei
Copy link
Member

@hawkingrei hawkingrei commented Jul 10, 2023 •
edited
Loading

What problem does this PR solve?

Issue Number: ref #45281

Problem Summary:

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 10, 2023
@hawkingrei
Copy link
Member Author

/retest

@hawkingrei hawkingrei force-pushed the add_bench_for_cache branch from 7648328 to 8857a4f Compare July 11, 2023 03:30
@hawkingrei hawkingrei requested a review from qw4990 July 11, 2023 04:00
@hawkingrei hawkingrei force-pushed the add_bench_for_cache branch from f7b8b14 to 10e58cf Compare July 11, 2023 06:03
@hawkingrei
Copy link
Member Author

/test tiprow_fast_test

@ti-chi-bot
Copy link

ti-chi-bot bot commented Jul 11, 2023

@hawkingrei: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test build
  • /test canary-scan-security
  • /test check-dev
  • /test check-dev2
  • /test mysql-test
  • /test pingcap/tidb/canary_ghpr_unit_test
  • /test pull-br-integration-test
  • /test pull-e2e-test
  • /test pull-integration-copr-test
  • /test pull-integration-jdbc-test
  • /test pull-integration-mysql-test
  • /test unit-test

Use /test all to run the following jobs that were automatically triggered:

  • pingcap/tidb/ghpr_build
  • pingcap/tidb/ghpr_check
  • pingcap/tidb/ghpr_check2
  • pingcap/tidb/ghpr_mysql_test
  • pingcap/tidb/ghpr_unit_test

In response to this:

/test tiprow_fast_test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@hawkingrei
Copy link
Member Author

/test tiprow_fast_test

@ti-chi-bot
Copy link

ti-chi-bot bot commented Jul 11, 2023

@hawkingrei: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test build
  • /test canary-scan-security
  • /test check-dev
  • /test check-dev2
  • /test mysql-test
  • /test pingcap/tidb/canary_ghpr_unit_test
  • /test pull-br-integration-test
  • /test pull-e2e-test
  • /test pull-integration-copr-test
  • /test pull-integration-jdbc-test
  • /test pull-integration-mysql-test
  • /test unit-test

Use /test all to run the following jobs that were automatically triggered:

  • pingcap/tidb/ghpr_build
  • pingcap/tidb/ghpr_check
  • pingcap/tidb/ghpr_check2
  • pingcap/tidb/ghpr_mysql_test
  • pingcap/tidb/ghpr_unit_test

In response to this:

/test tiprow_fast_test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ti-chi-bot ti-chi-bot bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 11, 2023
@hawkingrei
update
43ab4ba 
Signed-off-by: Weizhen Wang <[email protected]>
@hawkingrei
update
1e8dff3 
Signed-off-by: Weizhen Wang <[email protected]>
@hawkingrei
update
e1a5496 
Signed-off-by: Weizhen Wang <[email protected]>
@hawkingrei
update
4764294 
Signed-off-by: Weizhen Wang <[email protected]>
@hawkingrei hawkingrei force-pushed the add_bench_for_cache branch from 10e58cf to 4764294 Compare July 12, 2023 07:21
@ti-chi-bot ti-chi-bot bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 12, 2023
@hawkingrei
update
7c24a6d 
Signed-off-by: Weizhen Wang <[email protected]>
@hawkingrei
update
7bc89af 
Signed-off-by: Weizhen Wang <[email protected]>
@hawkingrei
update
353876c 
Signed-off-by: Weizhen Wang <[email protected]>
@hawkingrei
update
a260818 
Signed-off-by: Weizhen Wang <[email protected]>
@hawkingrei hawkingrei requested review from qw4990 and removed request for qw4990 July 12, 2023 08:41
qw4990
qw4990 reviewed Jul 12, 2023
View reviewed changes
statistics/handle/cache/internal/mapcache/map_cache.go Outdated
@@ -110,6 +125,8 @@ func (m *MapCache) Values() []*statistics.Table {

// Map implements StatsCacheInner
func (m *MapCache) Map() map[int64]*statistics.Table {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method can be removed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

qw4990
qw4990 reviewed Jul 12, 2023
View reviewed changes
statistics/handle/cache/internal/mapcache/bench_test.go Outdated
wg.Add(1)
go func(i int) {
defer wg.Done()
t1 := testutil.NewMockStatisticsTable(1, 1, true, false, false)
Copy link
Contributor

@qw4990 qw4990 Jul 12, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like we don't need to create a new table object each time? Otherwise this creation operation may affect the bench result.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately, When using the CopyAndUpdate, it will use the table id of structure.
so we make this structure difference. So it impacts each benchmark, which means it has no impact.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to insert the map and not update the map. so we need to make this structure difference.

qw4990
qw4990 reviewed Jul 12, 2023
View reviewed changes
statistics/handle/cache/internal/lru/bench_test.go Outdated

const defaultSize int64 = 1000

func BenchmarkLruPut(b *testing.B) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like BenchmarkLRUXXX is duplicated with BenchmarkMapXXX.
Why not we implement these benchmark methods on StastCacheInner interface, then we can remove some duplicated code, for example:

func benchmarkStatsCacheInner(b, c) {
...
}

func Benchmark(b) {
    for c := []Cache{LRU, Map} {
        benchmarkStatsCacheInner(b, c)
    }
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@hawkingrei
update
e4757b2 
Signed-off-by: Weizhen Wang <[email protected]>
@hawkingrei
update
13a9186 
Signed-off-by: Weizhen Wang <[email protected]>
@ti-chi-bot ti-chi-bot bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 12, 2023
xuyifangreeneyes
xuyifangreeneyes reviewed Jul 12, 2023
View reviewed changes
Copy link
Contributor

@xuyifangreeneyes xuyifangreeneyes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that COW guarantees that mapcache don't need to be thread-safe. Will any race condition happen in the current master code?

@hawkingrei
update
12cfd29 
Signed-off-by: Weizhen Wang <[email protected]>
@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jul 12, 2023
@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Jul 12, 2023
@hawkingrei
Copy link
Member Author

I thought that COW guarantees that mapcache don't need to be thread-safe. Will any race condition happen in the current master code?

This is my mistake. I brought the code for the next PR into this one. Now it is reverted.

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 12, 2023
@ti-chi-bot
Copy link

ti-chi-bot bot commented Jul 12, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: qw4990, xuyifangreeneyes

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [qw4990,xuyifangreeneyes]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot
Copy link

ti-chi-bot bot commented Jul 12, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-07-12 09:43:06.640725236 +0000 UTC m=+497878.411063934: ☑️ agreed by qw4990.
  • 2023-07-12 12:03:42.881204166 +0000 UTC m=+506314.651542863: ☑️ agreed by xuyifangreeneyes.

@ti-chi-bot ti-chi-bot bot merged commit ce0e9ed into pingcap:master Jul 12, 2023
@hawkingrei hawkingrei deleted the add_bench_for_cache branch July 12, 2023 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xuyifangreeneyes xuyifangreeneyes xuyifangreeneyes approved these changes

@qw4990 qw4990 qw4990 approved these changes

Assignees
No one assigned
Labels
approved lgtm release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@hawkingrei @qw4990 @xuyifangreeneyes