Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

planner: fix the inappropriate heuristic rule to estimate the EQ selectivity when out of range (#18543) #18997

Merged
merged 4 commits into from
Sep 1, 2020
Merged

planner: fix the inappropriate heuristic rule to estimate the EQ selectivity when out of range (#18543) #18997

merged 4 commits into from
Sep 1, 2020

Conversation

ti-srebot
Copy link
Contributor

cherry-pick #18543 to release-4.0

What problem does this PR solve?

Issue Number: close #18461

Problem Summary: If the estimated value is out of range, an inappropriate heuristic rule sel = 1/NDV*(modifyRows/totalRows) is used, which may cause unexpected low sel when a few rows are modified.

What is changed and how it works?

Change this rule to:

func outOfRangeEQSelectivity(ndv, modifyRows, totalRows int64) float64 {
	if modifyRows == 0 {
		return 0 // it must be 0 since the histogram contains the whole data
	}
	if ndv < outOfRangeBetweenRate {
		ndv = outOfRangeBetweenRate // avoid inaccurate selectivity caused by small NDV
	}
	selectivity := 1 / float64(ndv) // TODO: After extracting TopN from histograms, we can minus the TopN fraction here.
	if selectivity*float64(totalRows) > float64(modifyRows) {
		selectivity = float64(modifyRows) / float64(totalRows)
	}
	return selectivity
}

Check List

Tests

  • Unit test

Release note

  • planner: fix the inappropriate heuristic rule to estimate the EQ selectivity when out of range

@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot ti-srebot mentioned this pull request Aug 5, 2020
Merged
@ti-srebot ti-srebot added component/statistics sig/planner SIG: Planner type/4.0-cherry-pick type/enhancement The issue or PR belongs to an enhancement. labels Aug 5, 2020
@ti-srebot ti-srebot added this to the v4.0.6 milestone Aug 5, 2020
SunRunAway
SunRunAway reviewed Aug 8, 2020
View reviewed changes
Copy link
Contributor

@SunRunAway SunRunAway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 8, 2020
zz-jason
zz-jason approved these changes Sep 1, 2020
View reviewed changes
Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Sep 1, 2020
@zz-jason
Copy link
Member

zz-jason commented Sep 1, 2020

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Sep 1, 2020
@ti-srebot
Copy link
Contributor Author

Your auto merge job has been accepted, waiting for:

  • 18801
  • 19445
  • 19518
  • 19452
  • 19439
  • 19465
  • 19628
  • 19611
  • 19577
  • 19559
  • 19614
  • 19546
  • 19049

@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot
Copy link
Contributor Author

@ti-srebot merge failed.

@zz-jason
Copy link
Member

zz-jason commented Sep 1, 2020

/merge

@ti-srebot
Copy link
Contributor Author

Your auto merge job has been accepted, waiting for:

  • 18696
  • 18760
  • 19215
  • 19454
  • 18801
  • 19445
  • 19452
  • 19439
  • 19465
  • 19611
  • 19577
  • 19559
  • 19614
  • 19049

@zz-jason
Copy link
Member

zz-jason commented Sep 1, 2020

/merge

@ti-srebot
Copy link
Contributor Author

Your auto merge job has been accepted, waiting for:

  • 19465
  • 19611
  • 19577
  • 19559
  • 19614
  • 19049
  • 18696
  • 18760
  • 19215
  • 19454
  • 18801
  • 19445
  • 19452
  • 19465
  • 19601
  • 19611
  • 19577
  • 19559
  • 19614
  • 19049

@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot ti-srebot merged commit 83fc2d8 into pingcap:release-4.0 Sep 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SunRunAway SunRunAway SunRunAway left review comments

@zz-jason zz-jason zz-jason approved these changes

@eurekaka eurekaka Awaiting requested review from eurekaka

@winoros winoros Awaiting requested review from winoros

Assignees

@qw4990 qw4990

Labels
component/statistics sig/planner SIG: Planner status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/enhancement The issue or PR belongs to an enhancement. type/4.0-cherry-pick
Projects
None yet
Milestone
v4.0.6
Development

Successfully merging this pull request may close these issues.
4 participants
@ti-srebot @zz-jason @SunRunAway @qw4990