Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: disable probe short path in hash join #53079

Merged

Conversation

windtalker
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #51998

Problem Summary:
In TiDB's hash join, there is a short path in probe side:

if probeSideResult.NumRows() == 0 && !fetcher.UseOuterToBuild {
fetcher.finished.Store(true)
}

If the probe side is empty, and current join use inner side to build, then the join can mark to finished without waiting the build side. This short-path can cause random fail for the case that probe side is empty, but build side may meet error during execution. In this case, there is a data race between build side and probe side:

  • if probe side is detected to be empty before build side meet error, the hash join will return empty result without throwing error
  • if probe side is detected to be empty after build side meet error, then hash join will throw error

This will cause random fail

What changed and how does it work?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-triage-completed release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels May 7, 2024
Copy link

codecov bot commented May 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.7272%. Comparing base (87247c1) to head (8183d72).
Report is 4 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #53079        +/-   ##
================================================
+ Coverage   72.4348%   74.7272%   +2.2923%     
================================================
  Files          1492       1514        +22     
  Lines        428995     438504      +9509     
================================================
+ Hits         310742     327682     +16940     
+ Misses        98991      90324      -8667     
- Partials      19262      20498      +1236     
Flag Coverage Δ
integration 50.9968% <0.0000%> (?)
unit 71.4007% <100.0000%> (+0.0762%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 53.9957% <ø> (ø)
parser ∅ <ø> (∅)
br 50.4952% <ø> (+9.0516%) ⬆️

XuHuaiyu
XuHuaiyu previously approved these changes May 9, 2024
@ti-chi-bot ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels May 9, 2024
@XuHuaiyu XuHuaiyu dismissed their stale review May 9, 2024 07:44

mysql has different behavior

@ti-chi-bot ti-chi-bot bot removed the approved label May 9, 2024
@windtalker windtalker force-pushed the disable_probe_short_path_in_hashjoin branch from be269ea to 1829f3d Compare May 10, 2024 01:27
@ti-chi-bot ti-chi-bot bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels May 10, 2024
Copy link

ti-chi-bot bot commented May 10, 2024

@XuHuaiyu: Your lgtm message is repeated, so it is ignored.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ti-chi-bot ti-chi-bot bot added the approved label May 10, 2024
Copy link
Contributor

@yibin87 yibin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

ti-chi-bot bot commented May 10, 2024

@yibin87: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

LGTM

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@windtalker
Copy link
Contributor Author

/test check-dev2

Copy link

tiprow bot commented May 10, 2024

@windtalker: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot ti-chi-bot bot added the lgtm label May 10, 2024
Copy link

ti-chi-bot bot commented May 10, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: guo-shaoge, XuHuaiyu, yibin87

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [XuHuaiyu,guo-shaoge]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot removed the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label May 10, 2024
Copy link

ti-chi-bot bot commented May 10, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-05-09 07:16:04.313300222 +0000 UTC m=+1119118.070435795: ☑️ agreed by XuHuaiyu.
  • 2024-05-10 03:19:42.908991361 +0000 UTC m=+1191336.666126934: ☑️ agreed by guo-shaoge.

@windtalker
Copy link
Contributor Author

/test check-dev2

Copy link

tiprow bot commented May 10, 2024

@windtalker: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@windtalker
Copy link
Contributor Author

/test check-dev2

Copy link

tiprow bot commented May 10, 2024

@windtalker: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@windtalker
Copy link
Contributor Author

/test check-dev2

@windtalker
Copy link
Contributor Author

/test check-dev2

Copy link

tiprow bot commented May 10, 2024

@windtalker: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@windtalker
Copy link
Contributor Author

/test check-dev2

Copy link

tiprow bot commented May 10, 2024

@windtalker: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@windtalker
Copy link
Contributor Author

/test check-dev2

Copy link

tiprow bot commented May 10, 2024

@windtalker: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@windtalker
Copy link
Contributor Author

/test check-dev2

Copy link

tiprow bot commented May 10, 2024

@windtalker: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@windtalker
Copy link
Contributor Author

/test check-dev2

Copy link

tiprow bot commented May 10, 2024

@windtalker: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@windtalker
Copy link
Contributor Author

/test check-dev2

Copy link

tiprow bot commented May 10, 2024

@windtalker: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@windtalker
Copy link
Contributor Author

/test pull-integration-ddl-test

@windtalker
Copy link
Contributor Author

/test unit-test

Copy link

tiprow bot commented May 10, 2024

@windtalker: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test pull-integration-ddl-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copy link

tiprow bot commented May 10, 2024

@windtalker: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@windtalker
Copy link
Contributor Author

/test check-dev2

Copy link

tiprow bot commented May 11, 2024

@windtalker: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@windtalker
Copy link
Contributor Author

/test check-dev2

Copy link

tiprow bot commented May 11, 2024

@windtalker: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot ti-chi-bot bot merged commit 5d780eb into pingcap:master May 11, 2024
21 checks passed
@ti-chi-bot ti-chi-bot added the needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. label May 11, 2024
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-8.1: #53183.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request May 11, 2024
windtalker added a commit to ti-chi-bot/tidb that referenced this pull request May 11, 2024
terry1purcell pushed a commit to terry1purcell/tidb that referenced this pull request May 17, 2024
RidRisR pushed a commit to RidRisR/tidb that referenced this pull request May 23, 2024
@windtalker windtalker deleted the disable_probe_short_path_in_hashjoin branch July 22, 2024 02:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Random "truncate incorrect value"
5 participants