Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add hash join v2 to make hash join more efficient #53127

Open
windtalker opened this issue May 9, 2024 · 0 comments
Open

Add hash join v2 to make hash join more efficient #53127

windtalker opened this issue May 9, 2024 · 0 comments
Labels
type/enhancement The issue or PR belongs to an enhancement.

Comments

@windtalker
Copy link
Contributor

Enhancement

Our current hash join's implementation is relatively slow, this issue aims at adding a new hash join implementation. The new hash join implementation will at least

  • support concurrent build
  • simplify probe code, and make join probe more efficient
  • support use left side to build for semi join
@windtalker windtalker added the type/enhancement The issue or PR belongs to an enhancement. label May 9, 2024
This was referenced May 9, 2024
Closed
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue May 21, 2024
@windtalker
doc: add Hash join v2 design doc (#53196)
5f0c7f6 
ref #53127
RidRisR pushed a commit to RidRisR/tidb that referenced this issue May 23, 2024
@windtalker @RidRisR
doc: add Hash join v2 design doc (pingcap#53196)
0950f7e 
ref pingcap#53127
ti-chi-bot bot pushed a commit that referenced this issue Jun 14, 2024
@windtalker
executor: add hash join v2 (#53208)
152db14 
ref #53127
@windtalker windtalker mentioned this issue Jun 18, 2024
Merged
13 tasks
@windtalker windtalker mentioned this issue Jun 28, 2024
Merged
13 tasks
@wshwsh12 wshwsh12 mentioned this issue Jul 3, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jul 3, 2024
@windtalker
executor: Do not save long-lived unsafe.Pointer in hash join v2 (#54085)
7192413 
ref #53127
@windtalker windtalker mentioned this issue Jul 4, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jul 13, 2024
@wshwsh12
executor: support right outer join for hash join v2 (#54405)
1ce4236 
ref #53127
ti-chi-bot bot pushed a commit that referenced this issue Jul 18, 2024
@windtalker
executor: some minor refine for hash join v2 build (#54429)
e858f55 
ref #53127
@windtalker windtalker mentioned this issue Jul 19, 2024
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jul 19, 2024
@windtalker
executor: Refine partition number in hash join v2 (#54304)
267e1c4 
ref #53127
ti-chi-bot bot pushed a commit that referenced this issue Jul 22, 2024
@windtalker
executor: enable hash join v2 for temporary test (#54748)
5fe9aca 
ref #53127
@xhebox xhebox mentioned this issue Jul 30, 2024
Closed
@windtalker windtalker mentioned this issue Aug 2, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Aug 3, 2024
@windtalker
executor: Avoid too many concurrent atomic update in probe collision (#…
7e4608f 
…55165)

ref #53127
@windtalker windtalker mentioned this issue Aug 6, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Aug 6, 2024
@windtalker
executor: add heap object can move check for hash join v2 (#55200)
0e43ba0 
ref #53127
@windtalker windtalker mentioned this issue Aug 6, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Aug 6, 2024
@windtalker
executor: disable hash join v2 (#55224)
1cd11bc 
ref #53127
@windtalker windtalker mentioned this issue Aug 16, 2024
Merged
13 tasks
windtalker added a commit to windtalker/tidb that referenced this issue Sep 3, 2024
@windtalker
executor: enable hash join v2 for temporary test (pingcap#54748)
005ef57 
ref pingcap#53127
@windtalker windtalker mentioned this issue Sep 3, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Sep 3, 2024
@windtalker
executor: enable hash join v2 for temporary test (#55821)
2b54db5 
ref #53127
ti-chi-bot bot pushed a commit that referenced this issue Sep 4, 2024
@windtalker
executor: introduce tagged pointer in hash join v2 (#55470)
5ffe4b1 
ref #53127
@windtalker windtalker mentioned this issue Sep 4, 2024
Merged
13 tasks
@windtalker windtalker mentioned this issue Sep 5, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Sep 6, 2024
@windtalker
executor: some code refine of hash join v2 (#55887)
181c37a 
ref #53127
@windtalker windtalker mentioned this issue Sep 11, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Sep 24, 2024
@windtalker
executor: Add session variable for hash join v2 (#56023)
ad68216 
ref #53127
@windtalker windtalker mentioned this issue Sep 25, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Sep 25, 2024
@windtalker
executor: clear probe state after probe is done (#56274)
b7531c2 
ref #53127
@windtalker windtalker mentioned this issue Oct 23, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Oct 23, 2024
@windtalker
executor: enable hash join v2 by default for tests (#56790)
64ecfa1 
ref #53127
This was referenced Oct 28, 2024
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue Oct 31, 2024
@windtalker
executor: minor refine of hash join v2 (#57023)
b314a9c 
ref #53127
@wshwsh12 wshwsh12 mentioned this issue Nov 1, 2024
Merged
13 tasks
@windtalker windtalker mentioned this issue Nov 5, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Nov 5, 2024
@windtalker
exeuctor: add a helper variable to help ga hash join v2 step by step …
8a62d5a 
…for all kinds of join types (#57116)

ref #53127
ti-chi-bot bot pushed a commit that referenced this issue Nov 13, 2024
@wshwsh12
executor: support left outer semi join for hash join v2 (#57053)
8832684 
ref #53127
This was referenced Nov 18, 2024
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue Nov 18, 2024
@windtalker
executor: fix data race in nullmap in hash join v2 (#57451)
043f833 
ref #53127, close #57450
you06 pushed a commit to you06/tidb that referenced this issue Nov 18, 2024
@wshwsh12 @you06
executor: support left outer semi join for hash join v2 (pingcap#57053)
50294bb 
ref pingcap#53127

handle key-too-large error from MemBuffer

Signed-off-by: you06 <[email protected]>

test MemBuffer's oversize error to tidb error

Signed-off-by: you06 <[email protected]>

update errdoc

Signed-off-by: you06 <[email protected]>
@ti-chi-bot ti-chi-bot mentioned this issue Nov 18, 2024
Merged
13 tasks
@windtalker windtalker mentioned this issue Nov 18, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Nov 18, 2024
@ti-chi-bot
executor: fix data race in nullmap in hash join v2 (#57451) (#57456)
4cdd552 
ref #53127, close #57450
ti-chi-bot bot pushed a commit that referenced this issue Nov 19, 2024
@windtalker
executor: disable hash join v2 in release-8.5 by default (#57443)
faf322c 
ref #53127, close #57444
ti-chi-bot bot pushed a commit that referenced this issue Nov 19, 2024
@windtalker
executor: Use uint32 as element length in hash join v2 (#57458)
ac90676 
ref #53127
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this issue Nov 20, 2024
@wshwsh12 @ti-chi-bot
executor: support left outer semi join for hash join v2 (pingcap#57053)
8cf9fcc 
ref pingcap#53127

handle key-too-large error from MemBuffer

Signed-off-by: you06 <[email protected]>

test MemBuffer's oversize error to tidb error

Signed-off-by: you06 <[email protected]>

update errdoc

Signed-off-by: you06 <[email protected]>
@wshwsh12 wshwsh12 mentioned this issue Dec 23, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Dec 27, 2024
@wshwsh12
executor: support left outer anti semi join for hash join v2 (#58479)
3c40731 
ref #53127
@hawkingrei hawkingrei mentioned this issue Jan 2, 2025
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jan 2, 2025
@hawkingrei
planner: support left outer anti semi join for hash join v2 (#58648)
c44e991 
ref #53127
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@windtalker