Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Enable IntermediateHashAggregateRule #4354

Closed
zhztheplayer opened this issue Jan 4, 2024 · 3 comments
Closed

[VL] Enable IntermediateHashAggregateRule #4354

zhztheplayer opened this issue Jan 4, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@zhztheplayer
Copy link
Member

zhztheplayer commented Jan 4, 2024

Description

After oap-project/velox#468 is merged, the next thing is to enable IntermediateHashAggregateRule for pre-exchange aggregations that can be converted to Velox's "Partial" aggregation which leverages streaming aggregation optimizations like flushing and abandoning.

  • Flushing: Emit intermediate data frequently from mapper-side aggregation
  • Abandoning: When data reduction ratio is low from mapper-side aggregation, skip mapper-side aggregation permanently and flush input data immediately
@gaoyangxiaozhu
Copy link
Contributor

gaoyangxiaozhu commented Jan 10, 2024

does @zhztheplayer this issue been fixed for Q17/Q18's regression?

@zhztheplayer
Copy link
Member Author

@gaoyangxiaozhu The regression was fixed in #4312, I assume the issue was opened in wrong repo...

@zhztheplayer zhztheplayer transferred this issue from oap-project/velox Jan 10, 2024
@zhztheplayer
Copy link
Member Author

Closed in #4312

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants