Cherry pick push down join key filters for LEFT/RIGHT/ANTI joins#32
Open
trevor-wilson-polygonio wants to merge 1 commit intobranch-51from
Open
Cherry pick push down join key filters for LEFT/RIGHT/ANTI joins#32trevor-wilson-polygonio wants to merge 1 commit intobranch-51from
trevor-wilson-polygonio wants to merge 1 commit intobranch-51from
Conversation
) - Closes apache#19917. Reduce the number of rows retrieved by pushing down more filters when possible. Example: ```sql create table t1 (k int, v int); create table t2 (k int, v int); -- k=1 is pushed to t1 and t2 explain select * from t1 left join t2 on t1.k = t2.k where t1.k = 1; +---------------+------------------------------------------------------------+ | plan_type | plan | +---------------+------------------------------------------------------------+ | physical_plan | ┌───────────────────────────┐ | | | │ HashJoinExec │ | | | │ -------------------- │ | | | │ join_type: Left ├──────────────┐ | | | │ on: (k = k) │ │ | | | └─────────────┬─────────────┘ │ | | | ┌─────────────┴─────────────┐┌─────────────┴─────────────┐ | | | │ RepartitionExec ││ RepartitionExec │ | | | │ -------------------- ││ -------------------- │ | | | │ partition_count(in->out): ││ partition_count(in->out): │ | | | │ 1 -> 12 ││ 1 -> 12 │ | | | │ ││ │ | | | │ partitioning_scheme: ││ partitioning_scheme: │ | | | │ Hash([k@0], 12) ││ Hash([k@0], 12) │ | | | └─────────────┬─────────────┘└─────────────┬─────────────┘ | | | ┌─────────────┴─────────────┐┌─────────────┴─────────────┐ | | | │ FilterExec ││ FilterExec │ | | | │ -------------------- ││ -------------------- │ | | | │ predicate: k = 1 ││ predicate: k = 1 │ | | | └─────────────┬─────────────┘└─────────────┬─────────────┘ | | | ┌─────────────┴─────────────┐┌─────────────┴─────────────┐ | | | │ DataSourceExec ││ DataSourceExec │ | | | │ -------------------- ││ -------------------- │ | | | │ bytes: 0 ││ bytes: 0 │ | | | │ format: memory ││ format: memory │ | | | │ rows: 0 ││ rows: 0 │ | | | └───────────────────────────┘└───────────────────────────┘ | | | | +---------------+------------------------------------------------------------+ ``` - Changed `push_down_all_join` to push down inferred predicates independently of `left_preserved`/`right_preserved` semantics. - Added unit tests. Yes. No. --------- Co-authored-by: xudong.w <wxd963996380@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry pick apache#19918