Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] JSON predicate pushdown for Pinot #23966

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

robertzych
Copy link

@robertzych robertzych commented Oct 30, 2024

Description

initial version with PinotJsonContainsPredicate that converts contains(array, json_extract_scalar(jsonColumn, '$.key')) to json_match(jsonColumn, '$.key in array')

Limitations:

  1. arrays of type string or int

PinotJsonArrayContainsPredicate converts json_array_contains(json_extract(jsonColumn, ‘$.array’), value) to json_match(jsonColumn, ‘$.array[*] = value’)

TODO: Allow multiple predicates to be ORed
TODO: Feature flag that defaults to off since json_match will fail if JSON index hasn't been configured in Pinot.
TODO: tests

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

Copy link

cla-bot bot commented Oct 30, 2024

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Robert Zych.
This is most likely caused by a git client misconfiguration; please make sure to:

  1. check if your git client is configured with an email to sign commits git config --list | grep email
  2. If not, set it up using git config --global user.email [email protected]
  3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

@robertzych
Copy link
Author

@cla-bot check

Copy link

cla-bot bot commented Oct 30, 2024

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

Copy link

cla-bot bot commented Oct 30, 2024

The cla-bot has been summoned, and re-checked this pull request!

@robertzych
Copy link
Author

I just signed and submitted the individual CLA.

Copy link

cla-bot bot commented Oct 31, 2024

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

resolved style and deferred dependency issue
Copy link

cla-bot bot commented Oct 31, 2024

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@robertzych robertzych changed the title JSON predicate pushdown for Pinot (draft) [WIP] JSON predicate pushdown for Pinot Oct 31, 2024
Copy link

cla-bot bot commented Nov 1, 2024

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

Copy link

cla-bot bot commented Nov 1, 2024

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

1 participant