Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GLUTEN-8616] [VL] Make filescan limit for encrypted fallback as configurable #8621

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ArnavBalyan
Copy link
Contributor

What changes were proposed in this pull request?

  • Making parquet file scan limit as configurable.
  • Currently it's hardcoded to default 10, now available as a user config can be changed externally.
  • Default remains the same as before of 10.

How was this patch tested?

  • Existing UTs

@github-actions github-actions bot added CORE works for Gluten Core VELOX labels Jan 26, 2025
Copy link

#8616

Copy link

Run Gluten Clickhouse CI on x86

@ArnavBalyan ArnavBalyan force-pushed the arnavb/configurable-limit branch from acbcc3f to fc66b86 Compare January 26, 2025 09:48
Copy link

Run Gluten Clickhouse CI on x86

@ArnavBalyan ArnavBalyan force-pushed the arnavb/configurable-limit branch from fc66b86 to ce47d23 Compare January 26, 2025 12:06
Copy link

Run Gluten Clickhouse CI on x86

@ArnavBalyan
Copy link
Contributor Author

cc @zhztheplayer could you please review thanks!

.doc("If supplied, `limit` number of files will be checked to determine encryption " +
"and falling back java scan")
.intConf
.createWithDefault(10)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: checkValue is required here to ensure the data is non-negative.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@ArnavBalyan ArnavBalyan force-pushed the arnavb/configurable-limit branch from ce47d23 to 9a27d24 Compare January 28, 2025 17:02
Copy link

Run Gluten Clickhouse CI on x86

@ArnavBalyan
Copy link
Contributor Author

cc @jackylee-ch fixed, can you please take a look

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CORE works for Gluten Core VELOX
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants