Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GLUTEN-6067] [WIP] open spark 35 ut #8555

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

[GLUTEN-6067] [WIP] open spark 35 ut #8555

wants to merge 8 commits into from
+2,030 −1,514

Conversation

baibaichen
Copy link
Contributor

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

(Fixes: #6067)

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

@github-actions github-actions bot added the CORE works for Gluten Core label Jan 17, 2025
@github-actions GitHub Actions
Copy link

#6067

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI on x86

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 1, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 1, 2025

Run Gluten Clickhouse CI on x86

This was referenced Feb 3, 2025
Open
Merged
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 3, 2025

Run Gluten Clickhouse CI on x86

2 similar comments
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 3, 2025

Run Gluten Clickhouse CI on x86

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 3, 2025

Run Gluten Clickhouse CI on x86

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 4, 2025

Run Gluten Clickhouse CI on x86

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 5, 2025

Run Gluten Clickhouse CI on x86

@baibaichen
Copy link
Contributor Author

@CodiumAI-Agent /review

@CodiumAI-Agent
Copy link

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

6067 - Partially compliant

Compliant requirements:

  • Upgrade Spark version to 3.5 and ensure compilation passes.
  • Upgrade Delta version to 3.2 and ensure compilation passes.
  • Support Scala 2.13.

Non-compliant requirements:

  • Support native write.
  • Ensure CH backend unit tests pass (currently only MergeTree + Delta UT passed).
  • Address specific test failures and fallback scenarios.

Requires further human verification:

  • Ensure CH backend unit tests pass (currently only MergeTree + Delta UT passed).
  • Address specific test failures and fallback scenarios.
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Possible Issue

The enableSuite method no longer throws an exception for duplicate suite names, which could lead to unexpected behavior or silent overwrites. This change should be reviewed for correctness and potential unintended consequences.

if (!enabledSuites.containsKey(suiteName)) {
  val suiteSettings = new SuiteSettings
  enabledSuites.put(suiteName, suiteSettings)
  enabledSuites.get(suiteName)
}
enabledSuites.get(suiteName)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CORE works for Gluten Core
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[CH] Support CH backend with Spark 3.5.x
2 participants
@baibaichen @CodiumAI-Agent