Test DataFusion 44.0.0 with Sail

[shehabgamin]

Is your feature request related to a problem or challenge?

• Part of of [DISCUSS] More extensive pre-release testing #13661

We are trying to make sure the upgrade experience for DataFusion is better and to do so we are hoping to improve the testing of major downstream projects

For version 44.0.0 we would like to test with Sail prior to release.

Describe the solution you'd like

I would like to create a PR to test DataFusion Sail with a pre-released version of DataFusion (pinned to the main branch)

Describe alternatives you've considered

No response

Additional context

lakehq/sail#335

[shehabgamin]

Take

[shehabgamin]

Smooth Sailing testing commit 5d563d9 from DataFusion main branch.

The following test reports run tests on the PR branch and compares them against the tests from the Sail main branch:

• DataFusion 44.0.0 Pre-Release Testing lakehq/sail#335 (comment)
• DataFusion 44.0.0 Pre-Release Testing lakehq/sail#335 (comment)

Will re-test once there is a release candidate.

[Omega359]

Smooth Sailing testing commit 5d563d9 from DataFusion main branch.

The following test reports run tests on the PR branch and compares them against the tests from the Sail main branch:

* [DataFusion 44.0.0 Pre-Release Testing lakehq/sail#335 (comment)](https://github.com/lakehq/sail/pull/335#issuecomment-2556353768)

* [DataFusion 44.0.0 Pre-Release Testing lakehq/sail#335 (comment)](https://github.com/lakehq/sail/pull/335#issuecomment-2556416669)

After a brief review of the errors I suspect there may be something up with lists and structs but I would have to see the actual logic being tested to know for certain. Many of the errors are expected - functions not existing, etc.

[alamb]

After a brief review of the errors I suspect there may be something up with lists and structs but I would have to see the actual logic being tested to know for certain. Many of the errors are expected - functions not existing, etc.

I feel like we have had several bugs / issues with structs / lists the last few releases so it would be amazing if we could figure out what is going on before the release. @shehabgamin is this something you can help with?

[shehabgamin]

After a brief review of the errors I suspect there may be something up with lists and structs but I would have to see the actual logic being tested to know for certain. Many of the errors are expected - functions not existing, etc.

I feel like we have had several bugs / issues with structs / lists the last few releases so it would be amazing if we could figure out what is going on before the release. @shehabgamin is this something you can help with?

@Omega359 @alamb For sure, I will gladly take a look! If there are issues with lists and/or structs, it likely wouldn't be due to changes in the 44.0.0 pre-release that I tested, though. The Sail main branch and the PR branch that is testing the pre-release have the exact same errors (with the exception of slight rewording of some error logs).

This leads me to believe that any issues with lists and/or structs would likely be preexisting. I will take a close look at all of the errors, but, with that in mind, should we scope the issues into a separate GitHub issue if we do find errors? Also @Omega359, let me know if any errors in particular caught your eye.

[alamb]

This leads me to believe that any issues with lists and/or structs would likely be preexisting. I will take a close look at all of the errors, but, with that in mind, should we scope the issues into a separate GitHub issue if we do find errors? Also @Omega359, let me know if any errors in particular caught your eye.

That would make sense. Glad to hear the errors aren't new