You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Relates to #10440. There should be regular generation of tools support CSV files for supported Databricks platforms.
Steps:
Update tools/pom.xml so it can generate a proper classpath to run the tools generator code on Databricks. The only thing tricky here is getting the classpath right, because in Maven it doesn't really understand how to build a full classpath to run Spark, just enough to get compiles against it. Honestly it might be simplest to hack the classpath so we pick up Databricks jars via -cp /databricks/jars/* when running this.
Update jenkins/databricks/build.sh so it builds the verify target rather than the package target (so it generates the tools files)
Modify the databricks premerge and nightly CI so builds will fail if tools files are found to be checked out after a build. Likely need to loop in CI/CD team for this.
The text was updated successfully, but these errors were encountered:
cc @sameerz, @GaryShen2008. Based on discussion with the team, this would need collaboration between the Build team and the Tools team.
The Tools team (?) would need to update the generator script to work on Databricks environments. The Build team could then invoke that during mvn verify to generate the supportedOps csv files, for the build artifact.
I don't know of the priority to include this in 25.02.
Relates to #10440. There should be regular generation of tools support CSV files for supported Databricks platforms.
Steps:
The text was updated successfully, but these errors were encountered: