Skip to content

Commit

Permalink
Update version to 24.12.1-SNAPSHOT
Browse files Browse the repository at this point in the history
Wait for the pre-merge CI job to SUCCEED

Signed-off-by: nvauto <[email protected]>
  • Loading branch information
nvauto committed Jan 10, 2025
1 parent b5075ed commit 7a0aa10
Show file tree
Hide file tree
Showing 66 changed files with 199 additions and 199 deletions.
8 changes: 4 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,15 +127,15 @@ mvn -pl dist -PnoSnapshots package -DskipTests
Verify that shim-specific classes are hidden from a conventional classloader.

```bash
$ javap -cp dist/target/rapids-4-spark_2.12-24.12.0-SNAPSHOT-cuda11.jar com.nvidia.spark.rapids.shims.SparkShimImpl
$ javap -cp dist/target/rapids-4-spark_2.12-24.12.1-SNAPSHOT-cuda11.jar com.nvidia.spark.rapids.shims.SparkShimImpl
Error: class not found: com.nvidia.spark.rapids.shims.SparkShimImpl
```

However, its bytecode can be loaded if prefixed with `spark3XY` not contained in the package name

```bash
$ javap -cp dist/target/rapids-4-spark_2.12-24.12.0-SNAPSHOT-cuda11.jar spark320.com.nvidia.spark.rapids.shims.SparkShimImpl | head -2
Warning: File dist/target/rapids-4-spark_2.12-24.12.0-SNAPSHOT-cuda11.jar(/spark320/com/nvidia/spark/rapids/shims/SparkShimImpl.class) does not contain class spark320.com.nvidia.spark.rapids.shims.SparkShimImpl
$ javap -cp dist/target/rapids-4-spark_2.12-24.12.1-SNAPSHOT-cuda11.jar spark320.com.nvidia.spark.rapids.shims.SparkShimImpl | head -2
Warning: File dist/target/rapids-4-spark_2.12-24.12.1-SNAPSHOT-cuda11.jar(/spark320/com/nvidia/spark/rapids/shims/SparkShimImpl.class) does not contain class spark320.com.nvidia.spark.rapids.shims.SparkShimImpl
Compiled from "SparkShims.scala"
public final class com.nvidia.spark.rapids.shims.SparkShimImpl {
```
Expand Down Expand Up @@ -178,7 +178,7 @@ mvn package -pl dist -am -Dbuildver=340 -DallowConventionalDistJar=true
Verify `com.nvidia.spark.rapids.shims.SparkShimImpl` is conventionally loadable:
```bash
$ javap -cp dist/target/rapids-4-spark_2.12-24.12.0-SNAPSHOT-cuda11.jar com.nvidia.spark.rapids.shims.SparkShimImpl | head -2
$ javap -cp dist/target/rapids-4-spark_2.12-24.12.1-SNAPSHOT-cuda11.jar com.nvidia.spark.rapids.shims.SparkShimImpl | head -2
Compiled from "SparkShims.scala"
public final class com.nvidia.spark.rapids.shims.SparkShimImpl {
```
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ as a `provided` dependency.
<dependency>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<scope>provided</scope>
</dependency>
```
6 changes: 3 additions & 3 deletions aggregator/pom.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2021-2024, NVIDIA CORPORATION.
Copyright (c) 2021-2025, NVIDIA CORPORATION.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -22,13 +22,13 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<relativePath>../jdk-profiles/pom.xml</relativePath>
</parent>
<artifactId>rapids-4-spark-aggregator_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Aggregator</name>
<description>Creates an aggregated shaded package of the RAPIDS plugin for Apache Spark</description>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>

<properties>
<rapids.module>aggregator</rapids.module>
Expand Down
6 changes: 3 additions & 3 deletions api_validation/pom.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2020-2024, NVIDIA CORPORATION.
Copyright (c) 2020-2025, NVIDIA CORPORATION.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -22,11 +22,11 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-shim-deps-parent_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<relativePath>../shim-deps/pom.xml</relativePath>
</parent>
<artifactId>rapids-4-spark-api-validation_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>

<properties>
<rapids.module>api_validation</rapids.module>
Expand Down
6 changes: 3 additions & 3 deletions datagen/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@ Where `$SPARK_VERSION` is a compressed version number, like 330 for Spark 3.3.0.

After this the jar should be at
`target/datagen_2.12-$PLUGIN_VERSION-spark$SPARK_VERSION.jar`
for example a Spark 3.3.0 jar for the 24.12.0 release would be
`target/datagen_2.12-24.12.0-spark330.jar`
for example a Spark 3.3.0 jar for the 24.12.1 release would be
`target/datagen_2.12-24.12.1-spark330.jar`

To get a spark shell with this you can run
```shell
spark-shell --jars target/datagen_2.12-24.12.0-spark330.jar
spark-shell --jars target/datagen_2.12-24.12.1-spark330.jar
```

After that you should be good to go.
Expand Down
2 changes: 1 addition & 1 deletion datagen/ScaleTest.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ $SPARK_HOME/bin/spark-submit \
--conf spark.sql.parquet.datetimeRebaseModeInWrite=CORRECTED \
--class com.nvidia.rapids.tests.scaletest.ScaleTestDataGen \ # the main class
--jars $SPARK_HOME/examples/jars/scopt_2.12-3.7.1.jar \ # one dependency jar just shipped with Spark under $SPARK_HOME
./target/datagen_2.12-24.12.0-SNAPSHOT-spark332.jar \
./target/datagen_2.12-24.12.1-SNAPSHOT-spark332.jar \
1 \
10 \
parquet \
Expand Down
6 changes: 3 additions & 3 deletions datagen/pom.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2023-2024 NVIDIA CORPORATION.
Copyright (c) 2023-2025 NVIDIA CORPORATION.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -21,13 +21,13 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-shim-deps-parent_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<relativePath>../shim-deps/pom.xml</relativePath>
</parent>
<artifactId>datagen_2.12</artifactId>
<name>Data Generator</name>
<description>Tools for generating large amounts of data</description>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<properties>
<rapids.module>datagen</rapids.module>
<target.classifier/>
Expand Down
6 changes: 3 additions & 3 deletions delta-lake/delta-20x/pom.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2023-2024 NVIDIA CORPORATION.
Copyright (c) 2023-2025 NVIDIA CORPORATION.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<relativePath>../../jdk-profiles/pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-20x_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.0.x Support</name>
<description>Delta Lake 2.0.x support for the RAPIDS Accelerator for Apache Spark</description>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>

<properties>
<rapids.module>../delta-lake/delta-20x</rapids.module>
Expand Down
6 changes: 3 additions & 3 deletions delta-lake/delta-21x/pom.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2023-2024 NVIDIA CORPORATION.
Copyright (c) 2023-2025 NVIDIA CORPORATION.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<relativePath>../../jdk-profiles/pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-21x_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.1.x Support</name>
<description>Delta Lake 2.1.x support for the RAPIDS Accelerator for Apache Spark</description>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>

<properties>
<rapids.module>../delta-lake/delta-21x</rapids.module>
Expand Down
6 changes: 3 additions & 3 deletions delta-lake/delta-22x/pom.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2023-2024 NVIDIA CORPORATION.
Copyright (c) 2023-2025 NVIDIA CORPORATION.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<relativePath>../../jdk-profiles/pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-22x_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.2.x Support</name>
<description>Delta Lake 2.2.x support for the RAPIDS Accelerator for Apache Spark</description>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>

<properties>
<rapids.module>../delta-lake/delta-22x</rapids.module>
Expand Down
6 changes: 3 additions & 3 deletions delta-lake/delta-23x/pom.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2023-2024 NVIDIA CORPORATION.
Copyright (c) 2023-2025 NVIDIA CORPORATION.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-23x_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.3.x Support</name>
<description>Delta Lake 2.3.x support for the RAPIDS Accelerator for Apache Spark</description>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>

<properties>
<rapids.module>../delta-lake/delta-23x</rapids.module>
Expand Down
6 changes: 3 additions & 3 deletions delta-lake/delta-24x/pom.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2023-2024 NVIDIA CORPORATION.
Copyright (c) 2023-2025 NVIDIA CORPORATION.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<relativePath>../../jdk-profiles/pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-24x_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.4.x Support</name>
<description>Delta Lake 2.4.x support for the RAPIDS Accelerator for Apache Spark</description>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>

<properties>
<rapids.module>../delta-lake/delta-24x</rapids.module>
Expand Down
6 changes: 3 additions & 3 deletions delta-lake/delta-spark330db/pom.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2023-2024 NVIDIA CORPORATION.
Copyright (c) 2023-2025 NVIDIA CORPORATION.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-shim-deps-parent_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<relativePath>../../shim-deps/pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-spark330db_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Databricks 11.3 Delta Lake Support</name>
<description>Databricks 11.3 Delta Lake support for the RAPIDS Accelerator for Apache Spark</description>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>

<properties>
<rapids.module>../delta-lake/delta-spark330db</rapids.module>
Expand Down
6 changes: 3 additions & 3 deletions delta-lake/delta-spark332db/pom.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2023-2024 NVIDIA CORPORATION.
Copyright (c) 2023-2025 NVIDIA CORPORATION.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-shim-deps-parent_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<relativePath>../../shim-deps/pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-spark332db_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Databricks 12.2 Delta Lake Support</name>
<description>Databricks 12.2 Delta Lake support for the RAPIDS Accelerator for Apache Spark</description>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>

<properties>
<rapids.module>../delta-lake/delta-spark332db</rapids.module>
Expand Down
6 changes: 3 additions & 3 deletions delta-lake/delta-spark341db/pom.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2023-2024 NVIDIA CORPORATION.
Copyright (c) 2023-2025 NVIDIA CORPORATION.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-shim-deps-parent_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<relativePath>../../shim-deps/pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-spark341db_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Databricks 13.3 Delta Lake Support</name>
<description>Databricks 13.3 Delta Lake support for the RAPIDS Accelerator for Apache Spark</description>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>

<properties>
<rapids.compressed.artifact>false</rapids.compressed.artifact>
Expand Down
6 changes: 3 additions & 3 deletions delta-lake/delta-spark350db143/pom.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2023-2024 NVIDIA CORPORATION.
Copyright (c) 2023-2025 NVIDIA CORPORATION.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-shim-deps-parent_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<relativePath>../../shim-deps/pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-spark350db143_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Databricks 13.3 Delta Lake Support</name>
<description>Databricks 13.3 Delta Lake support for the RAPIDS Accelerator for Apache Spark</description>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>

<properties>
<rapids.compressed.artifact>false</rapids.compressed.artifact>
Expand Down
6 changes: 3 additions & 3 deletions delta-lake/delta-stub/pom.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2023-2024 NVIDIA CORPORATION.
Copyright (c) 2023-2025 NVIDIA CORPORATION.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>
<relativePath>../../jdk-profiles/pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-stub_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Delta Lake Stub</name>
<description>Delta Lake stub for the RAPIDS Accelerator for Apache Spark</description>
<version>24.12.0-SNAPSHOT</version>
<version>24.12.1-SNAPSHOT</version>

<properties>
<rapids.module>../delta-lake/delta-stub</rapids.module>
Expand Down
Loading

0 comments on commit 7a0aa10

Please sign in to comment.