Skip to content

Commit

Permalink
Merge pull request #102 from pradeepmaripala/spark-3.2
Browse files Browse the repository at this point in the history
readme changes for 3.2
  • Loading branch information
badrinathpatchikolla authored Oct 5, 2023
2 parents e485547 + 1751ce6 commit 8a87442
Show file tree
Hide file tree
Showing 3 changed files with 19 additions and 17 deletions.
26 changes: 14 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,30 +60,32 @@ The Almaren Framework provides a simplified consistent minimalistic layer over A
To add Almaren Framework dependency to your sbt build:

```
libraryDependencies += "com.github.music-of-the-ainur" %% "almaren-framework" % "0.9.10-3.2"
libraryDependencies += "com.github.music-of-the-ainur" %% "almaren-framework" % "0.9.11-3.2"
```

To run in spark-shell:

```
spark-shell --packages "com.github.music-of-the-ainur:almaren-framework_2.12:0.9.10-3.2"
spark-shell --packages "com.github.music-of-the-ainur:almaren-framework_2.12:0.9.11-3.2"
```


Almaren connector is available in
[Maven Central](https://mvnrepository.com/artifact/com.github.music-of-the-ainur)
repository.

| version | Connector Artifact |
|----------------------------|------------------------------------------------------------------|
| Spark 3.4.x and scala 2.13 | `com.github.music-of-the-ainur:almaren-framework_2.13:0.9.10-3.4` |
| Spark 3.4.x and scala 2.12 | `com.github.music-of-the-ainur:almaren-framework_2.12:0.9.10-3.4` |
| Spark 3.3.x and scala 2.13 | `com.github.music-of-the-ainur:almaren-framework_2.13:0.9.10-3.3` |
| Spark 3.3.x and scala 2.12 | `com.github.music-of-the-ainur:almaren-framework_2.12:0.9.10-3.3` |
| Spark 3.2.x and scala 2.12 | `com.github.music-of-the-ainur:almaren-framework_2.12:0.9.10-3.2` |
| Spark 3.1.x and scala 2.12 | `com.github.music-of-the-ainur:almaren-framework_2.12:0.9.10-3.1` |
| Spark 2.4.x and scala 2.12 | `com.github.music-of-the-ainur:almaren-framework_2.12:0.9.10-2.4` |
| Spark 2.4.x and scala 2.11 | `com.github.music-of-the-ainur:almaren-framework_2.11:0.9.10-2.4` |
| version | Connector Artifact |
|----------------------------|-------------------------------------------------------------------|
| Spark 3.5.x and scala 2.13 | `com.github.music-of-the-ainur:almaren-framework_2.13:0.9.11-3.5` |
| Spark 3.5.x and scala 2.12 | `com.github.music-of-the-ainur:almaren-framework_2.12:0.9.11-3.5` |
| Spark 3.4.x and scala 2.13 | `com.github.music-of-the-ainur:almaren-framework_2.13:0.9.11-3.4` |
| Spark 3.4.x and scala 2.12 | `com.github.music-of-the-ainur:almaren-framework_2.12:0.9.11-3.4` |
| Spark 3.3.x and scala 2.13 | `com.github.music-of-the-ainur:almaren-framework_2.13:0.9.11-3.3` |
| Spark 3.3.x and scala 2.12 | `com.github.music-of-the-ainur:almaren-framework_2.12:0.9.11-3.3` |
| Spark 3.2.x and scala 2.12 | `com.github.music-of-the-ainur:almaren-framework_2.12:0.9.11-3.2` |
| Spark 3.1.x and scala 2.12 | `com.github.music-of-the-ainur:almaren-framework_2.12:0.9.11-3.1` |
| Spark 2.4.x and scala 2.12 | `com.github.music-of-the-ainur:almaren-framework_2.12:0.9.11-2.4` |
| Spark 2.4.x and scala 2.11 | `com.github.music-of-the-ainur:almaren-framework_2.11:0.9.11-2.4` |

### Batch Example
```scala
Expand Down
8 changes: 4 additions & 4 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ lazy val scala212 = "2.12.10"
crossScalaVersions := Seq(scala212)
ThisBuild / scalaVersion := scala212

val sparkVersion = "3.2.1"
val sparkVersion = "3.2.4"
val majorVersionReg = "([0-9]+\\.[0-9]+).{0,}".r

val majorVersionReg(majorVersion) = sparkVersion
Expand All @@ -18,10 +18,10 @@ libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-sql" % sparkVersion % "provided",
"org.apache.spark" %% "spark-sql-kafka-0-10" % sparkVersion % "provided",
"org.apache.spark" %% "spark-avro" % sparkVersion,
"com.databricks" %% "spark-xml" % "0.14.0",
"com.databricks" %% "spark-xml" % "0.17.0",
"com.github.music-of-the-ainur" %% "quenya-dsl" % s"1.2.3-${majorVersion}",
"org.scalatest" %% "scalatest" % "3.2.14" % "test",
"org.postgresql" % "postgresql" % "42.2.8" % "test"
"org.scalatest" %% "scalatest" % "3.2.17" % "test",
"org.postgresql" % "postgresql" % "42.6.0" % "test"
)

enablePlugins(GitVersioning)
Expand Down
2 changes: 1 addition & 1 deletion project/build.properties
Original file line number Diff line number Diff line change
@@ -1 +1 @@
sbt.version=1.7.1
sbt.version=1.9.6

0 comments on commit 8a87442

Please sign in to comment.