Skip to content

Commit

Permalink
Merge pull request #15 from rohithvangapalli/spark-3.4
Browse files Browse the repository at this point in the history
Updated the spark and almaren versions
  • Loading branch information
badrinathpatchikolla authored Sep 21, 2023
2 parents 5254b0a + 03998c2 commit 85d45ea
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 11 deletions.
20 changes: 11 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
To add neo4j dependency to your sbt build:

```
libraryDependencies += "com.github.music-of-the-ainur" %% "neo4j-almaren" % "0.1.3-3.3"
libraryDependencies += "com.github.music-of-the-ainur" %% "neo4j-almaren" % "0.1.4-3.4"
```

Neo4j Connector was implemented using [https://github.com/neo4j-contrib/neo4j-spark-connector](https://github.com/neo4j-contrib/neo4j-spark-connector).
Expand All @@ -14,11 +14,11 @@ For more details check the following [link](https://github.com/neo4j-contrib/neo
To run in spark-shell:
For 2.12:
```
spark-shell --master "local[*]" --packages "com.github.music-of-the-ainur:almaren-framework_2.12:0.9.8-3.3,com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-3.3"
spark-shell --master "local[*]" --packages "com.github.music-of-the-ainur:almaren-framework_2.12:0.9.10-3.4,com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.4-3.4"
```
For 2.13:
```
spark-shell --master "local[*]" --packages "com.github.music-of-the-ainur:almaren-framework_2.13:0.9.8-3.3,com.github.music-of-the-ainur:neo4j-almaren_2.13:0.1.3-3.3"
spark-shell --master "local[*]" --packages "com.github.music-of-the-ainur:almaren-framework_2.13:0.9.10-3.4,com.github.music-of-the-ainur:neo4j-almaren_2.13:0.1.4-3.4"
```

### Connector Usage
Expand All @@ -31,12 +31,14 @@ repository. It can be used using the `--packages` option or the

| version | Connector Artifact |
|----------------------------|--------------------------------------------------------------|
| Spark 3.3.x and scala 2.13 | `com.github.music-of-the-ainur:neo4j-almaren_2.13:0.1.3-3.3` |
| Spark 3.3.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-3.3` |
| Spark 3.2.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-3.2` |
| Spark 3.1.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-3.1` |
| Spark 2.4.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.3-2.4` |
| Spark 2.4.x and scala 2.11 | `com.github.music-of-the-ainur:neo4j-almaren_2.11:0.1.3-2.4` |
| Spark 3.4.x and scala 2.13 | `com.github.music-of-the-ainur:neo4j-almaren_2.13:0.1.4-3.4` |
| Spark 3.4.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.4-3.4` |
| Spark 3.3.x and scala 2.13 | `com.github.music-of-the-ainur:neo4j-almaren_2.13:0.1.4-3.3` |
| Spark 3.3.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.4-3.3` |
| Spark 3.2.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.4-3.2` |
| Spark 3.1.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.4-3.1` |
| Spark 2.4.x and scala 2.12 | `com.github.music-of-the-ainur:neo4j-almaren_2.12:0.1.4-2.4` |
| Spark 2.4.x and scala 2.11 | `com.github.music-of-the-ainur:neo4j-almaren_2.11:0.1.4-2.4` |


## Source and Target
Expand Down
4 changes: 2 additions & 2 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ lazy val scala213 = "2.13.9"
crossScalaVersions := Seq(scala212,scala213)
ThisBuild / scalaVersion := scala213

val sparkVersion = "3.3.0"
val sparkVersion = "3.4.1"
val majorVersionReg = "([0-9]+\\.[0-9]+).{0,}".r

val majorVersionReg(majorVersion) = sparkVersion
Expand All @@ -17,7 +17,7 @@ scalacOptions ++= Seq("-deprecation", "-feature")
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % sparkVersion % "provided",
"org.apache.spark" %% "spark-sql" % sparkVersion % "provided",
"com.github.music-of-the-ainur" %% "almaren-framework" % s"0.9.8-${majorVersion}" % "provided",
"com.github.music-of-the-ainur" %% "almaren-framework" % s"0.9.10-${majorVersion}" % "provided",
"com.typesafe.scala-logging" %% "scala-logging" % "3.9.5",
"org.scalatest" %% "scalatest" % "3.2.14" % "test",
"org.neo4j" %% "neo4j-connector-apache-spark" % "4.1.5_for_spark_3"
Expand Down

0 comments on commit 85d45ea

Please sign in to comment.