Running sparklint on Databricks

[damanp]

Hi,
So I've just started using Databricks and I'm pretty stoked about being able to use sparklint and checking on my jobs.

I currently upload my jars to Databricks and then invoke the methods that I am interested in.

I tried including sparklint from the sbt file as mentioned, but the compilation always fails.
(Let me know if you would want to see the logs in detail. I will upload them if you would like to)

I then compiled only the sparklint code and uploaded it as a separate jar, attached it to my cluster, and in the other jar that holds my code, I added the lines:
spark.conf.set("spark.extraListeners", "com.groupon.sparklint.SparklintListener") spark.conf.set("spark.sparklint.port", "4242")

However, I am unable to access the port or see any sort of indication in the logs that sparklint has been active during the job execution.

Can you please help me get this going on Databricks?

[mrayandutta]

I am also interested to use Sparklint with Databricks notebooks .Can anybody provide any help on this ?