Skip to content

Run Jython code within the Storm framework for distributed and fault-tolerant realtime computation

Notifications You must be signed in to change notification settings

jimbaker/jython-storm-starter

Repository files navigation

Setup

This will be replaced by a much better setup. But for now:

You need Storm installed with its bin directory on $PATH.

Checkout the proxymaker branch of Jython and build it. It includes a necessary fix for using Jython with Clojure types, as used in Storm, in Jython trunk, as well as new custom proxy maker support for better Java integration

$ hg clone ssh://[email protected]/jimbaker/proxymaker
$ cd proxymaker 
$ ant                                                 # build development version (fastest way)
$ export PATH=$(pwd)/dist/bin:$PATH                   # add jython to your path

Two more steps:

  1. Build an "uber" jar file for Storm; this contains all necessary jars. Running storm classpath computes the necessary classpath for Storm dependencies:
$ CLASSPATH="$(storm classpath)" jython27 gen-storm-jar.py \
    -o uber.jar -i excl -i clamp --proxy=excl.plumbing
  1. Run the topology. Either stand alone mode, using run-exclamation-topology.py, which was bundled in the uber jar step:
$ CLASSPATH="$(storm classpath):$(pwd)/uber.jar" \
    java org.python.util.jython run-exclamation-topology.py

or on the storm cluster by submitting the uber jar; the corresponding cluster submitting code is in __run__.py:

$ storm jar uber.jar org.python.util.JarRunner

About

Run Jython code within the Storm framework for distributed and fault-tolerant realtime computation

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages