Spark Runtime Entry

It’s all about spark-submit and spark-class bin scripts. Spark-submit is a wrapper for spark-class.

For example, if you ran…

$> ./bin/run-example LogQuery

… The run-example script finds the example class passed in the command args, then invokes spark-submit, which intern invokes spark-class, which finds/creates a host and executes the program.

When running custom Scala Scripts you need to package your classes and dependencies in to a jar, then use spark-submit to execute….

$>./spark/bin/spark-submit --class ScalaScriptClass ./path/to/project/target/scala-x.xx/scalascript.jar