Spark Submit

Binding: %spark-submit

Submits a precompiled Spark application (a jar or script you've already built) as a job, the same way you would from the command line — without keeping an interactive Spark session open in the notebook. Use this for packaged jobs rather than exploratory Spark work, which belongs in Apache Spark instead.

%spark-submit
--class com.example.MyJob --master yarn /path/to/my-job.jar arg1 arg2

Configuration

Property	Default	Description
SPARK_HOME	(empty)	Location of the Spark distribution used to run spark-submit.

This is set by your administrator from the interpreter settings screen.