Launch Spark in Foreground via Supervisor We have a spark cluster that launches via supervisor. Excerpts: /etc/supervisor/conf.d/spark_master.conf: command=./sbin/start-master.sh directory=/opt/spark-1.4.1 /etc/supervisor/conf.d/spark_worker.conf: command=./sbin/start-slave.sh spark://spark-master:7077 directory=/opt/spark-1.4.1 The challenge for supervisor is these scripts launch a daemon process and detach, where supervisor expects things to run in foreground without a fork. So far, my efforts to convince supervisor that forking is okay or to convince spark not to fork have come to naught. Anyone find a better way? Thanks!

Solution I inferred from a previous version of the documentation:


/etc/supervisor/conf.d/spark_master.conf:
command=/opt/spark-1.4.1/bin/spark-class org.apache.spark.deploy.master.Master
directory=/opt/spark-1.4.1

/etc/supervisor/conf.d/spark_worker.conf:
command=/opt/spark-1.4.1/bin/spark-class org.apache.spark.deploy.worker.Worker spark://spark-master:7077
directory=/opt/spark-1.4.1


Launching via the `bin/spark-class` command stays in the foreground, and has the added satisfaction of not perpetuating the "slave" terminology.