After modifying the following I was able to run in cluster deploy mode on my standalone cluster. The job that finishes without errors in Client Deploy Mode is now failing with a typical distribution issue (serialization of a UDF)
Environment settings -> deploy mode cluster
removed SPARK_LOCAL_IP from the spark-env.sh
removed spark.driver.host from spark-default.conf
spark.driver.bindAddress=<hostname of the node on which spark-default.conf file is modified on>
I have three nodes and set each of their files to their own host name
(note : removed spark.executor.host as it doesn't even exist :-S )