can you share the spark submit command ?
it's very important to add the log4j file manually when submiting your application.
you can't use a log4j.properties file from the class-path.
your spark submit should contains this
--files "./log4j.properties"
-Dlog4j.configuration=log4j.properties