You can start/stop sparkContext
, not sparkSession
.
Additionally, you are running a python script, inside which you are calling your sparkSession
. When EOF is reached, sparkSession
terminates automatically. You are trying to recreate terminal line behavior, which is not possible in this case.
Please let me know in case of any doubts.