As mentioned in the official docs (https://spark.apache.org/docs/latest/api/python/user_guide/python_packaging.html#using-virtualenv), you have to either export environment variable PYSPARK_DRIVER_PYTHON and PYSPARK_PYTHON to the python binary you're using (i.e. the one in the bin folder of virtualenv directory) or set the path as sparkConf spark.pyspark.python and spark.pyspark.driver.python.