I just realized that my shell's SPARK_HOME and SPARK_PATH were causing this problem even though the correct venv entry had been added to my PATH. Everything started working once I removed SPARK_HOME and SPARK_PATH from my shell.