I recently encountered the same issue. To resolve this, I downgraded python version from 3.12.0 to 3.11.9. Because, as per Spark Python Supportability Matrix python version 3.12.0 is not compatible with any Spark version till 3.5.4.