Collect 3 or more thread dumps from the driver and any active executors, spacing them about 5 seconds apart. This should help identify where the Spark job is getting stuck.