You can configure env var SPARK_WORKER_CORES, which makes spark assume you have this number of cores regardless of what you actually have. If you are interested in other optimizations for Databricks you can check here https://zipher.cloud/
SPARK_WORKER_CORES