It’s tough to assist without more details, but here’s where DAB typically pulls cluster configurations from:
- First and foremost, check the job’s YAML file. Hope you have things sorted here.
- If you’re using the extension, double-check the selected cluster in the UI and the “Override Job Clusters in Bundle” checkbox.
- Look for cluster_id in your .databrickscfg file, in case you used Databricks CLI to configure.
- project.json file under .databricks folder will be holding the project level configuration for cluster.
- And databricks's own terraform files - .databricks/local/terraform/bundle.tf.json or terraform.tfstate - Remember, DAB is essentially a wrapper around Terraform, so these files likely hold the keys to resolving your issue.