Following on from Kashyap's anwser, i neededed to include the following to the json request

  "spark_conf": {
     "spark.databricks.cluster.profile":"singleNode",
     "spark.master":"local[*, 4]"
  },
 "custom_tags": {
  "ResourceClass":"SingleNode"
}

in order to be able to interact with tables and not present warning messages in the UI.