It seems that the above can be achieved using MERGE sql function as outlined in the documentation: https://docs.databricks.com/aws/en/sql/language-manual/delta-merge-into