I found out myself. In the image included in my question I hadn't included the token audience Uri to the resource...
All aforementioned steps in setting up the service principal were correct it seems.
Token uri needed to be https://vault.azure.net
See revised image below.