It turned out that there was no support for this until very recently. The corresponding discussion in github is this: https://github.com/grafana/grafana/pull/99279.

So if you encounter this as well, make sure to have the latest version of grafana running.