Ok, I don't know whether this is the smoothest solution, but at least I am now able to execute a headless Azure Pipeline Deployment to my target server.

Before I call the docker compose command, I execute this:

gpg --pinentry-mode loopback --passphrase "$(GPG_KEY_PASSPHRASE)" --decrypt $(GPG_PASSWORD_STORE_PATH)

where $(GPG_PASSWORD_STORE_PATH) represents the path to the .gpg file. Then my docker compose command works.