The solution that we found was to make sure that the certificate was marked as exportable, with this done and then regenerated through Digicert things worked without issue.

We stripped back the permissions to just get and list for the secrets and certificates and it continues to work. So that was unnecessary.