This was a symptom of a larger problem, I had to downgrade cert-manager to v1.13.1 as that is the latest version of cert-manager compatible with Rancher.