I had to remove the finalizer from the ingress - then it was deleted
kubectl patch ingress my-ingress -n my-namespace -p '{"metadata": {"finalizers":[]}}' --type=merge