Annotations can only be applied to the whole kubernetes resource because they are part of the resource metadata. One way to solve this is to create separate Ingress resources for each host or path with specific settings.