It's just because our manual failover API call eventually is directed to sentinels by internal algorithms. It's quite natural for sentinel to do so as sentinel has responsibilities to monitor nodes.
However, as you wonder, some people try to make changes so that we can deal with it like cluster (ask replicas instead of sentinels). For example, this is an issue in valkey project. https://github.com/redis/redis/issues/13118