I strongly advise against using AVRO for keys, even if you may find it convenient for your use case. The problem is, that the AVRO metadata, which will then be part of the key will also be included in the calculation of the partition number.
AVRO and other schema systems are supposed to facilitate evolution of data structures, but the key is not supposed to evolve in this sense, because it is used for partitioning.
If you use AVRO for keys, then, when your schema id changes in any way, you will get different partitioning, which can be a disaster in production. Even if you don't plan to change your key schemas, there are scenarios, where you might accidentally end up with a different schema id:
adding or change doc-Fields in your Schema: Who would think that this will have a potentially disasterous impact on your system?
starting with a fresh schema registry after accidentally loosing or purposefully deleting your data
Also check out this link: https://forum.confluent.io/t/partitioning-gotchas-dont-use-avro-json-or-protobuf-for-keys-and-be-aware-of-client-hashing-differences/2718