79612459

Date: 2025-05-08 13:15:52
Score: 2.5
Natty:
Report link

I strongly advise against using AVRO for keys, even if you may find it convenient for your use case. The problem is, that the AVRO metadata, which will then be part of the key will also be included in the calculation of the partition number.

AVRO and other schema systems are supposed to facilitate evolution of data structures, but the key is not supposed to evolve in this sense, because it is used for partitioning.

If you use AVRO for keys, then, when your schema id changes in any way, you will get different partitioning, which can be a disaster in production. Even if you don't plan to change your key schemas, there are scenarios, where you might accidentally end up with a different schema id:

Also check out this link: https://forum.confluent.io/t/partitioning-gotchas-dont-use-avro-json-or-protobuf-for-keys-and-be-aware-of-client-hashing-differences/2718

Reasons:
  • Blacklisted phrase (1): this link
  • Long answer (-0.5):
  • No code block (0.5):
  • Contains question mark (0.5):
  • Low reputation (1):
Posted by: selbstereg