79785231

Date: 2025-10-08 08:32:50
Score: 1
Natty:
Report link

A few years back I've had this problem.
We chose to forward the message from A to a new topic.
Now I am thinking about implementing a "smart" consumer:

With the help of a KafkaAdminClient (https://kafka-python.readthedocs.io/en/master/apidoc/KafkaAdminClient.html) you can get the current offset of the first group and get the messages up to that point.
Knowing your current and the other group's offset, it's possible to calculate a `max_records` for the manual poll method (https://kafka-python.readthedocs.io/en/master/apidoc/KafkaConsumer.html#kafka.KafkaConsumer.poll).

Still thinking about possible drawbacks, but I think it should work.

Reasons:
  • No code block (0.5):
  • Low reputation (0.5):
Posted by: Szabolcs Szepesi