79732485

Date: 2025-08-11 20:35:36
Score: 2
Natty:
Report link

If your environment is so concurrent that such rare/hypothetical situation becomes an issue (or you positively can not afford to left even a single record behind) - then go for the most concurrent option in your DBMS and allow dirty reads (set it READ UNCOMMITTED in other words).

Cause, alas, no other offset tracking option (e.g. mode=timestamp+incrementing) would help you there, as timestamp is set at the moment insert/update physically happens, NOT at the moment of commit.

Other option here would be to establish a kind of reconciliation workflow there. But that's gonna complicate things significantly.

P.S. As for "redesign this setup" - think of the following: you, practically, are "publishing" record into a DB table, and then trigger "re-publishing" it into Kafka topic.

Why not skip the DB altogether & just publish direct to Kafka?
Or, if you need your DB table for other purposes - just initiate both events at once: insert into table AND produce into Kafka topic?

Reasons:
  • Long answer (-0.5):
  • No code block (0.5):
  • Ends in question mark (2):
Posted by: Yuri G