Unfortunately, single consumer + multithreaded workers is a good exercise but only.

Under real conditions, this is not a reliable model because :

• all this has a reason when enable.auto.commit is set

• failure of a worker leads to the processing being lost together with the data source.

• failure of the whole JVM leads to all processing being lost and data cannot be restored on the next start, either next start is done with another consumer group name, and we need to know which messages were processed already

• fixing previous potential failure topics requires implementing a recovery mechanism, that is not a trivial job. Redis can be used to keep source data in the cache till the end of processing, but this way we add additional networking and serialization.