79750846

Date: 2025-08-29 23:43:51
Score: 1
Natty:
Report link

Instead of mixing async I/O and ProcessPool Workers in the same container, could you split it into dedicated processes? Because both async I/O and ProcessPool Worker use CPU alot.

What I'm saying is, let's split your application in two.
In the first application, Async I/O pull message from Kafka topic A and write it to Redis.
In the second application, ProcessPool Workers read topic A message from Redis, run the algorithm, and write the result to Redis. And also Increment the redis counter.
In the first application, Async I/O reads the result from Redis and push it to Kafka topic B.

This way, you can run each application in different containers, reducing performance issues. But with this way you need more RAM for Redis.

Reasons:
  • Long answer (-0.5):
  • No code block (0.5):
  • Contains question mark (0.5):
  • Low reputation (0.5):
Posted by: Ferhat Mousavi