You can use perfect tools like deepgram, gpt-4-mini(this is the best for speed), elevanlabs.
In that case, you can reduce the delay by 1.2 ~ 1.5s.
Hope you are doing well.