Are you sending the "KeepAlive" messages whenever you have silence?

I know Deepgram's documentation says they will close the connection if there's silence for more than 10 seconds (https://developers.deepgram.com/docs/audio-keep-alive), but we make sure to send the message before that threshold, around 5 seconds for us.

Another thing you might want to look into, are you using raw or compressed audio? Because if you're using compressed audio and there is some sort of blip in the connection, you need to make sure you store the first audio chunk (header) and re-send it when re-connecting before sending any additional audio chunks.