Anyone coming across the same thread, I have the answer: https://stackoverflow.com/a/79714613/9076546
Over here I'm transferring way less bytes per second through, and have the same problem. You can't reliably transfer continuous data (no matter the size) with HC-05, find a module that supports BLE, or switch to ESP32-