Apparently the latency is added because of mp4 container and its internal file structure. Instead of trying to tweak its properties I decide to make it simpler and now Im sending the actual jpeg frames. Final solution can be found here: https://github.com/bymoses/linux-phone-webcam