I had a similar issue, and I think I managed to solve it by increasing the value ofnet.netfilter.nf_conntrack_udp_timeout_stream using sysctl. It defaults to 120s, so this may be the setting that causes the timeout error in spark.