I posted an article in Medium with the details.
In summary you need to mount a
spark-defaults.conf
with the content
spark.jars.packages org.apache.spark:spark-sql-kafka-0-10_2.13:4.0.1 spark.jars.ivy /opt/spark/ivycache