A colleague of mine pointed me to Scala documentation over here. The working Scala code then becomes

.withColumn("next_tbl_rn", lead($"tbl_rn", 1, null, true).over(Window.orderBy($"rn")))

in which the fourth argument does the trick.