Per the error this isn't possible. You can only accept and return single objects per row, your object can be arrays/seqs but you cannot use dataframes at all in UDFs, the spark session does not exist on executors. It may run locally but will fail when run on a non local relation.