79772380

Date: 2025-09-23 08:43:35
Score: 5
Natty:
Report link

@Frank thanks you are right. when I rewrite body of udf whole example looks like:

import org.apache.spark.sql.functions.{udf, struct}

val reduceItems = (items: Row) => {
  // getting array of struct from struct
  val a = items.getAs[Seq[Row]]("second")
  // summing struct item in array
  a.map(_.getAs[Int]("navs")).reduce(_ + _)
}
val reduceItemsUdf = udf(reduceItems)

// passing struct of array of struct
h.select(reduceItemsUdf(struct("*")).as("r")).show()

and it works in spark 4, but I still do not know where was problem, why parameter can't be Seq?

Reasons:
  • Blacklisted phrase (0.5): thanks
  • Long answer (-0.5):
  • Has code block (-0.5):
  • Ends in question mark (2):
  • User mentioned (1): @Frank
  • Self-answer (0.5):
  • Looks like a comment (1):
  • Low reputation (1):
Posted by: Jiri Humpolicek