Just putting @Emma's comment under the question as the answer for others. Use df.to_spark()
df.to_spark()