79522898

Date: 2025-03-20 12:20:19
Score: 0.5
Natty:
Report link

I don't know exactly how monotonically_increasing_id is defined, but perhaps you can implement it within DuckDB by using a lambda function or udf?

Something like (untested!!!):

import duckdb
from pyspark.sql import functions as sf

def my_best_guess() -> duckdb.Type.something:
  # Probably need to do some type hacking here?
  return sf.monotonically_increasing_id()

duckdb.create_function(my_best_guess); 

duckdb.sql("SELECT my_best_guess, cols FROM ...")

Sorry I don't have a complete answer, but the above looks to be pointing in the right direction.

Reasons:
  • Long answer (-0.5):
  • Has code block (-0.5):
  • Contains question mark (0.5):
  • Low reputation (1):
Posted by: Dan Bolfter