79427170

Date: 2025-02-10 13:15:56
Score: 1
Natty:
Report link

Others have suggested the %run which would work but in my current project I opted for the ThreadPoolExecutor. It creates isolated threads on your spark pool on the same session.

with ThreadPoolExecutor(max_workers=int(max_parallel_counter)) as executor:
    mssparkutils.notebook.run(notebook_path)
Reasons:
  • Low length (0.5):
  • Has code block (-0.5):
  • Low reputation (1):
Posted by: JMan