79660210

Date: 2025-06-10 10:05:41
Score: 1.5
Natty:
Report link

Solved it, temporarily, by using the param use_cache for the Composer instance.
We have many dags and each dag uses many Variables: this results in the Composer instance to re-parse each dag with all the variables associated.

Really bad legacy pattern.

Waiting for the right time to change this structure, the use_cache param set to True makes parsing faster and the propagation of the Variables' changes slower - fine by me!
The parsing time dropped from almost 10 minutes to 10 seconds, no jokes.

Reasons:
  • No code block (0.5):
  • Self-answer (0.5):
  • Low reputation (0.5):
Posted by: Barbi