You probably want to set render_template_as_native_obj=True for your DAG:

https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/models/dag/index.html#:~:text=render_template_as_native_obj%20%E2%80%93%20If%20True%2C%20uses%20a%20Jinja%20NativeEnvironment%20to%20render%20templates%20as%20native%20Python%20types.%20If%20False%2C%20a%20Jinja%20Environment%20is%20used%20to%20render%20templates%20as%20string%20values.

render_template_as_native_obj – If True, uses a Jinja NativeEnvironment to render templates as native Python types. If False, a Jinja Environment is used to render templates as string values.