I'm having a similar scenerio in which I'm trying to run a query in the workflow as a step like below, But in the bigquery job it is always failing as the the table name is not proper because the query is getting passed as it is without the variable values getting applied
- execute_bqraw_to_bqcurated_transform:
try:
call: googleapis.bigquery.v2.jobs.insert
args:
projectId: ${args.project_id}
body:
jobReference:
location: ${args.location}
configuration:
query:
query: |
BEGIN;
TRUNCATE TABLE ${args.project_id}.${args.dataset_id}.${args.curated_table_name}`;
INSERT INTO `${args.project_id}.${args.dataset_id}.${args.curated_table_name}`
SELECT *
FROM `${args.project_id}.${args.dataset_id}.${args.raw_table_name}`;
COMMIT;
useLegacySql: false
result: bq_transform_job
retry:
predicate: ${error_handling}
max_retries: 5
backoff:
initial_delay: 10
max_delay: 120
multiplier: 2
next: log_bq_transform_job_id