79196380

Date: 2024-11-17 02:02:32
Score: 8 🚩
Natty:
Report link

Hello @jkff and @Graham: I am having dataflow streaming pipeline, which uses WriteToBigQuery after reading from original source. I have similar requirement where I want to achieve two things: 1. handling WriteToBigQuery error records and 2. execute big query SQL which has to be executed only after WriteToBigQuery. For 2nd requirement: Approach mentioned by @jkff wont work for me as it will submit separate dataflow steaming job which will be async with original job. I have question about Approach mentioned by @Graham, If I do that way, will that get executed everytime there is streaming data coming from original source?

Reasons:
  • Blacklisted phrase (1.5): I have question
  • Blacklisted phrase (1): I have similar
  • RegEx Blacklisted phrase (1): I want
  • Long answer (-0.5):
  • No code block (0.5):
  • Ends in question mark (2):
  • User mentioned (1): @jkff
  • User mentioned (0): @Graham
  • User mentioned (0): @jkff
  • User mentioned (0): @Graham
  • Looks like a comment (1):
  • Low reputation (0.5):
Posted by: ChitsC