79343758

Date: 2025-01-09 19:09:32
Score: 1
Natty:
Report link

You can create a stream for Datastream write to BigQuery in Append Only mode. So you will need to write your own MERGE routines for UPDATE, INSERT and UPDATE-INSERT changes type.

This mode allows you to better control how data will be merged in the final tables and the frequency the merges will be executed.

You can try using Dataform as your solution to schedule and execute the merges from the log tables to the target ones.

Reasons:
  • No code block (0.5):
  • Low reputation (0.5):
Posted by: Renan Benedicto Pereira