As you mention, your input PCollection contains dictionaries. You need a transformation step right before your WriteToBigQuery
to convert each dictionary into the required beam.Row
structure. A common error you might encounter here is a schema mismatch. The fields within the record beam.Row
must perfectly match the columns of your BigQuery table in both name and type. Any extra fields in record will cause a failure.