can u try adding below parameter when writing the dataframe to bigquery.
.option("allowFieldAddition", "True")
allowFieldAdditionAdds the ALLOW_FIELD_ADDITION SchemaUpdateOption to the BigQuery LoadJob. Allowed values are true and false.(Optional. Default to false).Supported only by the `INDIRECT` write method.
Is it possible to share the bigquery table schema or any small working example to demonstrate the issue?