I had to raise my error tolerability count way high

I'm curious which setting you are using for this count. As per this doc it seems checking the checkbox "Ignore unknown values" is all you need.

Also, inferring from the tag bigquery-dataframes, are you using the BigQuery DataFrames library to do your job? If yes please do share your sample code to get more precise help with the library.