Answer was quite simple in the end. I had CoPilot write me a PowerShell script to update all the TreatWarningsAsErrors tags. I would have preferred a way to do this from withing Visual Studio, but this worked.

Get-ChildItem -Path "path to solution" -Recurse -Filter *.csproj |
ForEach-Object {
    (Get-Content $_.FullName) -replace '<TreatWarningsAsErrors>true</TreatWarningsAsErrors>', '<TreatWarningsAsErrors>false</TreatWarningsAsErrors>' |
    Set-Content $_.FullName
}