it would be excelent if you provide job with step where you do terraform plan -out someplan.tfplan and ensure you use upload/download artifact only for someplan.tfplan

it is obvious you upload whole repo or some other stuff and not only terraform plan file. E.g. 200MB artifact compressed takes few seconds up and similar to download.