The solution was to request AWS to perform some actions on the RDS server.

After much effort, our team resorted to contacting AWS support, and through a lot of back and forth, we found that the temp files were inaccessible to us. After dropping all the FULLTEXT indexes, AWS support then went in and removed the dangling references/temp files. We then performed the upgrade to MySQL 8.0.35 and then re-added the FULLTEXT indexes.