I tried a bunch of stuff, including making sure I was using a compatible version of Python (I went with 3.8), switching from normal venv to Anaconda, updating pip, and I even attempted to manually download wheel files, but none were available for windows, only for Linux.

The thing that worked for me was going to https://visualstudio.microsoft.com/visual-cpp-build-tools/ and installing C++ build tools. After that, my PyBullet installation worked. That's it.