In case you are using NixOS, you will need to install swig system package instead of the pip one.

nix-shell -p swig
pip install "gymnasium[box2d]"

Here, I am installing swig temporarily and then installing the package. Alternatively, you can set it up in your configuration.nix