Well, I solve it. I carefully check my settings and process. Maybe the VScode cannot handle bash(MSYS2) perfectly.

I also read the official guide agagin Get Started with C++ and MinGW-w64 in Visual Studio Code

In the guide, it told us that we should add C:\msys64\ucrt64\bin into you PATH. Then set the PowerShell as the default terminal.

Then the output of terminal become clean and right.

USE the POWERSHELL as default terminal if you can.