I searched and finally found a similar post:

Debug a Python C/C++ Pybind11 extension in VSCode [Linux]

"Upon reaching your binded code in Python, you may have to click manually in the call stack (in the debug panel on the left) to actually switch into the C++ code."

From the above description, it seems that we need to click the call stack to switch to C++ code manually. So what I encountered is normal.