I would like to express my sincere gratitude to @ Christoph Rackwitz for his suggestion. By visiting the website he shared, I obtained useful information. Given that there are very few online tutorials mentioning the use of Nvidia GeForce RTX 50 series graphics cards for compiling Cuda and OpenCV library files at the current date, I am sharing my successful compilation experience here.
The version numbers of the various software and drivers I use are as follows:
OS : Windows 11
Cmake：3.28.0
Nvidia Cuda Version : 13.0
Cuda Toolkit：cuda 12.9
cudnn：9.13
Visual Studio：Microsoft Visual Studio Professional 2022 (x64)- LTSC 17.6，Version：17.6.22
OpenCV/OpenCV-contrib：4.13.0-dev, Make sure to download the latest repository files from the OpenCV's Github website. The source code of version 4.12 of OpenCV cannot fully support the Nvidia Cuda Toolkit, and it will cause many problems.
Python Interpreter：Python 3.13.5, I installed a standalone Python interpreter specifically for compiling the OpenCV library files used in Python programming.
CMake flags:
1.Check "WITH_CUDA", "OPENCV_DNN_CUDA" , OPENCV_DNN_OPENVINO(or OPENCV_DNN_OPENCL/OPENCV_DNN_TFLITE), individually, and do not check "BUILD_opencv_world", Set the path of OPENCV_EXTRA_MODULES_PATH, for example: D:/SoftWare/OpenCV_Cuda/opencv_contrib-4.x/modules;
2.Set the values of CUDA_ARCH_BIN and NVIDIA PTX ARCHs to 12.0, check WITH_CUDNN,
3. Check "OPENCV_ENABLE_NONFREE"; If you want to compile the OpenCV library file used for Python programming, the numpy library needs to be installed in the installation path of the Python interpreter. You also need to set the following several paths, for example:
PYTHON3_EXECUTABLE: D:/SoftWare/Python313/python.exe
PYTHON3_INCLUDE_DIR: D:/SoftWare/Python313/include
PYTHON3_LIBRARY: D:/SoftWare/Python313/libs/python310.lib
PYTHON3_NUMPY_INCLUDE_DIRS: D:/SoftWare/Python313/Lib/site-packages/numpy/_core/include
PYTHON3 PACKAGES PATH: D:/SoftWare/Python313/Lib/site-packages
4.Then check BUILD_opencv_python3 and ENABLE_FAST_MATH
After the configuration is completed, use CMake's "generate" function to create "OpenCV.sln". Open "OpenCV.sln" with Visual Studio, and complete the final compilation process by using "ALL BUILD" and "INSTALL". As long as there are no errors reported by Visual Studio, the OpenCV library files have been compiled successfully.