Arduino IDE is ideal for small size projects and intended for beginners. I used it for first projects. For some years for arduino based boards i used Eclipse IDE with Sloeber plugin. But having some issues with modern boards in Sloeber plugin i started to use PlatformIO extension with VS code.
I'm working on Mac, but for test purposes i installed VS Code on winwows 10 and tried both ways: install ESP-IDF tools separately then link it to VS Code ESP-IDF extesion as existing, and install inside ESP-IDF tools inside VS Code ESP-IDF extesion (as you did).
Both ways failed for me (i used not default, but custom location for ESP-IDF).
PlatformIO extension supporting more than 1300 boards, and for Espressif based boards you can select ESP-IDF or Arduino framework.
Installing PlatformIO extension on windows 10 and selecting ESP32, i tried to build demo projects using Arduino and ESP-IDF frameworks. Both of them working out of box.
Hope this will be helpfull for you. Happy coding.