It was a very simple fix. My wsl install was out of date. I ran wsl --update and after that, any project that I scaffolded with Docker support through Visual Studio worked perfectly.