Turns out the issue was not related to webpack configs or packages.json.

My environment was setup like this:

My project dir was linked to my WSL2 user home, but the base files lived in my windows system. When I made a copy of my project and saved it directly in my WSL2 file system everything worked.