I think you're ask too much from React, whose goal is building single page applications.
If you don't want big bundles because the initial loading time worries you; You will have to make a React application per folder; then make sure every link points to the right URL. This will add more complexity because is more deployments, duplicate code, etc.
I highly recommend React router, I think you should check it out... https://reactrouter.com/en/main/start/overview
React Router enables "client side routing".
And many others.