Exactly the same problem as you, except that when I run this project at home, even the first load is ultra fast. On the other hand, at work, the first load is catastrophic: 3 minutes waiting time for the vendor.scss file.
I'd advise you to drop the package.json tracks or reduce the number of vendors - that's not the problem.
For example, it takes 30 to localhost:8080 to render a 200kb jpg...
Once the first launch is completed, all subsequent launches run smoothly.