The question may already be answered, but here is the actual way to do it:
Add the following line in your .replit file, below the entrypoint line:
.replit
entrypoint
disableGuessImports = true
This prevents the packager manager to install these packages.