It seems that you have to run pipenv uninstall --all-dev to remove all dev packages first, and then reinstall those you need.
pipenv uninstall --all-dev