The way I go about this is using ipykernel in my virtual environments which I need to use with the Jupyter notebooks so that I can switch to the appropriate environments when using the notebook.

All I need to do is to switch between environments to make sure I only need the one that is for the Jupyter notebook.

P.S. a new utility is in town called Puppy. You might want to give this a read!