Yes, each environment must have ipykernel and jupyter. There is a convenient package, https://github.com/anaconda/nb_conda_kernels that allows to invoke jupyter notebook or jupyter lab from the base environment and have access to other knernels in other environments.