Reports

Adding some additional information gleaned from trial and error and from a formal Google support ticket...

(1) When trying to build Workbench Instances (not Managed or User Managed Notebooks) from Docker images, you are subject to the limitations of Google's Container-Optimized OS, which does not include package managers or the like (no pip or gsutil, for example).

(2) Unlike the User Managed Notebooks, when building a Workbench Instance, the default python environment is stored on the data disk (Instances are two-disk systems, with a boot disk and a data disk). Because of this, installations into the environment only need to happen once, and not on every stop-start cycle.

(3) Per @gogasca's answer, the post-startup script flags work fine. Here is a template for a two part script. The primary script is executed by the default 'root' user. It moves a file from a GCS bucket to the Instance, and also uses EOF to dynamically create a secondary script that is executed by the 'jupyter' user. This secondary script installs GDAL (which can be finicky) and python modules onto the data disk, and into the default 'base' python (conda) environment.

#!/bin/bash

# Creating log file with proper ownership, so that you can easily identify
# points of failure in the post-startup script(s).
if [ ! -d /home/jupyter/logs ]; then
  mkdir /home/jupyter/logs
  sudo chown jupyter /home/jupyter/logs
fi
touch /home/jupyter/logs/log.txt
sudo chown jupyter /home/jupyter/logs/log.txt

# Load some files onto Instance's data disk
if [ ! -e /home/jupyter/someSubDirectory/someFile.sh ]; then
  # Transfer files from GCS bucket
  gsutil cp gs://someBucket/someFile.sh /home/jupyter/someSubDirectory/someFile.sh &>> /home/jupyter/logs/log.txt
  # Give jupyter user ownership and updated permissions
  chown jupyter /home/jupyter/someSubDirectory/someFile.sh &>> /home/jupyter/logs/log.txt
  chmod a+x /home/jupyter/someSubDirectory/someFile.sh &>> /home/jupyter/logs/log.txt
else
  echo -e "Some message here.\n" &>> /home/jupyter/logs/log.txt
fi

# Create an EOF script to execute from within the Instance
cat << EOF > /home/jupyter/createEnvScript.sh
#!/bin/bash
# Install GDAL bindings and python module. The conda installation goes into
# the default conda environment, which is 'base'.
sudo apt-get update -y &>> /home/jupyter/logs/log.txt
sudo add-apt-repository ppa:ubuntugis/ppa -y &>> /home/jupyter/logs/log.txt
sudo apt-get install gdal-bin -y &>> /home/jupyter/logs/log.txt
sudo apt-get install libgdal-dev -y &>> /home/jupyter/logs/log.txt
export CPLUS_INCLUDE_PATH=/usr/include/gdal &>> /home/jupyter/logs/log.txt
export C_INCLUDE_PATH=/usr/include/gdal &>> /home/jupyter/logs/log.txt
conda install -c conda-forge gdal -y &>> /home/jupyter/logs/log.txt
#--------------
# Install various other modules, sending them into the default 'base' environment, per <https://docs.astral.sh/uv/configuration/environment/>
sudo pip install uv &>> /home/jupyter/logs/log.txt
sudo uv pip install earthengine-jupyter>=0.0.7 --link-mode=copy --python /opt/conda &>> /home/jupyter/logs/log.txt
sudo uv pip install earthengine-api>=0.1.418 --link-mode=copy --python /opt/conda &>> /home/jupyter/logs/log.txt
# ...other installations here...
EOF

# Execute EOF script
echo -e "Script created. Changing ownership and permissions\n" &>> /home/jupyter/logs/log.txt
chown jupyter:jupyter /home/jupyter/createEnvScript.sh &>> /home/jupyter/logs/log.txt
chmod a+x /home/jupyter/createEnvScript.sh &>> /home/jupyter/logs/log.txt
echo -e "Executing EOF script.\n" &>> /home/jupyter/logs/log.txt
# The user executing the primary post-startup script is 'root' (whoami  &>> /home/jupyter/logs/log.txt).
# The user that should execute the EOF script is 'jupyter'.
su - jupyter -c 'bash /home/jupyter/createEnvScript.sh' &>> /home/jupyter/logs/log.txt
echo -e "EOF script complete.\n" &>> /home/jupyter/logs/log.txt

After loading it into a GCS bucket (gs://someBucket/postStartupScript.sh in the template code that follows), a script like that above, can be baked into the Instance creation using the following structure:

gcloud workbench instances create someNotebookName \
--project=someProjectId \
--location=someLocation-abc \
--service-account-email=serviceAccountEmailAddress \
--network=projects/someProjectId/global/networks/primaryNetworkName \
--subnet=projects/someProjectId/regions/someLocation/subnetworks/subNetworkName \
--disable-public-ip \
--boot-disk-size=150 \
--boot-disk-type=PD_STANDARD \
--machine-type=n1-standard-4 \
--labels=env=uat \
--metadata=serial-port-enable=TRUE,use-collaborative=TRUE,env=someMetadataLabel1,group=someMetadataLabel2,host=someNotebookName,enable-guest-attributes=TRUE,report-system-health=TRUE,report-notebook-metrics=TRUE,post-startup-script=gs://someBucket/postStartupScript.sh,post-startup-script-behavior=run_once

(4) When building an Instance, selecting (a) the default settings, (i.e., do not specify use of a custom Docker container, but instead "Use the latest version", per Console workflow), is not the same as selecting (b), to use the latest Workbench-specific Docker container (gcr.io/deeplearning-platform-release/workbench-container:latest). With (a), you are using the latest version of a host image (JupyterLab on a host system), which runs on Debian 11. With (b), you are running JupyterLab out of a container, but the end user does not currently have the option to select the latest version of the Container-Optimized OS image that serves as the host in (a).

79102405