I think I was able to fix it.

First, I needed to update 'qemu', as otherwise compiling libGL.so wouldn't work (see details https://askubuntu.com/questions/1339558/cant-build-dockerfile-for-arm64-due-to-libc-bin-segmentation-fault). The command for this is (and I admit I'm not sure what it actually does):

docker run --rm --privileged multiarch/qemu-user-static --reset -p yes -c yes

Then, having this solved, I added following to my Dockerfile:

RUN apt-get clean
RUN apt-get update
RUN apt-get install libc-bin -y
RUN apt-get install libgl1-mesa-glx -y

Without the first step, 'RUN apt-get install libc-bin -y' was failing with a segmentation fault, and so was 'RUN apt-get install libgl1-mesa-glx -y'

With those, I was able to get the libGL.so compiled, but still not "included" by pyinstaller. But at this point, it was sufficient to manually copy the file from one place to the other, by adding the following to the Dockerfile:

RUN cp /usr/lib/aarch64-linux-gnu/libGL.so.1.7.0 $MY_DIRECTORY/dist/$MY_APP_NAME/_internal

Note that you need to adjust the line above to suit your exact needs, i.e. to copy the right file to the right location.