Turns out I forgot to import hdf5plugin in the separate ipynb that I used to read in the h5 file. It works now.
import hdf5plugin