For me the key was switching to using shallowRef() instead of ref() for the dom element references. Once I made that switch everything started working.
shallowRef()
ref()