Setting pointer-events: none; doesn't solve the problem entirely, as the select can still be controlled using keyboard.

One quick, html only way to fix that is using the newly available (at the time of writing) inert attribute, which blocks all interactions.