I don't have the reputation to reply to @crimson-egret but if you, like me, were wondering how the emulator persists - according to https://www.uninformativ.de/blog/postings/2022-09-02/0/POSTING-en.html

We'll be looking at binfmt_misc.c.

In line 777, you'll see that the struct called bm_register_operations gets registered as valid operations for the register file, which is where the Go program writes to. It's defined in line 717 and refers us to bm_register_write() for write operations. In line 660 inside bm_register_write(), we call open_exec(e->interpreter); the argument being the full path to the interpreter file (i.e., our QEMU binary), see function create_entry(). open_exec() comes from the include linux/fs.h and it opens the file for us, so we get a reference to an open file, which is stored in the e struct as well. Later on in line 699, the entire e is added to the entries list. This list will be consulted in check_file() in line 90 to see if an interpreter for a certain binary exists.

So, long story short, this is the magic: Running that tonistiigi/binfmt image instructs the kernel to open a file from that image and keep a pointer to it around, even long after this Docker image has been disposed of.