Naturally, right after I make a StackOverflow question I figure it out!

The answer is to call .persist() after the .reply call:

    agent
        .get('https://stackoverflow.com')
        .intercept({
        path: '/notarealendpoint',
        method: 'GET',
    })
        .reply(200, "foo")
        // this is new
        .persist();

See https://github.com/nodejs/undici/blob/main/types/mock-interceptor.d.ts#L10.

There's also a .times function if you only want the mock to persist N times.