For those in the future, the next paragraph in that chapter deals with healing this "shadow acne" or whatever it's called. It's because the SDF step ends up right inside the sphere, causing the bounce to terminate on the inside of the sphere. You fix this by ignoring any intersections too close to the SDF source/model surface.