It turns out that this behavior only happens when SKScene does not have a size. It works again after I specify the size of the scene:

      let scene = SKScene(size: view.frame.size)

I don't know how this is related. Could be a bug or some undocumented behavior.