Because your asumption is wrong, that is not how it works. A 3D scene is not a chalkboard.

You have to render your entire scene at every frame.