I was able to get it working.
It seems like setting atmosphere -> OFF doesn't instantly remove it for some reason.
However, you can set it ON and edit the atmosphere values to create a transparent atmosphere, which essentially provides the same effect.