Set explicitly width to None.

st_folium(
    folium.Map(
        location=[-20, 130], 
        zoom_start=4, 
        control_scale=True), 
    width=None, 
    height=400)

This will work as expected.

As a side note, you can also use use_container_width=True instead of setting the width.