You need to make sure that the map options for user interaction are enabled

<Map
  center={position}
  zoom={10}
  mapId={'MAP_ID'}
  options={{
    zoomControl: true,
    scrollwheel: true,
    draggable: true,
  }}
>
</Map>

Find more configurations here