A possible solution, I came across a note in Espressif Github which helped me partially resolve this issue, under the title Pin assignments for ESP32-S3

"not used in 1-line SD mode, but card's D3 pin must have a 10k pullup"

https://github.com/espressif/esp-idf/tree/346870a3/examples/storage/sd_card/sdmmc

I was using a SD card holder intended for SPI and the CS pin (which is D3 in MMC mode) did not have a pullup resistor on the card.

My initial benchmark test result is usually around 2MB/s, but it can slow down after that depending on the order of other I/O functions after the first write test.