I finally found that I can achieve that with disableUnderline like this:
slotProps={{ textField: { InputProps: { disableUnderline: true }, }, }}