IMEX does have effect, but it works in a crazy way : It depends on what proportion of data is returned as text by theTypeGuessRows parameter.

Detailed explanation is available here : https://web.archive.org/web/20211215203520/https://www.etl-tools.com/imex1.html

My solution is to always set IMEX=1 with TypeGuessRows=1 AND ensure my Excel files always include a first "dummy" row with the data type I need the ADO driver to interpret data as. This is also explained in this article.