Although quite some time has passed since the question was asked, I’d still like to share my solution—especially because the above methods didn’t work for me.
Newer excel has a functionality named "Get Data (Power Query)" – which can read text of UTF-8. So the csv files in UTF-8 encoding seemed to be loaded perfectly.
The functionality can be searched by name, or by the path on my mac: "Data" > "Get Data (Power Query)" > "Data (Power Query)" I hope this solution will help.