It's 2025, and I was stuck on this issue. I enabled all maps-related APIs and tried it all, disabling and enabling.
Google API Library doesn't show you that you still want to enable the legacy API. After enabling that, it works fine.
More info: https://developers.google.com/maps/documentation/javascript/directions
The funny thing is even Gemini doesn't help me on this. :D