For version > 8.3
If you're using latest image tag with below command, you may see similar issue.
--default-authentication-plugin=mysql_native_password
In that case, you might need to downgrade to version 8.3 and use this mysql_native_password authentication.
You might need to run docker rm mysql or docker compose rm mysql in order to downgrade and rerun docker compose up.
This issue provides more information.
For version 8.4
You may need to add this command --mysql-native-password=ON and remove --default-authentication-plugin=mysql_native_password
For version 9+
Version 9 removed this option --mysql-native-password as mentioned in the release notes.