In the meantime this got fixed in Docker Desktop for Mac 4.39.0.
Release notes:
Fixed a bug that caused all Java programs running on M4 Macbook Pro to emit a SIGILL error. See docker/for-mac#7583.