It turns out I didn't use the correct CMD to start the server, replaced with
CMD ["mvn", "exec:java", "-Dexec.mainClass={mymainclass}"]
then it just worked the same way as localhost.