There seems to be no official support for changing the border-radius on the <gmp-map> web component as of now.

Please file a feature request on the public issue tracker here and include a use-case scenario that would be reasonable for Engineers to check on this: https://issuetracker.google.com/issues/new?component=188853&template=787814