I was facing the same problem during job execution in Java Batch (JSR352).
In the beginning I sent mail using commons-email - but that library creates a connection for every single email. No wonder that after a short burst the mailserver considered a DOS attack and stopped responing to connection requests.
Then I switched to Simple Java Mail, which claims to boost performance using synchronous and asynchronous methods and allows you to reuse connections. But despite it says to be lightweight it pulls in a huge load of dependencies which also manifested in classpath issues.
So I went back to coding directly on JavaMail. It's not too bad after all, and you have full control on connections plus no further dependencies. Every partition in my batch run would need only one connection. Better but not good enough, as the mailserver still stopped responding.
Finally I combined JavaMail with smtp-connection-pool. Now the connections were shared across partitions and threads. The 10 threads running my batch in 150 partitions used 8 SMTP connections altogether, and the server did no longer consider a DOS attack. As a side effect performance raised dramatically since establishing less TLS sessions with my mailserver saved a few seconds each.
----------
Coming back to your approach:
Storing each connection in a hashmap is not too bad, but for a real connection pool you still want to know whether a connection is being used (borrowed) by a thread or idle. You want to close connections that are idle for too long. You want to make sure that a thread using the connection cannot close it accidentially. You want metrics to see how many connections were created, borrowed, destroyed, ...
All this is already implemented maturely in smtp-connection-pool. So why create from scratch?