What I found that resolved this issue for me is that I needed to enable inbound traffic on the VPC Network ACL. Although, all the docs say SSM does not initiate a TCP connection to your instances. The instances need to be able to get request responses from the requests it makes. I added inbound access for these port ranges that are ephemeral ports and SSM worked after that.

You do not need to add inbound rules to the security group because they are stateful.

From the docs

Security groups are stateful. For example, if you send a request from an instance, the response traffic for that request is allowed to reach the instance regardless of the inbound security group rules. Responses to allowed inbound traffic are allowed to leave the instance, regardless of the outbound rules.