As stated in the comments, eigenvectors only indicate the correct "direction" but not the correct "orientation". So a solution would be to test all the symmetries and keep the "best one". One possibility is to compute the distance between the bounding boxes of source and destination pointclouds after rotation for all potential rotation matrices (taking symmetries into account) and keep the rotation matrix with the lowest distance.

It would have been easier to give you a proper answer if you had provided some sample code, but here is a Jupyter notebook I created to explain exactly this (funny enough, also using the Stanford bunny as an example).