Answered: Project Settings -> Physics 2D -> Default Contact Offset -> 0.05 (leaves small barely gap, enough not to cause fast moving object go thru each other).

Box collider -> reduce by the same size to let sprite "settle" the gab.

Or leave it as it is. Also change in the sprite it self setting "Physical Shape", so colliders by default will be created based on it's size.