Adding the ImageData directly to the canvas bypasses all properties set in the Image object. The Image object does not affect the ImageData object (otherwise you would be unable to reuse it after all).

Instead of using AddXObjectFittedIntoRectangle, use AddImageWithTransformationMatrix.

You can calculate the transformation matrix using the AffineTransform class.