OpenCV's fisheye distortion model uses 4 distortion coefficients, so it expects D to be of size 4.

In your code D has 5 elements. Remove the trailing '0' and it should work.