X and Y are 2D matrices. The dot product you envision creates a matrix where each element follows f_XY[i, j] = cos(X[i, j] * Y[i, j] * 2π). This can be done simply as:

f_XY = np.cos(X * Y * 2 * np.pi)

[output] [1]: https://i.sstatic.net/628ChhBM.png