The error arises because your custom function mixes TensorFlow and NumPy operations. TensorFlow operations are essential for gradient calculation but you are using NumPy operations. So, to fix this error you need to replace all NumPy operations with their TensorFlow equivalents. This will enable the calculation of gradients for all operations involved. kindly refer to this gist.