In my case, this issue instruction helps, tf.test.is_built_with_cuda() return true.
tf.test.is_built_with_cuda()
please try installing TensorFlow v2.4 with CUDA 11.0 and cuDNN 8 as mentioned in the tested build configurations and check if you are facing the same issue.