SOLVED!

Thanks to Wayne from the comments, he guided me into CUDA Program files.

Some of the cuDNN files from downloaded 8.1 version weren't present in

• C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin

• C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\lib\x64

• C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\include

What worked:

Downloading a new cuDNN 8.1 .zip file from NVIDIA website

Extracting it into Downloads/

Copying files from bin/; include/ and lib/x64 into corresponding directories in

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\

Thats it.