Here I am answering my own question.
After trying everything you and the internet suggested, I gave up on using virtualenv.

As Matt suggested in the comments, I downloaded miniconda and created an environment there. I was able to download all the packages and the code is running nicely.

Thanks for your help.