I found out what the problem was in my case.

I initially had a gigantic image file of around 70 mb with a gigantic resolution. I'm no image expert. So what I did was compress, and re-compress the file over and over again until it got to 7 mb.

What I didn't do was "resize" it. So the dimension metadata of the image file was still saying its gigantic. And apparently Nextjs Image API went out of memory and just returned the unoptimized image.

After using a tool to resize it it worked.