I was able to fix this issue by changing my DNS settings and disabling IPv6 on my Windows host.
--------------------------------------------------------------------------------------------------------------------beginner friendly step by step guide :)
Went to my Wi-Fi adapter settings in Windows.

Disabled IPv6.

Manually set the DNS servers to public resolvers ( like Cloudflare 1.1.1.1 / 1.0.0.1 or Google 8.8.8.8 / 8.8.4.4).

After that, Docker containers were able to resolve domains like docker.io and alpinelinux.org without errors.

Root cause was that my system had kept DNS servers pushed from another network (in my case, my university Wi-Fi with AD policies). Those DNS servers were unreachable when I was back home, so Docker inherited broken DNS settings. Forcing my system to use reliable public DNS fixed the problem.