Oh I think it's because I am in Windows OS.

I checked the milvus-lite readme and it says it only supports Ubuntu and MacOS. So, in windows os, the milvus-lite package won't be installed along with pymilvus.

FYI:

Milvus Lite currently supports the following environments:

• Ubuntu >= 20.04 (x86_64 and arm64)
• MacOS >= 11.0 (Apple Silicon M1/M2 and x86_64)