I've managed to figure it out! Here is the explanation for those wondering..

For some reason, registering host ports using ".WithHttpEndpoint" produce this error in latest versions of Docker Desktop. But, if you revert Docker Desktop to several older minor versions, it solves it. (Anything before 4.40. it seems)

On the other hand, it seems that there is indeed a fix also on latest version of Docker Desktop, you have to pass host ports like this:

var mongoDb = builder.AddMongoDB("mongodb", 27017)
    .WithExternalHttpEndpoints();