The issue has been identified. Updates were being made to the wrong appsetting file. Updates were being made to Windows appsetting rather than the Linux appsetting. So the application was still using a mongodb connection string of 'localhost:27017' rather than the correct one (as displayed in my original post).
Appreciate all the responses for helping me looking into the problem.

Correct bindIp: 127.0.0.1,<LocalHostName>
Correct mongodb connection string: "mongodb://<hostname1>:27017,<hostname2>:27017?replicaSet=mysetName&readPreference=nearest"