A little late here but you can add OLLAMA_HOST to the containerEnv block in your devcontainer.json and it'll open that port for ollama.

{
  "name, etc...": "My Container",

  "containerEnv": {
    "OLLAMA_HOST": "http://host.docker.internal:11434"
  }
}