docker compose up --build --watch seems to be the answer. Weird how the tutorial I'm watching doesn't make this clear. I was under the impression docker compose up builds your containers even the documentation says it:

Usage docker compose up [OPTIONS] [SERVICE...] ....... Builds, (re)creates, starts, and attaches to containers for a service.

But apparently this isn't true and you need the --build option.