It's not the answer to your problem per say, but I was dragged to your post by google so it might help others.

I had a very similar issue to the one described, and the problem was created by the structure of my Dockerfile.

RUN mix deps.get
RUN mix deps.compile
COPY . .
RUN mix compile

The COPY command overwrited _build and deps so the dependencies were recompiling every time.

If you use elixir with docker make sure that you either build the docker image with _build (and deps) as volumes (docker build -v ...) like @Adam Millerchip suggested or run mix deps.clean --all && mix clean in your project before running the Dockerfile.