You could use cache mount in the Dockerfile link to Docker documentation

The cache is cumulative across builds, so you can read and write to the cache multiple times.

In the link there is this example for Python

RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt