TIL - Combining docker bind and cache mount types

bind mount

The bind mount allows the build process to read the source file without adding it to the image.

The following two cases produce a very similar result:

  1. COPY + RUN:
    COPY req.txt .
    RUN pip install -r req.txt
    
  2. RUN with bind mount
    RUN --mount=type=bind,source=req.txt pip install -r req.txt
    

The layer cache key of the RUN command with a bind mount has similar inputs to the COPY command, the hash of the source path, mode and ownership. The cache key of the RUN command with the bin mount also includes the arguments of the mount command, therefore modifying the source file or the mount arguments will invalidate the cache key.

Modifying the contents of the req.txt file invalidates the cache in both cases. In case 1 the COPY command cache will be invalidated and therefore all the downstream caches. In case 2 the RUN command cache will be invalidated and similarly all the downstream caches.

Differences between the two cases:

  • Files in the image. While the COPY command puts the req.txt in the image, the bind mount will not persist the file in the image.
  • Number of layers. COPY + RUN commands result in two layers while the single RUN command results in a single layer.

cache mount

The contents of the target directory never make it to the image.

RUN --mount=type=cache,id=pip,target=/root/.cache/pip,sharing=locked \
    pip install -r /tmp/req.txt

The layer cache key does not take into account the contents of the target directory.

The layer created never contains the files under the mount target directory, those files only exist in the machine local file system. This means that whatever goes into that directory is never backed up to the registry. A later COPY command might copied the files from the cache directory to the image. On ephemeral runners the target directory will always be empty, unless using something like buildkit-cache-dance tool.

In the example above, the installed packages go to into /usr/local/lib/python3.x/site-packages, which goes into the layer like any other RUN command. The wheels however, downloaded in /root/.cache/pip, a cache volume on the builder, are not present in the layer.

In the case of running multiple builds in a local machine, or a non-ephemeral runer, assuming that only a single dependency is updated between builds, the installation will be faster since only that dependency will need downloading.

Combining both mount types

RUN --mount=type=bind,source=uv.lock,target=uv.lock \
    --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
    --mount=type=cache,target=/root/.cache/uv \
    uv sync --frozen --no-install-project

On this example, the uv.lock and pyproject.toml files will not be present in the image and the downloaded packages will be kept in the builder file system.

  • The cache key will account for the lock and pyproject contents. The cache is invalidated if any of those files change.
  • The built layer only contains the installed environment.
  • Speed: in case of ephemeral runners, since the downloads do not exist in the layer, the packages will always be downloaded.