Docker Image Layers, Union Filesystems, and Image Caching Explained

Why does changing one line of code trigger a ten-minute pip install? Why does deleting a file in a Dockerfile not make the image smaller? Why can you run fifty containers from a 1 GB image without using 50 GB of disk? The answers all come from three ideas at the heart of Docker: image layers, the union filesystem that stacks them, and the cache that reuses them.

Understanding these ideas is the difference between images that are bloated and builds that are slow, and images that are small, builds that finish in seconds, and deployments that pull only what changed. This guide explains how layers are created and stored, how OverlayFS combines them with copy-on-write, how the BuildKit build cache decides what to reuse, and the practical techniques that follow. Examples are written for Docker Engine 29, as of October 2026.

Quick Answer

Image layers: each filesystem-changing Dockerfile instruction (RUN, COPY, ADD) creates a read-only layer containing only the files that changed. An image is an ordered stack of these layers plus a configuration.
Union filesystem: Docker uses OverlayFS to stack the read-only image layers and a thin writable container layer into one merged filesystem. Changes use copy-on-write, so the image is never modified and is shared by every container.
Image caching: layers are identified by content hashes, so identical layers are stored, pulled and pushed only once. The build cache reuses the result of a Dockerfile step when its instruction and inputs have not changed; once one step changes, every later step is rebuilt.
Practical rule: order your Dockerfile from least to most frequently changing, and clean up temporary files in the same RUN instruction that created them.

What Is an Image Layer?

A layer is a set of filesystem changes: files added, modified or deleted compared with the layer below it. It is stored as a compressed archive (a tarball). An image is simply an ordered list of layers plus a small JSON configuration that records metadata such as the default command, environment variables, working directory and user.

Figure 1: How Dockerfile instructions become read-only layers under a writable container layer

Which Instructions Create Layers?

Instruction	Creates a filesystem layer?	Notes
FROM	Reuses the base image's layers	Every layer of the base image becomes part of yours
RUN	Yes	Captures every file the command added, changed or deleted
COPY, ADD	Yes	Contains the copied files
WORKDIR	Only if it creates a directory	Usually a tiny or empty layer
ENV, ARG, LABEL, EXPOSE, USER, CMD, ENTRYPOINT, HEALTHCHECK, STOPSIGNAL, VOLUME	No	They change only the image configuration (metadata)

See the layers of any image with docker image history:

docker image history python:3.13-slim

IMAGE CREATED CREATED BY SIZE

<missing> 2 weeks ago CMD ["python3"] 0B

<missing> 2 weeks ago RUN /bin/sh -c set -eux; ... 37MB

<missing> 2 weeks ago ENV PYTHON_VERSION=3.13... 0B

...

Rows with a size of 0B are metadata instructions. The <missing> value in the IMAGE column is normal for pulled images: intermediate build steps are not separate images on your machine. The exact sizes you see will differ by version and platform.

How Layers Are Identified

Every layer is content-addressed: its identity is a SHA-256 hash of its content. This has two important consequences:

Deduplication. Two images that share a base image share exactly the same layer files on disk, in the registry, and over the network.
Integrity. If even one byte of a layer changes, its hash changes, so tampering or corruption is detected automatically.

Identifier	Hash of	Used for
Layer digest	The compressed layer archive	Downloading and uploading blobs to and from registries
DiffID	The uncompressed layer archive	Listed in the image configuration (rootfs.diff_ids) to identify layer content
Image digest	The image manifest	Pulling an exact, immutable image (image@sha256:...)

docker image inspect -f "{{json .RootFS.Layers}}" python:3.13-slim

Union Filesystems and OverlayFS

A container needs one ordinary-looking root filesystem, yet its image is made of many separate layers. A union filesystem solves this by presenting several directories stacked on top of one another as a single merged directory. On Linux, Docker uses the kernel's built-in OverlayFS for this.

The Four OverlayFS Directories

Directory	Role in a Docker container
lowerdir	One or more read-only directories: the image layers, stacked in order
upperdir	The read-write directory: the container's writable layer, where all changes go
workdir	An internal working directory OverlayFS needs to perform changes safely
merged	The unified view of lower and upper directories, mounted as the container's root (/)

When several layers contain the same path, the version in the higher layer wins. This is how a later Dockerfile instruction can override a file installed earlier.

Copy-on-Write: How Containers Change Files

Operation inside the container	What OverlayFS does
Read a file	Reads it directly from the highest layer that contains it. No copying.
Create a file	Writes it to the upperdir (writable layer) only.
Modify a file from the image	Copy-up: copies the whole file from its read-only layer into the upperdir, then modifies the copy. The original layer is untouched.
Delete a file from the image	Creates a whiteout entry in the upperdir that hides the file. The original still exists in its read-only layer.
Delete a directory from the image	Creates an opaque directory marker that hides the lower directory's contents.

This copy-on-write design is why containers start almost instantly: Docker does not copy the image, it only creates an empty upperdir and mounts the overlay. It is also why one image can back hundreds of containers while using its disk space only once.

Performance tip: Copy-up copies the entire file, even to change one byte. Modifying large files inside the container's filesystem (for example database files of several GB) is slow and duplicates their size. Write-heavy data belongs in volumes, which bypass the union filesystem entirely.

See OverlayFS on a Real Container

On a host using the classic overlay2 storage driver, docker inspect shows the directories directly:

docker run -d --name demo nginx:1.29

docker inspect -f "{{json .GraphDriver.Data}}" demo

mount | grep overlay

The output lists LowerDir, UpperDir, WorkDir and MergedDir paths under /var/lib/docker/overlay2/. Create a file in the container, and it appears in the UpperDir on the host:

docker exec demo sh -c "echo hello > /tmp/hello.txt"

sudo ls $(docker inspect -f "{{.GraphDriver.Data.UpperDir}}" demo)/tmp/

Use docker diff demo to list everything the container has added (A), changed (C) or deleted (D) compared with its image, and docker ps -s to see the size of each container's writable layer.

Docker Engine 29: Fresh installations of Docker Engine 29 use the containerd image store by default. Layers are then managed by containerd's overlayfs snapshotter under /var/lib/containerd, instead of the overlay2 graph driver under /var/lib/docker/overlay2. The OverlayFS mechanism (lower, upper and merged directories, copy-up and whiteouts) is exactly the same; only the storage location and some docker inspect output differ. Check which one a host uses with docker info.

Why Deleting Files Does Not Shrink an Image

Because deletions are only whiteouts, a file added in one layer and deleted in a later layer still occupies space in the earlier layer. The image gets bigger, not smaller:

# Bad: the archive stays in layer 1, layer 2 only hides it

RUN curl -fsSLO https://example.com/tool.tar.gz

RUN tar -xzf tool.tar.gz && rm tool.tar.gz

# Good: download, extract and clean up in one layer

RUN curl -fsSLO https://example.com/tool.tar.gz \

&& tar -xzf tool.tar.gz \

&& rm tool.tar.gz

The same applies to package manager caches. Clean them in the same RUN instruction:

RUN apt-get update \

&& apt-get install -y --no-install-recommends curl ca-certificates \

&& rm -rf /var/lib/apt/lists/*

The even better solution is a multi-stage build: build in one stage and copy only the results into a clean final stage, so build tools and temporary files never reach the final image. See our Complete Guide to Multi-Stage Docker Builds.

Security: A secret that is added in one layer and deleted in a later layer is still inside the image. Anyone who pulls it can extract the earlier layer and read the secret. Never COPY secrets into an image; use build secrets (RUN --mount=type=secret) instead.

Layer Caching Across Images and Machines

Content addressing makes layers reusable everywhere, not only during builds:

Where	How layers are reused
Local disk	Images built on the same base share its layers. docker system df -v shows shared and unique size per image.
docker pull	Docker downloads only layers it does not already have ("Already exists" in the output).
docker push	Docker uploads only layers the registry does not already have ("Layer already exists").
Registries	Each unique layer is stored once, no matter how many images or tags reference it.
Containers	Every container from an image shares its read-only layers; each has only its own small writable layer.

This is why using a common base image across your services pays off: servers that already have it pull only your small application layers during a deployment.

Reading image sizes: The SIZE column of docker image ls shows each image's full size, including layers it shares with other images, so adding up the column overstates real disk use. Use docker system df for actual usage.

The Docker Build Cache

When you run docker build, BuildKit checks each instruction against its cache. If it finds a cached result for the same instruction with the same inputs, built on the same parent, it reuses it and prints CACHED. Otherwise it runs the instruction.

How BuildKit Decides Whether a Step Is Cached

Instruction	Cache key is based on	Cache is invalidated when
FROM	The base image reference	The base image changes (for example after --pull fetches a newer one)
RUN	The command string, plus the result of all previous steps	The command text changes, or any earlier step changed
COPY, ADD	A checksum of the contents of the copied files	Any copied file's content changes (file modification times are ignored)
ENV, ARG	The values	A value changes; this also affects every later step
Any step	Its parent	Any earlier step was rebuilt

The most important rule: Once a step is rebuilt, every step after it is rebuilt too, even if those steps did not change. A cache miss cascades down the rest of the stage.

The RUN Cache Does Not Check the Outside World

BuildKit caches RUN based on the command text, not on what the command would download today. RUN apt-get update or RUN pip install requests will happily reuse a result from months ago. To get fresh packages, pin versions and change them deliberately, rebuild with --no-cache, or use --pull to refresh the base image. Combining apt-get update and apt-get install in one RUN ensures the package index is never older than the install step.

Write Cache-Friendly Dockerfiles

Order Instructions From Least to Most Frequently Changing

Application source code changes on almost every commit; dependencies change occasionally; the base image and system packages change rarely. Put them in that order, from bottom (rarely changes) to top (changes often):

# Slow: any code change re-runs pip install

FROM python:3.13-slim

WORKDIR /app

COPY . .

RUN pip install --no-cache-dir -r requirements.txt

CMD ["python", "app.py"]

# Fast: dependencies are reinstalled only when requirements.txt changes

FROM python:3.13-slim

WORKDIR /app

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD ["python", "app.py"]

The same pattern applies to every ecosystem:

Language	Copy first (dependency manifest)	Then install	Then copy the source
Node.js	package.json, package-lock.json	npm ci	COPY . .
Python	requirements.txt or pyproject.toml and lock file	pip install	COPY . .
Go	go.mod, go.sum	go mod download	COPY . .
Java (Maven)	pom.xml	mvn dependency:go-offline	COPY src ./src

Use a .dockerignore File

Everything sent in the build context can affect COPY . . and invalidate its cache. A .dockerignore file keeps unrelated changes, such as Git metadata, local virtual environments and logs, from triggering rebuilds, and keeps secrets out of the image:

.git

node_modules

__pycache__

.venv

*.log

.env

Dockerfile*

docker-compose*.yaml

Use Cache Mounts for Package Managers

Even with good ordering, changing one dependency reinstalls them all. A cache mount keeps the package manager's download cache between builds without storing it in the image:

RUN --mount=type=cache,target=/root/.cache/pip \

pip install -r requirements.txt

Package manager	Cache mount target
pip	/root/.cache/pip
npm	/root/.npm
Go modules and build cache	/go/pkg/mod and /root/.cache/go-build
Maven	/root/.m2
apt	/var/cache/apt and /var/lib/apt (with sharing=locked)

Targets assume the build runs as root; adjust the paths if your build stage uses another user. Do not combine a pip cache mount with --no-cache-dir, which would disable the cache you just mounted.

Keep Layers Purposeful, Not Minimal

Fewer layers is not automatically better. Merging everything into one giant RUN makes any change rebuild everything. Group commands that belong together and change together, and separate things that change at different rates. Note that the older docker build --squash option is not supported by BuildKit; multi-stage builds are the modern way to produce a lean final image.

Sharing the Build Cache in CI

CI runners usually start with an empty machine, so the local build cache is lost on every run. BuildKit can export the cache to an external location and import it on the next build:

docker buildx build \

--cache-from type=registry,ref=registry.example.com/api:buildcache \

--cache-to type=registry,ref=registry.example.com/api:buildcache,mode=max \

-t registry.example.com/api:1.4.0 --push .

Cache backend	Where the cache is stored	Typical use
inline	Inside the pushed image itself	Simple setups; supports only mode=min
registry	A separate cache image in a registry	Most CI systems
local	A directory on disk	Self-hosted runners with persistent storage
gha	The GitHub Actions cache service	GitHub Actions workflows

mode=min (the default) exports only the layers in the final image; mode=max also exports intermediate stages, giving more cache hits for multi-stage builds at the cost of a larger cache. With the default docker build driver, these backends require the containerd image store, which is the default on fresh Docker Engine 29 installations; otherwise use a docker-container builder (docker buildx create --use).

Inspecting and Managing Layers and Cache

Command	What it shows or does
docker image history <image>	Layers, the instruction that created each, and their sizes
docker image inspect <image>	Layer digests and image configuration
docker diff <container>	Changes in a container's writable layer
docker ps -s	Writable layer size of each container
docker system df -v	Shared versus unique disk use for images, containers and build cache
docker buildx du	Detailed build cache usage
docker builder prune	Remove unused build cache
docker builder prune --filter until=72h	Remove build cache not used in the last 72 hours
docker build --no-cache .	Build without using the cache
dive <image>	Third-party tool that browses each layer's files and highlights wasted space

To stop the build cache growing without limit on a build server, configure BuildKit garbage collection in /etc/docker/daemon.json and restart Docker:

{

"builder": {

"gc": {

"enabled": true,

"defaultKeepStorage": "20GB"

}

Common Layer and Cache Problems

Symptom	Cause	Fix
Every code change reinstalls all dependencies	COPY . . comes before the install step	Copy dependency manifests first, install, then copy the source
Image is huge even after deleting files	Files were deleted in a later layer than they were added	Clean up in the same RUN, or use a multi-stage build
Build uses outdated packages	Cached RUN steps do not check for new versions	Pin versions, use --pull, or rebuild with --no-cache
Cache never hits in CI	Fresh runners have no local cache	Use --cache-from and --cache-to with a registry or gha backend
COPY . . invalidates the cache unexpectedly	Unrelated files (logs, .git) changed in the build context	Add them to .dockerignore
Container writes are slow, disk fills quickly	Heavy writes to the container layer trigger copy-up	Use volumes for data and logs
"max depth exceeded" error	Too many layers stacked in the image	Combine related steps or use a multi-stage build
Host disk fills on a build server	Build cache is never cleaned	Run docker builder prune regularly or configure BuildKit GC

Frequently Asked Questions

What is the difference between an image layer and a container layer?

Image layers are read-only and shared by every container created from the image. The container layer is a thin read-write layer created for each container; it holds all changes that container makes and is deleted when the container is removed.

Does each RUN instruction create a new layer?

Yes. Every RUN, COPY and ADD instruction creates a layer. Metadata instructions such as ENV, CMD and EXPOSE change only the image configuration and do not add filesystem content.

Why does my build say CACHED for apt-get update?

Because BuildKit caches RUN steps by their command text and position, not by what the command would download today. Combine apt-get update with apt-get install in one RUN, and rebuild with --no-cache or --pull when you need fresh packages.

Is OverlayFS the only union filesystem Docker supports?

On modern Linux, OverlayFS (through the overlay2 driver or containerd's overlayfs snapshotter) is the default and recommended option. Others exist for special cases, such as btrfs, zfs, fuse-overlayfs for some rootless setups, and vfs, which copies every layer and is very slow. Older options such as AUFS and devicemapper have been removed.

Do more layers make containers slower?

Not noticeably for normal images. Reading a file looks up the layers from top to bottom, and the kernel caches lookups efficiently. Layer count matters more for build cache efficiency and maintainability than for runtime speed, although there is a practical upper limit on how many layers can be stacked.

How do I see which layer makes my image large?

Run docker image history --no-trunc <image> to see the size of each layer and the instruction that created it, or use the dive tool to browse the files in each layer and find wasted space.

Conclusion and Next Steps

Layers, OverlayFS and caching are three views of one design. Each Dockerfile instruction produces an immutable, content-addressed layer; OverlayFS stacks those layers under a thin writable layer using copy-on-write, so containers start instantly and share disk space; and the same content addressing lets Docker cache layers during builds, pulls and pushes. Order your Dockerfile from stable to volatile, clean up in the same layer, use .dockerignore, cache mounts and multi-stage builds, and export the build cache in CI. Your images will be smaller, your builds faster and your deployments lighter.

Continue the Docker course with these tutorials on BitCodeMatrix:

Docker Architecture Explained — where images, layers and containerd fit in the bigger picture
Complete Guide to Multi-Stage Docker Builds — keep build tools out of your final image
Build Smaller, Faster and More Secure Docker Images — practical size reduction techniques
Docker Volumes and Persistent Storage — where write-heavy data should live instead of the container layer