kubernetes

EmptyDir, ConfigMap and Secret Volumes

Kubernetes EmptyDir, ConfigMap and Secret Volumes Explained

By Shubhankar Tripathi • • 5 min read

Kubernetes EmptyDir, ConfigMap and Secret Volumes Explained

bitcodematrix.com | Kubernetes Series

1. Introduction

Not every volume holds a database. Most Pods use a few small, Kubernetes-native volume types every day: an emptyDir for scratch space, a ConfigMap volume to deliver configuration files, and a Secret volume to deliver credentials and certificates. They are simple to declare, but they come with behaviours that surprise people: configuration that updates by itself in one case and never in another, files that turn out to be symbolic links, secrets that sit in memory, and directories that hide what the image shipped.

This article explains how these three volume types work, how to use them correctly, how they differ from environment variables, how updates propagate, how to secure them, and how to troubleshoot them. It also covers projected volumes, which combine several sources in one directory. It complements the earlier articles in this series on ephemeral storage and persistent storage.

2. The Three Volume Types at a Glance

Aspect

emptyDir

ConfigMap

Secret

Purpose

Scratch space and sharing between containers

Non-sensitive configuration

Sensitive data such as passwords, tokens and certificates

Content comes from

Nothing; starts empty

A ConfigMap object in the API

A Secret object in the API

Writable by the container

Yes

No, mounted read-only

No, mounted read-only

Backed by

Node disk, or memory if requested

Node-local storage managed by the kubelet

Memory (tmpfs), never written to the node's disk

Lifetime

The Pod on that node

The Pod; follows the object while mounted

The Pod; follows the object while mounted

Updates after start

Not applicable

Propagated to the volume, with a delay

Propagated to the volume, with a delay

Size limit

Node capacity, or sizeLimit

About 1 MiB per object

About 1 MiB per object

 

All three are Pod-scoped: they exist only as long as the Pod and are defined in the Pod's volumes section. ConfigMaps and Secrets must be in the same namespace as the Pod that mounts them.

3. emptyDir Volumes

3.1 How emptyDir Behaves

An emptyDir volume is created when a Pod is assigned to a node, and it starts empty. Every container in the Pod can mount it, at the same or at different paths. It has these properties:

  • It survives container restarts. If a container crashes and is restarted by the kubelet, the data in the emptyDir is still there.

  • It is deleted permanently when the Pod is removed from the node, whether because it was deleted, evicted or rescheduled elsewhere.

  • By default it lives on the node's disk and counts toward the Pod's ephemeral storage. With medium set to Memory it is a tmpfs and counts toward memory instead.

  • The sizeLimit field caps its size. If a disk-backed emptyDir exceeds it, the Pod is evicted.

  • It is created world-writable by default, so containers that run as non-root users can normally write to it. Use fsGroup in the Pod security context if you need specific group ownership.

3.2 A Typical Use: Sharing Data Between Containers

The classic pattern is an init container that prepares data and an application container that uses it, or a sidecar that exchanges files with the main container:

apiVersion: v1

kind: Pod

metadata:

  name: shared-demo

spec:

  initContainers:

  - name: prepare

    image: busybox:1.36

    command: ["sh", "-c", "echo prepared > /work/ready.txt"]

    volumeMounts:

    - name: work

      mountPath: /work

  containers:

  - name: app

    image: busybox:1.36

    command: ["sh", "-c", "cat /work/ready.txt; sleep 3600"]

    volumeMounts:

    - name: work

      mountPath: /work

  volumes:

  - name: work

    emptyDir:

      sizeLimit: 500Mi

3.3 Good and Bad Uses

Good uses

Poor uses

Temporary working files, sort buffers and build output

Primary copies of data that you cannot recreate

Caches that can be rebuilt

Database files

Handing files from an init container to the app

Anything that must survive Pod rescheduling

Scratch directories when the root file system is read-only

Uploaded user content

Checkpoints that only speed up recovery

Message queues with delivery guarantees

 

Always set a sizeLimit, so that a runaway writer produces a clear eviction instead of filling the node. Remember the difference between the media: the Memory variant is fast but its contents consume the container's memory allowance and can cause an out-of-memory kill.

4. ConfigMap Volumes

4.1 Creating a ConfigMap

A ConfigMap holds configuration as key-value pairs. The values can be short strings or whole files. You can write one as YAML, or create it from files or literals:

apiVersion: v1

kind: ConfigMap

metadata:

  name: web-config

  namespace: shop

data:

  LOG_LEVEL: info

  nginx.conf: |

    server {

      listen 8080;

      location / {

        root /usr/share/nginx/html;

      }

    }

kubectl create configmap web-config -n shop \

  --from-file=nginx.conf --from-literal=LOG_LEVEL=info

Keys must consist of letters, digits, dashes, underscores and dots, because in a volume they become file names. Binary content goes in the binaryData field.

4.2 Mounting a ConfigMap

spec:

  containers:

  - name: web

    image: nginx:1.27

    volumeMounts:

    - name: config

      mountPath: /etc/web

      readOnly: true

  volumes:

  - name: config

    configMap:

      name: web-config

      defaultMode: 0644

With this, the directory /etc/web contains one file per key: LOG_LEVEL and nginx.conf. To expose only some keys, or to give them different file names, use the items list:

  volumes:

  - name: config

    configMap:

      name: web-config

      items:

      - key: nginx.conf

        path: default.conf

The defaultMode field sets file permissions. In YAML, write it in octal, such as 0644. If you write the manifest in JSON, you must use the decimal equivalent. A Pod that references a missing ConfigMap does not start and reports a mount failure, unless you set optional to true on the volume.

4.3 What the Files Really Are

If you list the mounted directory, you will notice that the files are symbolic links:

kubectl exec <pod> -n shop -- ls -la /etc/web

The entries point into a hidden directory called ..data, which itself points to a timestamped directory that holds the real files. This is how the kubelet performs atomic updates: it writes a new timestamped directory and then switches the ..data link in one step, so an application never sees a half-written set of files. Most programs follow symlinks without noticing. Some tools that copy or check files, however, treat them differently, so use options that follow links when needed.

4.4 Updates Propagate, With Limits

When you edit a ConfigMap that is mounted as a volume, the kubelet eventually refreshes the files in running Pods, typically within a minute or two but not instantly. Three important exceptions exist:

  • subPath mounts never update. If you mount a single key using subPath, the file is a fixed copy and stays stale until the Pod restarts.

  • Environment variables never update. Values injected through env or envFrom are read once when the container starts.

  • Immutable ConfigMaps do not change at all. See Section 4.6.

Even when the files update, most applications read their configuration only at start-up, so they will not notice. Section 7 describes reload patterns.

4.5 The Directory-Hiding Trap and subPath

Mounting a volume at a path hides whatever the image already had in that directory. Mounting a ConfigMap at /etc/nginx, for example, makes the image's own files under /etc/nginx disappear. There are two common solutions:

  • Mount the ConfigMap at a new, dedicated directory such as /etc/web, and tell the application to read its configuration from there.

  • Use subPath with a single key to place one file into an existing directory without hiding its neighbours. The price is that this file will no longer receive updates.

    volumeMounts:

    - name: config

      mountPath: /etc/nginx/conf.d/default.conf

      subPath: nginx.conf

      readOnly: true

4.6 Immutable ConfigMaps

Setting immutable: true on a ConfigMap or Secret locks it. Its data can no longer be changed, only the object can be deleted and recreated. This protects against accidental edits that would break running Pods, and it reduces load on the API server, because the kubelet no longer needs to watch the object for changes. A good way to use this is to version the names, for example web-config-v3, and update the Deployment to refer to the new name, which triggers a normal rolling update.

4.7 Volume or Environment Variables?

Aspect

Volume

Environment variables

Best for

Whole files, structured configuration, certificates

A few simple settings

Updates in running Pods

Yes, except with subPath

No; needs a restart

Limits

One file per key, size limit applies

Large values are awkward; names must be valid variable names

Visibility

Readable files in the container

Visible to every process and often in diagnostic output

 

5. Secret Volumes

5.1 What a Secret Is, and Is Not

A Secret is like a ConfigMap for sensitive data. The API stores its values base64-encoded, but base64 is an encoding, not encryption. Anyone who can read the Secret can decode it, and by default the data is stored unencrypted in etcd, so you must enable encryption at rest and restrict access, as described in Section 8.

Kubernetes defines several Secret types, which add validation of expected keys:

Type

Purpose

Opaque

Generic key-value data, the default

kubernetes.io/tls

A certificate and private key, with the keys tls.crt and tls.key

kubernetes.io/dockerconfigjson

Credentials for pulling images from a private registry

kubernetes.io/basic-auth and kubernetes.io/ssh-auth

Credentials in a conventional format

 

5.2 Creating Secrets

kubectl create secret generic db-creds -n shop \

  --from-literal=username=app \

  --from-literal=password='change-me'

 

kubectl create secret tls web-tls -n shop \

  --cert=tls.crt --key=tls.key

When you write a Secret as YAML, you can use the stringData field to provide plain text and let the API server encode it. If you use the data field, the values must be base64. Encode with echo -n so that no trailing newline is included, which is a frequent cause of failed logins.

5.3 Mounting a Secret

spec:

  containers:

  - name: app

    image: example/app:1.0

    volumeMounts:

    - name: db-creds

      mountPath: /etc/secrets

      readOnly: true

  volumes:

  - name: db-creds

    secret:

      secretName: db-creds

      defaultMode: 0400

Each key becomes a file, for example /etc/secrets/username and /etc/secrets/password. Choose restrictive permissions with defaultMode, and make sure the user that the application runs as can read the files. When you also set fsGroup, Kubernetes adjusts group ownership and permissions of the files, so check the resulting modes if the application cannot read them.

5.4 Why Files Are Better Than Environment Variables

  • Environment variables are inherited by child processes and often show up in crash reports, debug endpoints and process listings. Files can be given tight permissions.

  • Files in a volume can be refreshed when the Secret changes, which allows credential rotation without a restart, if the application re-reads them.

  • The volume is held in memory (tmpfs) on the node, so the secret is not written to the node's disk.

Environment variables remain acceptable for low-risk values or for applications that cannot read files, but treat them as the weaker option.

5.5 TLS Example

volumes:

- name: tls

  secret:

    secretName: web-tls

    items:

    - key: tls.crt

      path: server.crt

    - key: tls.key

      path: server.key

      mode: 0400

Here the certificate and key are mounted under friendlier file names, with the private key restricted to its owner. Certificate renewal then becomes a matter of updating the Secret, for example through a certificate management controller, and letting the application reload the files.

6. Projected Volumes

A projected volume combines several sources into a single directory. It can include ConfigMaps, Secrets, downwardAPI data (such as Pod labels) and service account tokens, and so it saves you from mounting four volumes at four paths:

volumes:

- name: all-in-one

  projected:

    sources:

    - configMap:

        name: web-config

    - secret:

        name: db-creds

    - serviceAccountToken:

        path: token

        audience: api.example.com

        expirationSeconds: 3600

The serviceAccountToken source is especially valuable. It provides a short-lived, audience-bound token that the kubelet refreshes automatically, and that is the recommended way to give workloads an identity, instead of long-lived tokens stored in Secrets. File names across all sources must be unique inside the projected directory.

7. Making Applications Pick Up Changes

Updating a ConfigMap or Secret changes the files, but the application must notice. There are four common approaches:

  1. Rolling restart. Change the configuration and restart the Pods, for example with kubectl rollout restart deployment/<name>. It is simple and works with every application, and the rolling update keeps the service available.

  2. Checksum annotation. In Helm charts, put a hash of the ConfigMap into a Pod template annotation, so that any change to the content changes the template and triggers a rolling update automatically.

spec:

  template:

    metadata:

      annotations:

        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}

  1. Versioned immutable objects. Create web-config-v4 and point the Deployment at it. This gives you a clear history and easy rollback.

  2. In-place reload. Run an application that watches its files, or a small sidecar that detects changes and signals the main process, for example with SIGHUP. This avoids restarts but needs application support, and it must not rely on subPath mounts.

Choose restarts or versioned objects by default, because they are predictable and testable. Use in-place reloading only when restarts are costly, and test the reload path under realistic conditions.

8. Security Considerations

  • Control who can read Secrets. Anyone who can create a Pod in a namespace can mount any Secret from that namespace, so restrict both Secret access and Pod creation with RBAC, and avoid broad permissions to list or get Secrets.

  • Encrypt Secrets at rest in etcd, with the API server's encryption configuration or a KMS provider, and protect etcd backups.

  • Keep Secrets out of Git in plain form. Base64 offers no protection. Use encrypted-in-Git tools or an external secret manager that synchronises values into the cluster.

  • Consider external secret stores. Integrations such as the Secrets Store CSI driver or an external secrets operator fetch secrets from a vault or cloud service, and avoid long-lived copies in the API.

  • Mount read-only and limit scope. Mount only the Secrets and keys that a container needs, with restrictive file modes.

  • Do not log them. Avoid printing configuration files or environment variables that contain secrets, and be careful with debugging output.

  • Use short-lived credentials where possible, such as projected service account tokens, and rotate long-lived ones regularly.

Kubernetes also limits exposure on the node side: with the Node authorizer, a kubelet can read only the Secrets that are referenced by Pods scheduled to its own node, and Secret volumes are kept in memory so that they do not persist on disk after the Pod is gone.

9. Inspecting and Operating These Volumes

kubectl get configmap web-config -n shop -o yaml

kubectl get secret db-creds -n shop -o jsonpath='{.data.password}' | base64 -d

kubectl exec <pod> -n shop -- ls -la /etc/web

kubectl exec <pod> -n shop -- cat /etc/web/nginx.conf

kubectl describe pod <pod> -n shop

The describe output lists each volume and its source, and shows mount errors in the events. Be careful when decoding Secrets in shared terminals or scripts, because the values then appear in shell history and screen output.

A few operational habits help. Name objects after the application and purpose, and version them when you use the immutable pattern. Keep an eye on the size of objects, because the roughly 1 MiB limit applies to each ConfigMap and Secret, and large values belong in a persistent volume or an external store. Finally, remember that the kubelet watches every mounted ConfigMap and Secret, so thousands of objects can add load, and immutable objects reduce this.

10. Troubleshooting

Symptom

Likely cause and check

Pod stuck in ContainerCreating, configmap or secret not found

The object is missing or in another namespace. Create it in the Pod's namespace, fix the name, or set optional if the Pod can run without it.

Application does not see the new configuration

It reads configuration only at start-up, the file is a subPath mount, the value came from an environment variable, or the object is immutable. Restart the Pods or use a reload pattern.

Changes appear only after a delay

Normal. Propagation takes time. Wait a minute or two, and do not rely on instant updates.

Existing files in a directory disappeared

The volume was mounted over it. Mount at a different path, or use subPath for individual files.

Permission denied reading a mounted file

The defaultMode or file mode is too strict for the user that the process runs as, or fsGroup changed ownership. Review runAsUser, fsGroup and mode.

A tool fails on mounted files

The files are symbolic links. Use options that follow links, or copy the files with link dereferencing.

Login fails with a correct-looking password

The Secret value was encoded with a trailing newline. Recreate it with echo -n or use stringData.

TLS errors after rotation

Certificate and key do not match, or the application did not reload. Verify the pair and restart or reload.

Object rejected as too large

It exceeds the size limit of about 1 MiB. Split it, or move the data to a volume or external store.

Data in an emptyDir vanished

The Pod was rescheduled or evicted, which deletes it. Use a persistent volume for data that must survive.

Pod evicted and an emptyDir was large

The emptyDir exceeded its sizeLimit or the node ran low on ephemeral storage. Set limits and size disks.

Container OOM-killed with a Memory emptyDir

tmpfs usage counts toward container memory. Reduce writes or use a disk-backed emptyDir.

 

11. Best Practices

  • Use emptyDir only for data that you can recreate, and always set a sizeLimit.

  • Keep configuration out of images, and mount ConfigMaps at dedicated directories to avoid hiding image files.

  • Prefer volumes over environment variables for files and for anything that must change without a restart, and treat subPath as a trade-off because it blocks updates.

  • Use immutable, versioned ConfigMaps and Secrets for predictable rollouts and fewer API watches.

  • Mount Secrets as read-only files with restrictive modes, and avoid exposing them as environment variables where you can.

  • Enable encryption at rest, apply least-privilege RBAC, and never commit unencrypted Secrets to version control.

  • Use external secret managers and short-lived credentials for production workloads that handle sensitive data.

  • Trigger rolling restarts automatically when configuration changes, through checksums or versioned names.

  • Test configuration changes and rotation in a non-production environment, including the application's reload behaviour.

12. Conclusion

emptyDir, ConfigMap and Secret volumes are the workhorses of Pod configuration. An emptyDir gives containers a shared scratch area that lives and dies with the Pod. A ConfigMap volume turns configuration into files that can be updated atomically. A Secret volume does the same for sensitive data, held in memory and protected by permissions and RBAC.

Most problems come from a handful of misunderstandings: subPath and environment variables do not update, mounting a volume hides the directory beneath it, base64 is not encryption, and applications must be told to reload. Keep those in mind, version your configuration, secure your secrets properly and bound your scratch space, and these simple volume types will serve your workloads reliably.