kubernetes

Kubernetes Service Discovery: How Pods Find Other Pods

Learn how Kubernetes Pods discover Services using DNS, selectors and EndpointSlices, including namespaces, headless Services, troubleshooting and practical labs.

By Shubhankar Tripathi • • 5 min read

Kubernetes Service Discovery: How Pods Find Other Pods

A practical guide to Kubernetes DNS, Services, EndpointSlices, namespaces, headless Services, troubleshooting and real-world application communication.

Introduction

In a Kubernetes application, Pods frequently need to communicate with other Pods. A frontend may call an API, an API may call a database, and one microservice may call another microservice. The challenge is that Pod IP addresses are not stable application identities. Pods can be recreated, replaced during deployments, or scheduled onto different Nodes.

Kubernetes solves this problem by providing Service discovery. Instead of requiring an application to track individual Pod IP addresses, applications can normally communicate with a Service by name. Kubernetes DNS provides the name-resolution mechanism, while the Service and its backend EndpointSlices provide the relationship between the stable Service identity and the current backend endpoints.

This article explains the complete path: application → DNS name → Service → EndpointSlice → backend Pod, and shows how to troubleshoot each layer.

1. What Is Kubernetes Service Discovery?

Service discovery is the mechanism by which an application finds a network endpoint for another application or service. In Kubernetes, the most common pattern is to create a Service in front of a set of Pods and allow client Pods to discover that Service through cluster DNS.

• The Service provides a stable logical endpoint.

• The Service selector identifies matching Pods when a selector is configured.

• The EndpointSlice API tracks the network endpoints backing the Service.

• Kubernetes DNS publishes DNS records for Services and Pods.

• A service-proxy implementation routes Service traffic toward backend endpoints.

Kubernetes documentation describes the Service API as providing a stable, long-lived IP address or hostname for a service implemented by one or more backend Pods, even though the individual Pods can change over time.

2. The Core Problem: Pod IPs Can Change

Suppose an API application has three Pods:

api-7d8c9f6b5d-a1b2c   10.244.1.10
api-7d8c9f6b5d-c3d4e   10.244.2.14
api-7d8c9f6b5d-f5g6h   10.244.3.21


A frontend application could technically connect directly to these IP addresses, but this is not a good service-discovery model. If a Pod is deleted and recreated, its replacement may receive a different IP address.

The application should therefore depend on a stable logical identity rather than individual Pod IPs.

3. The Kubernetes Service Discovery Architecture

Component

Responsibility

Pod

Runs the application and has a Pod IP.

Service

Provides a stable logical endpoint for a group of backend Pods.

Service selector

Identifies Pods that belong to the Service when a selector is used.

EndpointSlice

Stores the network endpoints associated with a Service.

Kubernetes DNS

Provides DNS records that allow workloads to discover Services by name.

Service proxy/dataplane

Routes Service traffic toward appropriate backend endpoints.


The important distinction is that DNS discovery and traffic routing are related but are not the same operation. DNS answers the question, “What network identity should I use?” The Service dataplane then handles traffic sent to the Service endpoint.

4. How a Pod Finds a Service

Consider a Service named `orders` in the `production` namespace. A Pod in the same namespace can normally use the short name:

orders


A client can also use the namespace-qualified name:

orders.production


The fully qualified Service DNS form is:

orders.production.svc.cluster.local


The cluster domain is configurable; `cluster.local` is the common default rather than a value that every cluster must use.

Kubernetes documentation states that a normal Service receives an A and/or AAAA DNS record that resolves to the Service's cluster IP, while a headless Service resolves to the set of selected Pod IPs.

5. DNS Search Domains Make Short Names Practical

Kubernetes configures Pod DNS settings so applications can commonly resolve Services using short names. A client Pod's DNS search list normally includes its own namespace and the cluster's default domain.

cat /etc/resolv.conf


You may see search entries similar to:

search default.svc.cluster.local svc.cluster.local cluster.local
nameserver 10.96.0.10
options ndots:5


The exact values depend on the cluster configuration. The important operational point is that a query such as `orders` is interpreted using the Pod's DNS search configuration.

6. Namespace Matters

A Service name without a namespace is resolved relative to the namespace of the client Pod. This is one of the most common causes of DNS confusion.

For example, assume:

Object

Namespace

Client Pod

frontend

orders Service

backend


From the `frontend` namespace, this query looks for `orders` in the `frontend` namespace:

nslookup orders


To reach the Service in `backend`, specify the namespace:

nslookup orders.backend


Or use the fully qualified name:

nslookup orders.backend.svc.cluster.local


7. What Happens When You Call a Service Name?

Suppose an application executes:

curl http://orders:8080


1. The application asks the Pod's configured DNS resolver to resolve `orders`.

2. Kubernetes cluster DNS processes the Service DNS query.

3. For a normal Service, the DNS response points to the Service's cluster IP.

4. The client sends traffic toward that Service IP and port.

5. The Service dataplane selects an appropriate backend endpoint.

6. Traffic is delivered to one of the Pods represented by the Service's EndpointSlices.

The exact packet-processing path depends on the cluster's networking implementation. Kubernetes provides kube-proxy as a default Service proxy implementation, while some Pod network implementations provide their own Service proxy integrated with their networking stack.

8. Services Use Selectors to Find Backend Pods

A typical Service selects Pods using labels.

apiVersion: v1
kind: Service
metadata:
  name: orders
  namespace: production
spec:
  selector:
    app: orders
  ports:
    - name: http
      port: 8080
      targetPort: 8080
  type: ClusterIP


The backend Pods might have:

apiVersion: v1
kind: Pod
metadata:
  name: orders-pod
  namespace: production
  labels:
    app: orders


The Service selector `app: orders` matches the Pod label. The control plane then manages EndpointSlices representing the matching network endpoints.

9. EndpointSlices: The Backend Map

EndpointSlices are a critical part of modern Kubernetes Service networking. They represent sets of network endpoints associated with Services and allow Kubernetes to scale Service backend information efficiently.

For a Service with a selector, the control plane automatically creates and manages EndpointSlices containing references to Pods matching the selector. EndpointSlices can include endpoint conditions such as `ready`, `serving`, and `terminating`.

kubectl get endpointslices
kubectl get endpointslices -l kubernetes.io/service-name=orders
kubectl describe endpointslice <endpoint-slice-name>


Kubernetes documentation identifies EndpointSlices as the source of truth for kube-proxy when it determines how to route internal Service traffic.

10. Service Discovery Is Not the Same as Pod Discovery

A common misconception is that every application discovers another application by directly discovering Pod IP addresses. The normal Kubernetes microservice pattern is different.

Approach

What the client discovers

Typical use

Normal Service

Service cluster IP through DNS

Stable application endpoint

Headless Service

Set of backend Pod IPs through DNS

Client-side discovery or stateful workloads

Direct Pod IP

Individual Pod address

Specialized troubleshooting or tightly controlled cases


For most application-to-application communication, use a Service name rather than relying on individual Pod IPs.

11. Normal Service vs Headless Service

A normal Service has a cluster IP. Its DNS record resolves to the Service's virtual IP.

apiVersion: v1
kind: Service
metadata:
  name: api
spec:
  selector:
    app: api
  ports:
    - port: 80
      targetPort: 8080


A headless Service sets `clusterIP: None`.

apiVersion: v1
kind: Service
metadata:
  name: database
spec:
  clusterIP: None
  selector:
    app: database
  ports:
    - port: 5432
      targetPort: 5432


For a headless Service, DNS returns the IP addresses of the selected Pods instead of a single cluster IP. This is useful when the client needs to know individual backend endpoints, including common patterns used with StatefulSets.

Headless Service discovery therefore changes the discovery model: the client receives a set of endpoint addresses instead of one virtual Service IP.

12. Why StatefulSets Commonly Use Headless Services

StatefulSets provide stable Pod identities. A headless Service can provide DNS records associated with those stable identities, making it possible for applications to address individual StatefulSet Pods.

<pod-hostname>.<service-name>.<namespace>.svc.cluster.local


The exact DNS names depend on the StatefulSet Pod hostname, the headless Service name, namespace and cluster domain. This pattern is useful for distributed systems where individual members need stable identities rather than anonymous load-balanced access.

13. Service Ports vs Container Ports

Service discovery does not remove the need to configure ports correctly.

Field

Meaning

Service `port`

Port exposed by the Service.

Service `targetPort`

Port on the selected backend endpoint.

Container `containerPort`

A declaration associated with a container port; it does not itself publish the port outside the Pod.


ports:
  - port: 80
    targetPort: 8080


A client can therefore call the Service on port 80 while the application listens on port 8080 in the backend Pod.

14. Readiness and Service Discovery

Service endpoint availability is closely related to Pod readiness. EndpointSlice conditions contain readiness and serving information for endpoints. A Pod that is not ready should generally not receive normal Service traffic.

This is why readiness probes are important in production applications. A Pod may be running while its application is still initializing or unable to serve requests.

readinessProbe:
  httpGet:
    path: /ready
    port: 8080
  periodSeconds: 5
  failureThreshold: 3


A readiness failure is different from a liveness failure: readiness controls whether the workload should receive traffic, while liveness can trigger a container restart when the liveness probe repeatedly fails.

15. DNS Resolution and Traffic Routing Are Different Layers

Layer

Question

Typical Kubernetes component

Discovery

What name/address represents this Service?

Cluster DNS

Service membership

Which Pods are backend endpoints?

Service selector + EndpointSlice controller

Traffic routing

Which backend receives this Service traffic?

Service proxy/dataplane

Application

Can the backend actually process the request?

Application + readiness/health behavior


This separation is extremely useful during troubleshooting. If DNS resolution fails, investigate DNS. If DNS resolves but the Service has no endpoints, investigate selectors and endpoint conditions. If endpoints exist but traffic fails, investigate the application, network policy, service dataplane and surrounding network path.

16. Debugging Service Discovery

Step 1: Verify the Service

kubectl get svc -n production orders
kubectl describe svc -n production orders


Step 2: Verify the Service selector

kubectl get svc -n production orders -o yaml
kubectl get pods -n production --show-labels


Step 3: Check EndpointSlices

kubectl get endpointslices -n production   -l kubernetes.io/service-name=orders


If there are no expected endpoints, compare the Service selector with the Pod labels. Kubernetes' Service debugging documentation specifically identifies selector mismatch as a common cause of an empty endpoint list.

Step 4: Test DNS from inside the cluster

kubectl exec -it <client-pod> -- nslookup orders
kubectl exec -it <client-pod> -- nslookup orders.production
kubectl exec -it <client-pod> -- nslookup orders.production.svc.cluster.local


Step 5: Test the Service

kubectl exec -it <client-pod> --   curl -v http://orders:8080


Step 6: Test the Pod directly

If appropriate for troubleshooting, test one backend Pod directly. This helps determine whether the problem is the application itself or the Service path.

kubectl get pods -n production -o wide


17. Troubleshooting DNS Failures

If a Service name cannot be resolved, check the DNS layer before changing the Service.

• Confirm the client Pod is using the expected DNS configuration.

• Confirm the Service exists in the namespace you are querying.

• Use a namespace-qualified name when the Service is in another namespace.

• Check the cluster DNS deployment and its logs.

• Verify that the DNS service can access the Service and EndpointSlice resources.

kubectl get pods -n kube-system
kubectl get svc -n kube-system
kubectl logs -n kube-system -l k8s-app=kube-dns


In clusters using CoreDNS, the DNS deployment must have sufficient permissions to list and watch relevant resources such as Services and EndpointSlices. Kubernetes provides a dedicated DNS troubleshooting guide for this scenario.

18. Troubleshooting an Empty EndpointSlice

Suppose:

kubectl get endpointslices   -l kubernetes.io/service-name=orders


shows no expected endpoints.

1. Inspect the Service selector.

2. Inspect the labels on the Pods.

3. Confirm the Pods are in the same namespace as the Service.

4. Check Pod readiness and termination state.

5. Confirm the Service port and targetPort are correct.

kubectl get svc orders -o yaml
kubectl get pods --show-labels
kubectl get pods -o wide


19. Cross-Namespace Service Discovery

Namespaces provide logical boundaries for Kubernetes resources. Service DNS names include the namespace so that applications can intentionally communicate across namespaces.

# Same namespace
curl http://orders:8080

# Different namespace
curl http://orders.backend:8080

# Fully qualified
curl http://orders.backend.svc.cluster.local:8080


This is particularly useful in environments where teams or environments are separated into namespaces, for example `frontend`, `backend`, `staging` and `production`.

20. Service Discovery in a Microservices Architecture

Consider a typical application:

Service

Namespace

Consumes

frontend

production

api

api

production

orders, users

orders

production

database

users

production

database

database

production

Internal only


The frontend does not need to know the IP addresses of API Pods. It can call `api`. The API does not need to know the IP addresses of order Pods. It can call `orders`. Kubernetes DNS and Services provide stable discovery while the backend Pods can change.

21. What Happens During a Deployment?

During a Deployment rollout, old Pods can be replaced by new Pods. Their IP addresses can change. A Service remains the stable logical endpoint, while EndpointSlices are updated as backend endpoints change.

1. Deployment creates or updates Pods.

2. New Pods become ready according to their readiness behavior.

3. EndpointSlice information changes as endpoint state changes.

4. Service traffic is routed using the current backend endpoint set.

5. Old Pods can be removed without requiring clients to change the Service name.

This is one of the main reasons Service-based discovery is more resilient than hard-coding Pod IP addresses.

22. Headless Service and Client-Side Load Balancing

With a headless Service, DNS can return multiple backend Pod IPs. The application becomes responsible for consuming those addresses appropriately. Some distributed applications use this pattern when they need awareness of individual members rather than a single virtual Service endpoint.

Do not choose a headless Service merely because it sounds faster. It changes the discovery and routing model and should be selected because the application requires endpoint-level awareness.

23. Common Mistakes

Mistake 1: Using Pod IPs in application configuration

Pod IPs are not intended to be stable application identities. Use a Service name for normal application communication.

Mistake 2: Forgetting the namespace

A short Service name resolves within the client's namespace. Use `service.namespace` or the full DNS name for cross-namespace communication.

Mistake 3: Selector mismatch

A Service can exist even when its selector matches no Pods. Always inspect labels and EndpointSlices.

Mistake 4: Confusing DNS with routing

Successful DNS resolution does not guarantee that the backend application is reachable or healthy.

Mistake 5: Assuming every Service returns Pod IPs

Normal Services resolve to the Service cluster IP. Headless Services resolve to the selected Pod IPs.

Mistake 6: Testing from the wrong namespace

A DNS query such as `orders` can fail simply because the client is in a different namespace.

24. Security Considerations

Service discovery makes communication easier, but it is not a security authorization mechanism. A Service name being resolvable does not by itself mean that an application should be allowed to communicate with another application.

• Use NetworkPolicy where traffic restrictions are required and supported by the cluster network implementation.

• Avoid exposing internal Services externally without a business or operational requirement.

• Use TLS or application-level authentication where sensitive traffic requires it.

• Treat namespace separation as organizational and resource isolation, not as a complete network security boundary.

• Review DNS and Service access as part of the application's overall security model.

25. Hands-On Lab: Discover a Service from Another Pod

Create a Deployment and a ClusterIP Service:

kubectl create deployment web --image=nginx:1.27 --replicas=3

kubectl expose deployment web   --name=web   --port=80   --target-port=80   --type=ClusterIP

kubectl get svc web
kubectl get endpointslices -l kubernetes.io/service-name=web


Create a temporary client Pod:

kubectl run dns-client   --rm -it   --restart=Never   --image=busybox:1.36   -- sh


From the client Pod, resolve the Service:

nslookup web
nslookup web.default.svc.cluster.local


Then test the Service:

wget -qO- http://web:80


Finally, inspect the backend endpoints:

kubectl get endpointslices   -l kubernetes.io/service-name=web


Now delete one backend Pod and observe the endpoint set again. The Service name remains unchanged while the backend Pod set changes.

26. Practical Architecture Example

Imagine a production platform with a public frontend, internal API services and a database.

User
  |
  v
External Entry Point
  |
  v
frontend Service
  |
  v
frontend Pods
  |
  v
api Service
  |
  v
API Pods
  |
  v
orders Service
  |
  v
Orders Pods
  |
  v
database Service
  |
  v
Database Pods


The important design principle is that applications communicate using stable Service identities. The underlying Pod IPs can change without forcing every client configuration to change.

27. Interview Questions

What problem does Kubernetes Service discovery solve?

Answer: It allows workloads to find stable application endpoints even though backend Pods can be replaced and their IP addresses can change.

How does a Pod normally find a Service?

Answer: Through Kubernetes DNS using the Service name.

What does a normal Service DNS record resolve to?

Answer: The Service's cluster IP, using an A and/or AAAA record depending on the Service IP families.

What does a headless Service DNS record resolve to?

Answer: The IP addresses of the selected backend Pods rather than a Service cluster IP.

Why is namespace important in Service discovery?

Answer: A short Service name is resolved in the client's namespace; a different namespace must be specified for cross-namespace access.

What is EndpointSlice?

Answer: A Kubernetes API resource representing sets of network endpoints associated with a Service.

What is a common cause of a Service with no backend endpoints?

Answer: A Service selector does not match the labels on the intended Pods.

Does DNS perform the actual application load balancing?

Answer: DNS discovery and Service traffic routing are separate layers. Normal Service DNS returns the Service cluster IP, after which the Service dataplane routes traffic toward backend endpoints.

Why are readiness probes important for Service traffic?

Answer: Readiness state influences whether an endpoint should receive normal Service traffic.

When would you use a headless Service?

Answer: When clients need endpoint-level discovery, such as certain StatefulSet or distributed-system patterns.

28. Production Checklist

• Use Service names instead of hard-coded Pod IP addresses for normal application communication.

• Confirm the client and Service namespaces when debugging DNS.

• Verify Service selectors match intended Pod labels.

• Inspect EndpointSlices when a Service does not receive traffic.

• Check Pod readiness before assuming the backend is available.

• Verify Service port and targetPort values.

• Test DNS resolution from a real client Pod.

• Separate DNS troubleshooting from Service routing and application troubleshooting.

• Use headless Services only when endpoint-level discovery is actually required.

• Use NetworkPolicy and application authentication/TLS for security requirements.

29. Summary

Kubernetes Service discovery allows applications to communicate without depending on unstable Pod IP addresses. The normal pattern is simple: create a Service, select the appropriate backend Pods, and allow client workloads to discover the Service through Kubernetes DNS.

A normal Service gives clients a stable Service identity and cluster IP. EndpointSlices represent the current backend endpoints. The Service dataplane routes traffic toward those endpoints. A headless Service changes the model by allowing DNS to return the selected Pod addresses directly.

For production troubleshooting, remember the sequence: check the DNS name and namespace → inspect the Service → verify selectors → inspect EndpointSlices → check Pod readiness → test the application and network path. Understanding these layers turns Service discovery from a black box into a predictable Kubernetes networking workflow.

30. Related Kubernetes Topics

• Kubernetes Service Explained: ClusterIP, NodePort and LoadBalancer

• Kubernetes Pod Networking Explained

• Kubernetes DNS and CoreDNS Explained

• Kubernetes Ingress Explained

• Kubernetes Gateway API Explained

• Kubernetes NetworkPolicy Explained

• Kubernetes StatefulSet Explained

• Kubernetes Container Probes: Liveness, Readiness and Startup Probes

Official Kubernetes References

DNS for Services and Pods — https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/

EndpointSlices — https://kubernetes.io/docs/concepts/services-networking/endpoint-slices/

Services, Load Balancing, and Networking — https://kubernetes.io/docs/concepts/services-networking/

Debugging DNS Resolution — https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/

Debug Services — https://kubernetes.io/docs/tasks/debug/debug-application/debug-service/