## Why Kubernetes Egress Control Usually Fails the First Time
Most Kubernetes security conversations start with ingress: how users reach an application, how TLS is terminated, which load balancer is exposed, and whether an ingress controller is configured correctly. That is important, but it misses a large part of the risk. Once an attacker lands inside a workload, outbound access often determines how far they can go.
Unrestricted pod egress allows compromised containers to reach cloud metadata services, internal APIs, databases in other namespaces, package repositories, command-and-control infrastructure, and third-party SaaS platforms. In a flat cluster network, a small application vulnerability can become a much larger incident.
Kubernetes NetworkPolicy gives teams a native way to restrict pod-to-pod and pod-to-external traffic. The problem is that the first attempt at egress filtering often breaks something fundamental: DNS resolution, service discovery, monitoring, admission webhooks, object storage access, container registry access, or application calls to external APIs.
This article focuses on a practical pattern: how to build Kubernetes egress policies that meaningfully reduce risk while preserving the traffic applications actually need.
## What NetworkPolicy Can and Cannot Do
Before writing YAML, it is important to understand the boundaries of the tool.
Kubernetes NetworkPolicy is enforced by the cluster networking plugin, not by Kubernetes itself. If your CNI does not implement NetworkPolicy, the objects may be accepted by the API server but have no effect. Common production CNIs such as Calico, Cilium, Antrea, and many managed Kubernetes networking layers support NetworkPolicy, but the details and extensions vary.
Portable Kubernetes NetworkPolicy supports rules based on:
– Pod selectors
– Namespace selectors
– IP blocks
– Ports and protocols
– Ingress and egress direction
Native NetworkPolicy does not understand domain names such as api.stripe.com or login.microsoftonline.com. Some CNIs add FQDN-aware policy extensions, but those are vendor-specific. For a portable baseline, assume that standard NetworkPolicy controls IPs, labels, and ports.
There are three behavioral rules that matter most:
1. Pods are non-isolated by default.
2. A pod becomes isolated for egress when a NetworkPolicy selecting that pod includes Egress in policyTypes.
3. NetworkPolicies are additive. If any matching policy allows the traffic, the traffic is allowed.
That additive behavior is powerful, but it also means you need a disciplined policy model. A pile of one-off exceptions becomes hard to audit quickly.
## A Production-Safe Egress Strategy
A good egress design usually follows this order:
1. Inventory real outbound dependencies.
2. Apply a default deny egress policy to a small namespace first.
3. Explicitly allow DNS.
4. Allow required in-cluster services.
5. Allow approved external destinations.
6. Add observability before scaling enforcement.
7. Promote the pattern namespace by namespace.
Avoid applying a cluster-wide default deny on Friday afternoon. Kubernetes makes it easy to deploy a restrictive policy in seconds, but troubleshooting broken service discovery across dozens of namespaces is not a good incident response exercise.
## Step 1: Confirm NetworkPolicy Enforcement
Start by confirming that your CNI actually enforces NetworkPolicy. A simple test namespace is safer than assumptions.
“`bash
kubectl create namespace netpol-test
kubectl run client -n netpol-test –image=curlimages/curl:8.10.1 — sleep 3600
kubectl run server -n netpol-test –image=nginx:1.27 –port=80
kubectl expose pod server -n netpol-test –port=80
“`
Test connectivity before any policy:
“`bash
kubectl exec -n netpol-test client — curl -m 3 http://server
“`
Now apply a default deny egress policy:
“`yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-egress
namespace: netpol-test
spec:
podSelector: {}
policyTypes:
– Egress
egress: []
“`
Apply it:
“`bash
kubectl apply -f default-deny-egress.yaml
kubectl exec -n netpol-test client — curl -m 3 http://server
“`
If the request still succeeds, your cluster may not be enforcing NetworkPolicy, or your CNI may require additional configuration. Do not proceed to a production rollout until this is resolved.
## Step 2: Build the Baseline Default Deny
A namespace-level default deny egress policy is the foundation. It selects all pods in the namespace and allows no outbound traffic unless another policy permits it.
“`yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-egress
namespace: payments
spec:
podSelector: {}
policyTypes:
– Egress
egress: []
“`
This policy should usually be installed per application namespace, not blindly across the whole cluster. System namespaces such as kube-system, monitoring, ingress, and service mesh namespaces often need special handling.
A useful convention is to label namespaces by policy maturity:
“`bash
kubectl label namespace payments egress-policy=restricted
kubectl label namespace dev-sandbox egress-policy=observe
“`
The label does not enforce anything by itself, but it makes it easier to report, automate, and review policy rollout.
## Step 3: Allow DNS the Right Way
DNS is the most common thing teams accidentally break. Without DNS, applications may fail in ways that look unrelated to networking: database clients time out, OAuth flows fail, service discovery stops, and health checks go red.
First, inspect how DNS is labeled in your cluster:
“`bash
kubectl get pods -n kube-system –show-labels | grep -E ‘coredns|kube-dns’
kubectl get svc -n kube-system
“`
Many clusters expose DNS as a service named kube-dns in the kube-system namespace, backed by CoreDNS pods. Labels vary by distribution, so verify before copying examples.
A common policy to allow DNS from the payments namespace looks like this:
“`yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-dns-egress
namespace: payments
spec:
podSelector: {}
policyTypes:
– Egress
egress:
– to:
– namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
– protocol: UDP
port: 53
– protocol: TCP
port: 53
“`
Why include TCP 53? Most DNS queries use UDP, but TCP is used for larger responses, truncation, DNSSEC scenarios, and some resolver behavior. Blocking TCP 53 can create intermittent failures that are difficult to diagnose.
If your CoreDNS pods use different labels, adjust the podSelector. For example, some clusters use k8s-app: coredns. The key is to target the DNS pods, not to open broad egress to the entire kube-system namespace.
## Step 4: Allow Required In-Cluster Services
Applications often need to talk to services in the same namespace or a small number of shared namespaces. Examples include Redis, PostgreSQL, internal APIs, message brokers, and telemetry collectors.
Use labels to describe application identity. For example, suppose a frontend pod needs to talk to a backend API on port 8080 in the same namespace.
“`yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: frontend-to-api-egress
namespace: payments
spec:
podSelector:
matchLabels:
app: frontend
policyTypes:
– Egress
egress:
– to:
– podSelector:
matchLabels:
app: payments-api
ports:
– protocol: TCP
port: 8080
“`
For cross-namespace traffic, combine namespaceSelector and podSelector. For example, allow application pods to send metrics to an OpenTelemetry collector in the observability namespace:
“`yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-otel-egress
namespace: payments
spec:
podSelector: {}
policyTypes:
– Egress
egress:
– to:
– namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: observability
podSelector:
matchLabels:
app.kubernetes.io/name: opentelemetry-collector
ports:
– protocol: TCP
port: 4317
– protocol: TCP
port: 4318
“`
This is much better than allowing the entire cluster CIDR. It also creates documentation. Anyone reviewing the namespace can see that workloads are expected to send telemetry only to the collector.
## Step 5: Handle External APIs Without Opening the Internet
Standard NetworkPolicy does not support FQDN rules. That creates a challenge for applications that need to reach external services such as payment gateways, identity providers, email APIs, cloud storage endpoints, or software update services.
There are several practical approaches, each with tradeoffs.
### Option A: Use Stable IP Ranges Where Available
Some providers publish stable IP ranges. If your application depends on an external provider with documented CIDRs, you can use ipBlock rules.
“`yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-approved-payment-provider
namespace: payments
spec:
podSelector:
matchLabels:
app: payments-api
policyTypes:
– Egress
egress:
– to:
– ipBlock:
cidr: 203.0.113.0/24
ports:
– protocol: TCP
port: 443
“`
This is simple and portable, but it only works when the provider publishes reliable ranges. Be careful with large cloud provider CIDR lists. Allowing all of a hyperscaler region may be only marginally better than allowing the whole internet.
### Option B: Route Egress Through a Proxy
For many organizations, the cleanest model is to route application egress through an explicit HTTP or HTTPS proxy, secure web gateway, firewall, or egress gateway. Kubernetes NetworkPolicy then allows pods to reach only the proxy. Domain-level allow lists live at the proxy layer, where FQDN policy is native.
“`yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-egress-proxy
namespace: payments
spec:
podSelector: {}
policyTypes:
– Egress
egress:
– to:
– namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: security
podSelector:
matchLabels:
app: egress-proxy
ports:
– protocol: TCP
port: 3128
“`
Then set proxy environment variables in the deployment:
“`yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: payments-api
namespace: payments
spec:
template:
spec:
containers:
– name: app
image: registry.example.com/payments-api:1.8.4
env:
– name: HTTPS_PROXY
value: http://egress-proxy.security.svc.cluster.local:3128
– name: HTTP_PROXY
value: http://egress-proxy.security.svc.cluster.local:3128
– name: NO_PROXY
value: .svc,.cluster.local,10.0.0.0/8,127.0.0.1,localhost
“`
This pattern is attractive for regulated environments because it centralizes logging and policy. It also maps well to cyber insurance and compliance questions about outbound filtering, data exfiltration controls, and auditability.
### Option C: Use CNI-Specific FQDN Policies
If you use a CNI that supports FQDN-based egress policies, such as Cilium or Calico Enterprise features, you may be able to write policies directly against domain names. This can be very effective, but it reduces portability.
The engineering question is not whether vendor-specific features are bad. The question is whether your team understands the operational dependency. If your disaster recovery plan includes rebuilding clusters on another platform, policy portability matters.
## Step 6: Protect Cloud Metadata Services
One of the highest-value egress controls is blocking access to cloud instance metadata endpoints from ordinary workloads. Metadata services can expose credentials or identity tokens depending on platform and configuration.
Common metadata endpoints include link-local addresses such as 169.254.169.254. Exact behavior varies by cloud provider and node configuration.
With standard NetworkPolicy, blocking a specific destination is awkward because policies are allow lists, not deny lists. The practical approach is to use default deny egress and avoid creating any allow rule to metadata addresses. If you later add broad private network exceptions, be careful not to accidentally include metadata ranges.
For example, this broad rule is risky:
“`yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: risky-private-network-egress
namespace: payments
spec:
podSelector: {}
policyTypes:
– Egress
egress:
– to:
– ipBlock:
cidr: 0.0.0.0/0
except:
– 169.254.169.254/32
ports:
– protocol: TCP
port: 443
“`
The except clause helps, but allowing 0.0.0.0/0 to port 443 still permits almost any HTTPS destination. Use this only when you have another enforcement layer, such as a firewall, NAT gateway policy, secure web gateway, or CNI-specific DNS policy.
## Step 7: Test Policies Like Application Code
Network policies should be version-controlled, reviewed, and tested. A broken egress policy can be just as disruptive as a bad deployment.
Useful test commands include:
“`bash
kubectl exec -n payments deploy/payments-api — nslookup kubernetes.default.svc.cluster.local
kubectl exec -n payments deploy/payments-api — wget -S -T 3 -O – https://example.com
kubectl exec -n payments deploy/payments-api — nc -vz redis.payments.svc.cluster.local 6379
kubectl describe networkpolicy -n payments
“`
For repeatable checks, create a temporary diagnostic pod in the same namespace and with the same labels as the workload you are testing. Labels matter because NetworkPolicy selection depends on them.
“`bash
kubectl run netshoot -n payments –rm -it –image=nicolaka/netshoot –labels=app=payments-api — bash
“`
Inside the pod, test DNS, internal services, and approved external endpoints. Do not assume that a successful test from a different namespace or unlabeled debug pod proves the application will work.
## A Recommended Namespace Policy Layout
For most SMB and mid-market environments running production Kubernetes, a maintainable layout looks like this:
– default-deny-egress.yaml for the namespace baseline
– allow-dns.yaml for CoreDNS access
– allow-observability.yaml for metrics, traces, and logs
– allow-internal-services.yaml for required application dependencies
– allow-egress-proxy.yaml or allow-approved-external-cidrs.yaml for outbound internet dependencies
Keep policies small and named by intent. A policy named allow-egress is not helpful during an audit. A policy named payments-api-to-egress-proxy tells the next engineer what it is supposed to do.
Use labels consistently:
“`yaml
metadata:
labels:
security.computerbutler.io/control: egress-filtering
security.computerbutler.io/owner: platform-team
security.computerbutler.io/tier: restricted
“`
Labels make it easier to build reports, identify exceptions, and automate governance later.
## Operational Pitfalls to Avoid
### Allowing Entire Namespaces Too Broadly
Allowing egress to every pod in kube-system, observability, or shared-services is easy, but it undermines segmentation. Target the specific pods and ports required.
### Forgetting That Policies Are Additive
A permissive policy matching the same pod can override your carefully restricted model. Regularly review all policies that select sensitive workloads.
“`bash
kubectl get networkpolicy -n payments
kubectl describe networkpolicy -n payments
“`
### Treating Dev and Production the Same
Development namespaces often have broader outbound needs. Production workloads should be more predictable. Use different tiers rather than forcing one policy model onto every namespace.
### Ignoring Non-HTTP Protocols
Databases, message queues, LDAP, NTP, syslog, and telemetry protocols all have different ports and connection patterns. Inventory real traffic before enforcement.
### Skipping Observability
When policies drop traffic, the default Kubernetes experience may not clearly tell you why. CNI flow logs, firewall logs, service mesh telemetry, or egress proxy logs are extremely valuable. If you cannot see denied connections, troubleshooting becomes guesswork.
## Why This Matters for Business Risk
For business owners and IT leaders, Kubernetes egress filtering is not just a technical hardening exercise. It directly supports several practical risk management goals.
First, it limits blast radius. A compromised application should not automatically have outbound access to every internal service or internet destination.
Second, it reduces data exfiltration paths. Attackers often need outbound connectivity to move stolen data or establish persistence.
Third, it improves audit readiness. A documented egress model helps answer security questionnaires, cyber insurance reviews, and compliance assessments more convincingly than a flat network design.
Finally, it forces dependency clarity. Many organizations do not know which external systems their applications rely on until something breaks. Egress policy work often becomes a useful application discovery project.
## Practical Summary and Key Takeaways
Kubernetes egress NetworkPolicy is most successful when treated as an engineering rollout, not a one-time security checkbox. Start with one namespace, verify CNI enforcement, apply default deny, allow DNS carefully, then add only the internal and external destinations the workload requires.
Key takeaways:
– NetworkPolicy only works if your CNI enforces it.
– Default deny egress is the foundation, but it should be rolled out gradually.
– DNS must be explicitly allowed, usually to CoreDNS on both UDP and TCP 53.
– Use podSelector and namespaceSelector for internal service access instead of broad CIDR rules.
– Standard Kubernetes NetworkPolicy does not support FQDN rules, so external SaaS access often requires stable CIDRs, an egress proxy, or CNI-specific extensions.
– Avoid broad 0.0.0.0/0 exceptions unless another control layer is inspecting and restricting traffic.
– Version-control policies, test them with workload-matching labels, and monitor denied traffic.
Done well, Kubernetes egress control gives organizations a practical zero-trust building block inside the cluster. It makes applications more predictable, improves containment during security incidents, and gives IT leaders a clearer answer to a deceptively simple question: what is this workload allowed to talk to?
