🐳 Section 14 · Question #21

What is namespace

A Namespace in Kubernetes is a built-in mechanism for logically partitioning a single physical cluster into isolated virtual workspaces for different teams, projects, or environ...

🐳 Docker Kubernetes 📄 Source Markdown

🟢 Junior Level

30-Second Summary

A Namespace in Kubernetes is a built-in mechanism for logically partitioning a single physical cluster into isolated virtual workspaces for different teams, projects, or environments (dev, stage, prod).

  • Name Collision Prevention: Resource names must be unique within a single namespace, but identical names can coexist cleanly across distinct namespaces (e.g., payment-service in dev and payment-service in prod).
  • Access Control (RBAC): Administrators can grant developers full administrative privileges in a sandbox namespace while restricting access to read-only logs or denying access entirely in production.
  • Resource Governance: Compute consumption can be constrained per namespace using hard CPU and memory ceilings (ResourceQuota).

Real-World Analogy

A Namespace is like an individual apartment inside a large residential high-rise:

  • The high-rise building itself is the physical Kubernetes cluster of servers.
  • Apartments 101, 102, and 103 represent separate namespaces (dev, stage, prod).
  • Each apartment has its own kitchen appliance labeled “Refrigerator” (the order-db pod). Residents never mistake their neighbor’s refrigerator for their own because they are separated into distinct physical units.

Basic Manifest and Commands

apiVersion: v1
kind: Namespace
metadata:
  name: billing-prod
  labels:
    environment: production
    team: billing

# List all active namespaces in the cluster:
kubectl get namespaces

# Launch a pod inside a specific namespace:
kubectl run test-pod --image=nginx -n billing-prod

# Switch default namespace context (omitting the -n flag for subsequent commands):
kubectl config set-context --current --namespace=billing-prod

[!WARNING] A Namespace is NOT Physical or Network Isolation! By default, Kubernetes operates a flat network: any Pod in the dev namespace can initiate direct TCP connections to a database running in prod. Hard network isolation requires explicit NetworkPolicy enforcement.

🟡 Middle Level

Resource Scoping: Namespace-Scoped vs. Cluster-Scoped

Namespace-Scoped Resources (Project Level) Cluster-Scoped Resources (Global Infrastructure)
Pod, Deployment, StatefulSet, ReplicaSet Node (Physical/virtual worker server)
Service, Ingress, EndpointSlice PersistentVolume (Physical SAN/EBS disk storage)
ConfigMap, Secret StorageClass (Dynamic volume provisioners)
PersistentVolumeClaim (Storage claim) ClusterRole, ClusterRoleBinding (Global security)
ServiceAccount, Role, RoleBinding Namespace (The virtual boundary itself)

Cross-Namespace Communication & CoreDNS

Pods across different namespaces communicate seamlessly over the internal cluster network using Fully Qualified Domain Names (FQDN):

http://<service-name>.<namespace-name>.svc.cluster.local:<port>
  • Within the same namespace: A short unqualified hostname suffices: http://postgres:5432.
  • Across different namespaces: The FQDN is required: http://postgres.billing-prod.svc.cluster.local:5432.

Capacity Governance: ResourceQuota vs. LimitRange

To prevent runaway workloads from consuming the entire cluster, administrators deploy two complementary guardrails:

1. ResourceQuota (Total Aggregate Ceiling per Namespace)

apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-quota
  namespace: dev-team
spec:
  hard:
    requests.cpu: "20"        # Sum of all requests.cpu cannot exceed 20 cores
    requests.memory: 50Gi     # Sum of all requests.memory cannot exceed 50 GB
    limits.cpu: "40"
    limits.memory: 100Gi
    pods: "30"                # Maximum of 30 total pods in this namespace

2. LimitRange (Per-Container Defaults and Constraints)

apiVersion: v1
kind: LimitRange
metadata:
  name: container-limits
  namespace: dev-team
spec:
  limits:
  - default:                  # Injected limit if omitted in Pod spec
      cpu: "1"
      memory: "1Gi"
    defaultRequest:           # Injected request if omitted in Pod spec
      cpu: "200m"
      memory: "256Mi"
    max:                      # Hard ceiling for an individual container
      cpu: "4"
      memory: "4Gi"
    type: Container

🔴 Senior Level

The Multi-Tenancy Security Illusion

A common question in enterprise architectural reviews:

“Is it safe to host public-facing untrusted workloads and private banking ledger databases in separate Namespaces within the same Kubernetes cluster?”

Answer: NO, not without dedicated isolation technologies. Namespaces isolate metadata visibility via the API Server, but leave shared underlying substrates exposed:

  1. Shared Linux Kernel: Pods from dev and prod may be co-scheduled onto the exact same worker node. Any Linux kernel privilege escalation vulnerability (e.g., Dirty Pipe, Dirty COW, or kernel eBPF flaws) allows an attacker to escape container namespaces, obtain host root privileges, and compromise adjacent prod containers.
  2. Default Flat Network: Without a CNI supporting NetworkPolicy (e.g., Cilium or Calico) and an enforced default-deny ingress rule, all containers can inspect or probe peer traffic.
  3. Enterprise Multi-Tenancy Solution: Deploy physically separate clusters, or enforce hardware-virtualized microVM container runtimes (Kata Containers, gVisor) that provide isolated kernel instances per pod.

The Namespace Terminating Freeze Trap (Finalizers Mechanism)

A notorious operational issue: an engineer runs kubectl delete namespace staging, but the namespace remains frozen in Terminating status for days.

Root Cause:

  1. The NamespaceLifecycle admission controller flags the namespace as Terminating and initiates recursive cascade deletion of all child resources.
  2. The namespace metadata includes spec.finalizers: ["kubernetes"].
  3. If any nested resource cannot complete its teardown (e.g., a PersistentVolumeClaim waits for an unresponsive AWS EBS controller, or an operator CRD fails graceful cleanup), the finalizer chain blocks indefinitely.

Recovery Procedure:

# 1. Discover remaining blocked resources inside the terminating namespace:
kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1 kubectl get --show-kind --ignore-not-found -n staging

# 2. Strip finalizers from the hanging resource:
kubectl patch pvc stuck-pvc -n staging -p '{"metadata":{"finalizers":null}}' --type=merge

# 3. Emergency bypass: Directly wipe the namespace finalizer array via the Raw API:
kubectl get namespace staging -o json | jq '.spec.finalizers=[]' | kubectl replace --raw "/api/v1/namespaces/staging/finalize" -f -

Pod Security Standards (PSA) at Namespace Scope

In Kubernetes 1.25+, deprecated PodSecurityPolicies were superseded by Pod Security Admission (PSA), configured directly via Namespace labels:

apiVersion: v1
kind: Namespace
metadata:
  name: secure-prod
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest
    pod-security.kubernetes.io/warn: restricted

The API Server admission controller will automatically reject any Pod manifest attempting to run as root, mount hostPath, or use privileged Linux capabilities.

4 Tricky Questions

1. What happens if an engineer applies a Deployment manifest without resources.requests inside a Namespace that has an active CPU ResourceQuota?

Answer: The API Server immediately rejects the Pod creation with an admission validation error:

Error from server (Forbidden): pods "order-app-xxx" is forbidden: failed quota: compute-quota: must specify cpu for: order-app

When a ResourceQuota enforces aggregate requests.cpu, the quota controller strictly requires that every container deployed in that namespace explicitly defines its own requests.cpu. To resolve this without forcing developers to modify every YAML file, create a LimitRange in that namespace specifying defaultRequest. The mutating admission controller will automatically inject default resource requests into incoming pods before quota validation occurs.

2. Why is PersistentVolume (PV) a Cluster-Scoped resource, while PersistentVolumeClaim (PVC) is Namespace-Scoped?

Answer: This reflects the architectural separation between storage provisioning and application consumption:

  • PersistentVolume (PV) represents a physical underlying storage asset (e.g., an AWS EBS volume, SAN LUN, or local NVMe array). It is provisioned and governed globally by cluster infrastructure administrators.
  • PersistentVolumeClaim (PVC) is an application-level request for storage issued by developers within their project namespace (e.g., “Allocate 50 GB of fast block storage”).
  • Kubernetes matches and binds the local PVC to a compatible global PV. When the Namespace is deleted, the claim (PVC) is destroyed, while the underlying physical disk (PV) is handled according to its global reclaimPolicy (Retain preserves the data, Delete purges the cloud volume).

3. How can an attacker who compromises a container in the development namespace access an unauthenticated database in the production namespace under default Kubernetes settings?

Answer: In a vanilla Kubernetes cluster:

  1. Network Layer: The default CNI network plugin provides flat, unrestricted pod-to-pod routing across the entire cluster without namespace boundary isolation. An attacker inside a development container shell can issue curl http://db.production.svc.cluster.local:5432 directly to the production database IP.
  2. DNS Layer: CoreDNS resolves internal cluster hostnames globally across all namespaces for every query. Mitigation: Enforce a NetworkPolicy with an Ingress rule on the production database, explicitly allowing traffic only from pods carrying specific namespace labels via namespaceSelector.

4. Can an active Pod or Service be moved from one Namespace to another using kubectl?

Answer: No, moving an active resource between namespaces is impossible. The metadata.namespace field is strictly immutable once an object is created. In the underlying etcd key-value store, the primary key includes the namespace: /registry/pods/<namespace>/<pod-name>. To migrate a workload to a new namespace, you must:

  1. Export the existing manifest: kubectl get deployment my-app -n old-ns -o yaml > app.yaml.
  2. Update the metadata.namespace attribute to the new namespace.
  3. Apply the updated manifest: kubectl apply -f app.yaml.
  4. Delete the original deployment from the previous namespace.

🎯 Interview Cheat Sheet

Core Namespace Architecture

  • Logical Boundary: Virtual cluster slice enabling duplicate resource naming across environments (dev/stage/prod).
  • No Inherent Security: Namespaces provide neither physical nor network isolation by default; flat CNI networking allows cross-namespace traffic.
  • Cross-Namespace DNS: Reachable via FQDN: <service-name>.<namespace>.svc.cluster.local.
  • Resource Governance: ResourceQuota enforces aggregate namespace capacity; LimitRange sets per-container defaults and boundaries.
  • Stuck in Terminating: Caused by unreleased finalizers (spec.finalizers) on lingering child resources (PVCs, CRDs).
  • Security Baseline: Enforce Pod Security Standards via namespace labels (pod-security.kubernetes.io/enforce: restricted).

Resource Scoping Comparison

| Scope | Key Characteristics | Resources | | :— | :— | :— | | Namespaced | Scoped to individual tenant projects | Pod, Service, Deployment, PVC, ConfigMap, Secret, ServiceAccount, Role | | Cluster-wide | Shared across the entire infrastructure | Node, PersistentVolume, StorageClass, ClusterRole, Namespace |

Red Flags (What NOT to Say)

  • ❌ “Namespaces physically isolate containers onto separate worker nodes.” — Pods from different namespaces run concurrently on the same worker nodes.
  • ❌ “Namespaces block all network traffic between dev and prod environments automatically.” — CNI networks are flat; isolation requires explicit NetworkPolicy.
  • ❌ “You can reassign an active pod to a different namespace by editing its YAML.” — The metadata.namespace field is immutable in etcd.