Namespace isolation

Pods don’t accept connections from pods in other namespaces unless a NetworkPolicy in their namespace or a global rule lets them. The global rules are Calico policies in the tenant-isolation directory of the prp/calico repository. What users see is described in Exposing to other namespaces.

Who decides what

• Namespace administrators decide which other namespaces may reach their pods, with ordinary NetworkPolicies in their own namespace. When admins open a tenant service to another namespace on an owner’s behalf, they do it the same way: a NetworkPolicy in the destination namespace, labeled nrp.ai/policy=cross-ns-allow, with the reason in the nrp.ai/reason annotation. The owner can change or remove it.
• The global rules in prp/calico hold only what namespace administrators mustn’t control: the baseline, services kept to their namespace, exceptions into system namespaces (label nautilus.io/system-namespace=true), the platform services that reach every pod, and services that are public anyway.

The policies

• Tier tenant-isolation (tier.yaml): evaluated after the default tier, where Kubernetes NetworkPolicies live. A pod that an ingress NetworkPolicy selects is decided by its namespace’s NetworkPolicies alone; the rules below apply only to pods that no NetworkPolicy selects.
• Baseline tenant-isolation.baseline: a Calico NetworkPolicy in every namespace that denies ingress from pods in other namespaces. It doesn’t match traffic from outside the cluster, from nodes or from host-network pods.
• Global rules, checked before the baseline, first match wins:
  • Services that don’t authenticate are kept to their own namespace: a Pass from that namespace, then a Deny from every other source, including the shared services, host-network pods and nodes. Stalwart’s FoundationDB and Redis are kept this way.
  • shared-services.yaml (user namespaces) and system-cross-namespace-allow.yaml (system namespaces) pass connections from the shared-service namespaces (ingress controllers, monitoring and the operators that connect to the pods they manage) and the portal, and, in system namespaces, to specific services such as Stalwart’s mail ports or the TURN REST service.
  • internet-visible.yaml passes every namespace to what the internet already reaches: the TCP ports behind public LoadBalancer VIPs, and Prometheus and Thanos, whose Ingresses are public. An in-cluster client of a VIP keeps its pod address, so without this rule it would be blocked while the internet gets through.

A namespace belongs in the shared-service list only if its pods connect to pods in other namespaces, since anyone who can run a pod there reaches every pod in the cluster. The same holds for namespaces exempt from the psp-no-host-network Gatekeeper constraint: their host-network pods count as nodes and aren’t isolated. Keep both lists short.

RBAC keeps namespace administrators from creating, changing or deleting Calico policies, and the Gatekeeper constraint protect-tenant-isolation-networkpolicy (prp/gatekeeper) reserves the baseline for cluster administrators.

New namespaces

The portal creates the baseline with each namespace. For namespaces made any other way, the CronJob kube-system/tenant-isolation-baseline (tenant-isolation/baseline-reconciler.yaml, applied by hand with kubectl apply) creates missing baselines every minute. It only creates: it never changes or deletes a baseline, and Gatekeeper lets its service account do nothing else. A failed run shows as a failed Job:

kubectl -n kube-system get jobs -l app=tenant-isolation-baseline

Changing the rules

Edit the files in prp/calico and merge to master; CI applies them with calicoctl apply.

• CI runs for every pushed branch, so push work in progress with git push -o ci.skip.
• A push that changes .gitlab-ci.yml doesn’t deploy; push the files it applies in a later commit.
• calicoctl apply doesn’t delete: remove a policy that leaves the repository with calicoctl delete globalnetworkpolicy <name> in the calicoctl pod in kube-system.

To keep a service to its namespace, copy the Stalwart pair at the top of system-cross-namespace-allow.yaml. After adding or removing a LoadBalancer Service, regenerate the internet-visible rules and commit the result:

kubectl get svc -A -o json | tenant-isolation/generate-internet-visible.py > tenant-isolation/internet-visible.yaml

Checking a connection

List the policies that apply, then probe from a short-lived pod in the source namespace. A dropped connection times out; nc prints nothing and exits non-zero.

kubectl get globalnetworkpolicies.crd.projectcalico.org | grep tenant-isolation
kubectl -n <namespace> get networkpolicies.crd.projectcalico.org,networkpolicies

kubectl -n <source-namespace> apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata: {name: netprobe}
spec:
  restartPolicy: Never
  activeDeadlineSeconds: 600
  containers:
    - name: probe
      image: alpine
      command: [sleep, '500']
      resources: {requests: {cpu: 50m, memory: 32Mi}, limits: {cpu: 50m, memory: 32Mi}}
EOF
kubectl -n <source-namespace> wait --for=condition=Ready pod/netprobe
kubectl -n <source-namespace> exec netprobe -- nc -z -w 4 <pod-ip> <port> && echo open || echo blocked
kubectl -n <source-namespace> delete pod netprobe

A probe can also look blocked when the source pod’s own namespace has an egress NetworkPolicy that selects it.