kubernetes-security-baseline-lab/deployments/cilium

Cilium deployment profiles

The repository uses one shared Cilium baseline with separate networking profiles for nested k3d and dedicated VM clusters.

Profile layout

deployments/cilium/
├── values/
│   ├── common.yaml
│   └── profiles/
│       ├── k3d.yaml
│       └── flatcar-k3s.yaml
└── README.md

common.yaml owns settings that should remain consistent across environments, including kube-proxy replacement, IPAM, Hubble, and the Cilium operator. Profile files own routing, masquerading, socket load-balancing, and endpoint values that depend on the node topology.

k3d profile

The k3d profile uses native routing because the k3d nodes share a Docker bridge on the Flatcar host. It explicitly sets the pod CIDR required by Cilium native routing, enables BPF masquerading, and limits socket load-balancing to the host namespace. These settings address the extra network layer introduced by nested Docker containers.

Ansible installs this profile before ArgoCD when kube-proxy is disabled. The published k3d API port is dynamic, so Ansible supplies the API host and port at runtime. This prevents a workstation-specific port from being committed to Git.

Dedicated Flatcar K3s profile

The VM profile uses Geneve tunneling as the portable default. Native routing can be enabled when the cloud or virtualization network routes the Kubernetes pod CIDR between nodes. In that case, set routingMode: native, configure the native routing CIDR, and verify the required underlay routes before rollout.

The VM profile does not enable the nested k3d socket load-balancing workaround. Replace REPLACE_WITH_CONTROL_PLANE_API_IP before using the profile directly with ArgoCD, or provide the endpoint through Ansible.

Ownership and bootstrap order

When kube-proxy is disabled, Cilium must be installed before ordinary cluster workloads because it provides the CNI and service datapath. Ansible owns this initial bootstrap. ArgoCD may then manage the installed Cilium release for ongoing declarative upgrades.

Ansible and ArgoCD should not continuously manage the same Helm release with different values. Select one profile and keep the bootstrap values and ArgoCD values aligned.

The Cilium kube-proxy-free documentation describes the required API endpoint settings and the native and tunnel routing choices: https://docs.cilium.io/en/stable/network/kubernetes/kubeproxy-free/