add environment-specific cilium profiles

2026-08-16 20:04:14 -04:00
parent 1a372b7eac
commit 6c79340d9a
14 changed files with 188 additions and 56 deletions
@@ -0,0 +1,60 @@
+# Cilium deployment profiles
+
+The repository uses one shared Cilium baseline with separate networking
+profiles for nested k3d and dedicated VM clusters.
+
+## Profile layout
+
+```text
+deployments/cilium/
+├── values/
+│   ├── common.yaml
+│   └── profiles/
+│       ├── k3d.yaml
+│       └── flatcar-k3s.yaml
+└── README.md
+```
+
+`common.yaml` owns settings that should remain consistent across environments,
+including kube-proxy replacement, IPAM, Hubble, and the Cilium operator.
+Profile files own routing, masquerading, socket load-balancing, and endpoint
+values that depend on the node topology.
+
+## k3d profile
+
+The k3d profile uses native routing because the k3d nodes share a Docker bridge
+on the Flatcar host. It explicitly sets the pod CIDR required by Cilium native
+routing, enables BPF masquerading, and limits socket load-balancing to the host
+namespace. These settings address the extra network layer introduced by nested
+Docker containers.
+
+Ansible installs this profile before ArgoCD when kube-proxy is disabled. The
+published k3d API port is dynamic, so Ansible supplies the API host and port at
+runtime. This prevents a workstation-specific port from being committed to
+Git.
+
+## Dedicated Flatcar K3s profile
+
+The VM profile uses Geneve tunneling as the portable default. Native routing can
+be enabled when the cloud or virtualization network routes the Kubernetes pod
+CIDR between nodes. In that case, set `routingMode: native`, configure the
+native routing CIDR, and verify the required underlay routes before rollout.
+
+The VM profile does not enable the nested k3d socket load-balancing workaround.
+Replace `REPLACE_WITH_CONTROL_PLANE_API_IP` before using the profile directly
+with ArgoCD, or provide the endpoint through Ansible.
+
+## Ownership and bootstrap order
+
+When kube-proxy is disabled, Cilium must be installed before ordinary cluster
+workloads because it provides the CNI and service datapath. Ansible owns this
+initial bootstrap. ArgoCD may then manage the installed Cilium release for
+ongoing declarative upgrades.
+
+Ansible and ArgoCD should not continuously manage the same Helm release with
+different values. Select one profile and keep the bootstrap values and ArgoCD
+values aligned.
+
+The Cilium kube-proxy-free documentation describes the required API endpoint
+settings and the native and tunnel routing choices:
+<https://docs.cilium.io/en/stable/network/kubernetes/kubeproxy-free/>