add environment-specific cilium profiles

deployments/cilium/README.md

@@ -0,0 +1,60 @@
+# Cilium deployment profiles
+
+The repository uses one shared Cilium baseline with separate networking
+profiles for nested k3d and dedicated VM clusters.
+
+## Profile layout
+
+```text
+deployments/cilium/
+├── values/
+│   ├── common.yaml
+│   └── profiles/
+│       ├── k3d.yaml
+│       └── flatcar-k3s.yaml
+└── README.md
+```
+
+`common.yaml` owns settings that should remain consistent across environments,
+including kube-proxy replacement, IPAM, Hubble, and the Cilium operator.
+Profile files own routing, masquerading, socket load-balancing, and endpoint
+values that depend on the node topology.
+
+## k3d profile
+
+The k3d profile uses native routing because the k3d nodes share a Docker bridge
+on the Flatcar host. It explicitly sets the pod CIDR required by Cilium native
+routing, enables BPF masquerading, and limits socket load-balancing to the host
+namespace. These settings address the extra network layer introduced by nested
+Docker containers.
+
+Ansible installs this profile before ArgoCD when kube-proxy is disabled. The
+published k3d API port is dynamic, so Ansible supplies the API host and port at
+runtime. This prevents a workstation-specific port from being committed to
+Git.
+
+## Dedicated Flatcar K3s profile
+
+The VM profile uses Geneve tunneling as the portable default. Native routing can
+be enabled when the cloud or virtualization network routes the Kubernetes pod
+CIDR between nodes. In that case, set `routingMode: native`, configure the
+native routing CIDR, and verify the required underlay routes before rollout.
+
+The VM profile does not enable the nested k3d socket load-balancing workaround.
+Replace `REPLACE_WITH_CONTROL_PLANE_API_IP` before using the profile directly
+with ArgoCD, or provide the endpoint through Ansible.
+
+## Ownership and bootstrap order
+
+When kube-proxy is disabled, Cilium must be installed before ordinary cluster
+workloads because it provides the CNI and service datapath. Ansible owns this
+initial bootstrap. ArgoCD may then manage the installed Cilium release for
+ongoing declarative upgrades.
+
+Ansible and ArgoCD should not continuously manage the same Helm release with
+different values. Select one profile and keep the bootstrap values and ArgoCD
+values aligned.
+
+The Cilium kube-proxy-free documentation describes the required API endpoint
+settings and the native and tunnel routing choices:
+<https://docs.cilium.io/en/stable/network/kubernetes/kubeproxy-free/>

deployments/cilium/cilium-values.yaml

@@ -1,37 +0,0 @@
-# Cilium is the CNI, network-policy engine, and eBPF service datapath for the
-# production-style K3s profile. Set this endpoint before bootstrapping a
-# cluster because kube-proxy is intentionally disabled.
-kubeProxyReplacement: true
-k8sServiceHost: "127.0.0.1"
-k8sServicePort: 6443
-
-ipam:
-  mode: kubernetes
-
-# k3d nodes share a directly reachable Docker bridge. Native routing avoids
-# nested VXLAN service paths that can prevent pod access to ClusterIP services.
-routingMode: native
-autoDirectNodeRoutes: true
-# K3s allocates pod addresses from this cluster-wide CIDR. Native routing
-# requires the CIDR so Cilium can distinguish pod traffic from underlay traffic.
-ipv4NativeRoutingCIDR: 10.42.0.0/16
-
-# The k3d nodes run as nested containers on the Flatcar host. eBPF masquerade
-# keeps return traffic for cross-node service backends inside the Cilium path.
-bpf:
-  masquerade: true
-
-# Keep socket load-balancing in the node namespace for the nested k3d profile.
-# Pod traffic is handled by the eBPF service datapath instead.
-socketLB:
-  hostNamespaceOnly: true
-
-hubble:
-  enabled: true
-  relay:
-    enabled: true
-  ui:
-    enabled: true
-
-operator:
-  replicas: 1

deployments/cilium/values/common.yaml

@@ -0,0 +1,16 @@
+# Shared Cilium settings for every cluster profile.
+# Environment-specific routing and API endpoint settings live under profiles/.
+kubeProxyReplacement: true
+
+ipam:
+  mode: kubernetes
+
+hubble:
+  enabled: true
+  relay:
+    enabled: true
+  ui:
+    enabled: true
+
+operator:
+  replicas: 1

deployments/cilium/values/profiles/flatcar-k3s.yaml

@@ -0,0 +1,14 @@
+# Dedicated VM profile.
+# Geneve tunneling is the portable default when the VM network does not have
+# explicit routes for the Kubernetes pod CIDR between nodes.
+routingMode: tunnel
+tunnelProtocol: geneve
+
+# Dedicated VMs do not need the nested-container socket-LB workaround.
+socketLB:
+  hostNamespaceOnly: false
+
+# Set these for an ArgoCD-managed installation, or override them through
+# Ansible when the control-plane address is environment-specific.
+k8sServiceHost: "REPLACE_WITH_CONTROL_PLANE_API_IP"
+k8sServicePort: 6443

deployments/cilium/values/profiles/k3d.yaml

@@ -0,0 +1,21 @@
+# Nested k3d profile.
+# k3d nodes are Docker containers sharing a directly reachable bridge network.
+# Native routing avoids the nested VXLAN service path that previously blocked
+# pod access to ClusterIP services.
+routingMode: native
+autoDirectNodeRoutes: true
+
+# K3s allocates pod addresses from this cluster-wide CIDR. Native routing
+# requires the CIDR so Cilium can distinguish pod traffic from underlay traffic.
+ipv4NativeRoutingCIDR: 10.42.0.0/16
+
+# Nested nodes need BPF masquerading for return traffic across the Docker
+# bridge. Keep socket load-balancing in the node namespace for this profile.
+bpf:
+  masquerade: true
+
+socketLB:
+  hostNamespaceOnly: true
+
+# Ansible supplies k8sServiceHost and k8sServicePort during bootstrap because
+# the published workstation port is dynamically allocated by k3d.