61 lines
2.4 KiB
Markdown
61 lines
2.4 KiB
Markdown
# Cilium deployment profiles
|
|
|
|
The repository uses one shared Cilium baseline with separate networking
|
|
profiles for nested k3d and dedicated VM clusters.
|
|
|
|
## Profile layout
|
|
|
|
```text
|
|
deployments/cilium/
|
|
├── values/
|
|
│ ├── common.yaml
|
|
│ └── profiles/
|
|
│ ├── k3d.yaml
|
|
│ └── flatcar-k3s.yaml
|
|
└── README.md
|
|
```
|
|
|
|
`common.yaml` owns settings that should remain consistent across environments,
|
|
including kube-proxy replacement, IPAM, Hubble, and the Cilium operator.
|
|
Profile files own routing, masquerading, socket load-balancing, and endpoint
|
|
values that depend on the node topology.
|
|
|
|
## k3d profile
|
|
|
|
The k3d profile uses native routing because the k3d nodes share a Docker bridge
|
|
on the Flatcar host. It explicitly sets the pod CIDR required by Cilium native
|
|
routing, enables BPF masquerading, and limits socket load-balancing to the host
|
|
namespace. These settings address the extra network layer introduced by nested
|
|
Docker containers.
|
|
|
|
Ansible installs this profile before ArgoCD when kube-proxy is disabled. The
|
|
published k3d API port is dynamic, so Ansible supplies the API host and port at
|
|
runtime. This prevents a workstation-specific port from being committed to
|
|
Git.
|
|
|
|
## Dedicated Flatcar K3s profile
|
|
|
|
The VM profile uses Geneve tunneling as the portable default. Native routing can
|
|
be enabled when the cloud or virtualization network routes the Kubernetes pod
|
|
CIDR between nodes. In that case, set `routingMode: native`, configure the
|
|
native routing CIDR, and verify the required underlay routes before rollout.
|
|
|
|
The VM profile does not enable the nested k3d socket load-balancing workaround.
|
|
Replace `REPLACE_WITH_CONTROL_PLANE_API_IP` before using the profile directly
|
|
with ArgoCD, or provide the endpoint through Ansible.
|
|
|
|
## Ownership and bootstrap order
|
|
|
|
When kube-proxy is disabled, Cilium must be installed before ordinary cluster
|
|
workloads because it provides the CNI and service datapath. Ansible owns this
|
|
initial bootstrap. ArgoCD may then manage the installed Cilium release for
|
|
ongoing declarative upgrades.
|
|
|
|
Ansible and ArgoCD should not continuously manage the same Helm release with
|
|
different values. Select one profile and keep the bootstrap values and ArgoCD
|
|
values aligned.
|
|
|
|
The Cilium kube-proxy-free documentation describes the required API endpoint
|
|
settings and the native and tunnel routing choices:
|
|
<https://docs.cilium.io/en/stable/network/kubernetes/kubeproxy-free/>
|