113 lines
5.5 KiB
Markdown
113 lines
5.5 KiB
Markdown
# Kubernetes Security Baseline
|
|
|
|
An auditable, GitOps-managed Kubernetes security baseline: local k3d for a
|
|
fast reproducible demo, optional Flatcar plus Terraform and Ansible for a cloud
|
|
K3s host, Cilium eBPF networking, CIS Benchmark checks in CI, Kyverno
|
|
admission policies, and Falco runtime detection.
|
|
|
|
## Repository layout
|
|
|
|
Deployment and GitOps configuration is grouped under `deployments/`:
|
|
|
|
```text
|
|
deployments/
|
|
├── argocd/ # app-of-apps and child Application definitions
|
|
├── cilium/ # Cilium Helm values for the optional kube-proxy-free path
|
|
└── falco/ # Falco Operator values and custom resource definitions
|
|
```
|
|
|
|
The platform payloads remain separate from their deployment definitions. Kyverno
|
|
policies live under `policies/`, runtime demonstration workloads live under
|
|
`test-workloads/`, and cluster provisioning code remains under `terraform/`,
|
|
`ansible/`, and `local-quickstart/`.
|
|
|
|
The planned automation work is tracked in [`TODO.md`](TODO.md), including
|
|
secure Ansible secret input and Flatcar bootstrap automation.
|
|
|
|
## Phase 1: provision a local cluster
|
|
|
|
The primary quick-start path requires Docker, k3d, and kubectl:
|
|
|
|
```bash
|
|
k3d cluster create --config local-quickstart/k3d-cluster-config.yaml
|
|
kubectl get nodes -o wide
|
|
```
|
|
|
|
Expected result: one server and two agent nodes in `Ready` state. The bundled
|
|
Traefik is disabled so later ArgoCD-managed components own add-ons explicitly.
|
|
|
|
Delete the lab with `k3d cluster delete security-baseline`.
|
|
|
|
For repeatable CI and runtime testing, the repository also includes a Flatcar
|
|
Linux Proxmox host definition under `scripts/proxmox/`. The current test
|
|
environment runs k3d on a dedicated Flatcar VM with kernel access for Falco.
|
|
|
|
## Production-style option
|
|
|
|
`terraform/cloud-cluster/` and `ansible/bootstrap-k3s.yml` provide an optional,
|
|
documented AWS path. It uses Flatcar as the host OS, disables Flannel and
|
|
kube-proxy, and installs Cilium as the CNI and eBPF service datapath. It
|
|
requires your own AWS credentials, an existing EC2 key pair, a Flatcar AMI ID
|
|
for the chosen region, and a tightly scoped admin CIDR.
|
|
It is not needed for the portfolio demo and is not run in CI.
|
|
|
|
The Proxmox scripts also create Flatcar-based hosts for a self-hosted Gitea
|
|
Actions runner and for the dedicated k3d test environment. They use Ignition
|
|
for first-boot configuration and do not contain account-specific credentials.
|
|
|
|
## GitOps and security validation
|
|
|
|
ArgoCD is bootstrapped once and then manages the platform applications through
|
|
the app-of-apps definition in `deployments/argocd/`. The managed components are
|
|
Kyverno, Falco, and the security demonstration workloads. Cilium values and an
|
|
optional Cilium Application are also stored under `deployments/` for the
|
|
kube-proxy-free profile.
|
|
|
|
The kube-bench workflow is retained in `.github/workflows/` for GitHub Actions
|
|
and uses a Gitea-compatible artifact action when run by Gitea Actions. It
|
|
creates an ephemeral k3d cluster, waits for the Kubernetes API, runs the K3s
|
|
benchmark, and uploads the report without failing the workflow solely because
|
|
the benchmark contains findings that require review.
|
|
|
|
Kyverno admission tests have verified that privileged containers, root
|
|
containers, host networking or host PID access, unapproved registries, and
|
|
application workloads in the `default` namespace are rejected. Falco runtime
|
|
testing has verified that the suspicious shell workload produces the custom
|
|
shell detection while the compliant workload does not.
|
|
|
|
The local k3d cluster runs its node containers on one Flatcar host kernel, so
|
|
multiple Falco Pods can observe the same kernel event. This can create duplicate
|
|
notifications during local demonstrations. The production-style separate-VM
|
|
path gives each node its own kernel. See [`docs/runtime-detection.md`](docs/runtime-detection.md)
|
|
for the topology distinction and alerting implications.
|
|
|
|
Falco alerts are forwarded through Falcosidekick to Slack using a manually
|
|
populated Kubernetes Secret. The alert path has been tested end to end. Slack
|
|
priority filtering is being tuned so lower-priority events remain available in
|
|
Falco logs without overwhelming the notification channel.
|
|
|
|
## Build status
|
|
|
|
- [x] Phase 1: local k3d definition and optional Terraform/Ansible path
|
|
- [x] Phase 2: ArgoCD app-of-apps deployment and remote Flatcar test environment
|
|
- [x] Phase 3: kube-bench CI scan
|
|
- [x] Architecture revision: Flatcar hosts and Falco kernel access
|
|
- [x] Phase 4: Kyverno policy set, CIS mapping, and admission tests
|
|
- [x] Phase 5: Falco rules, Falcosidekick, and Slack webhook delivery
|
|
- [x] Phase 6: test workloads and runtime verification
|
|
- [ ] Phase 7: architecture and design documentation (runtime topology documented; diagram and evidence pending)
|
|
- [ ] Phase 8: final portfolio polish
|
|
- [ ] Cilium kube-proxy replacement: optional profile configured; production-style runtime validation pending
|
|
- [ ] Alert operations: priority filtering and duplicate-event tuning in progress
|
|
- [ ] Infrastructure automation: Ansible-driven Flatcar bootstrap and secure optional secret population pending
|
|
- [ ] Flatcar lifecycle management: update, rollback, recovery, and image rotation guide pending
|
|
|
|
## Remaining work
|
|
|
|
- Push and verify the Slack priority filtering change.
|
|
- Add the architecture diagram, CIS evidence summary, and demo capture.
|
|
- Automate Flatcar bootstrap, kubeconfig handling, ArgoCD bootstrap, and optional
|
|
secret population with Ansible.
|
|
- Write the Flatcar lifecycle management guide.
|
|
- Finish the design trade-off documentation and final repository review.
|