Admin: Add Navbox deployment template before Related (via update-page on MediaWiki MCP Server)

2026-07-06T04:02:14Z

Add Navbox deployment template before Related (via update-page on MediaWiki MCP Server)

← Older revision		Revision as of 09:32, 6 July 2026
Line 177:		Line 177:

	At this point you have a running cluster with networking and storage: nodes joined and labelled, namespaces created, Cilium providing pod networking, and Longhorn providing persistent volumes. The remaining work — ingress controllers, the image registry, PostgreSQL and Redis/Valkey, the RustFS object store, monitoring, and the Care backend and frontend themselves — is covered in [[Deployment/Kubernetes/Deploying Care onto the cluster\|Deploying Care onto the cluster]].		At this point you have a running cluster with networking and storage: nodes joined and labelled, namespaces created, Cilium providing pod networking, and Longhorn providing persistent volumes. The remaining work — ingress controllers, the image registry, PostgreSQL and Redis/Valkey, the RustFS object store, monitoring, and the Care backend and frontend themselves — is covered in [[Deployment/Kubernetes/Deploying Care onto the cluster\|Deploying Care onto the cluster]].

			{{Navbox deployment}}

	{{Related}}		{{Related}}

Admin: OHC identity seed

2026-07-04T23:07:42Z

OHC identity seed

New page

{{Doc header
|type=guide
|domain=deployment
|title=Provisioning the cluster
|order=4
}}

This page describes how the Kubernetes cluster that hosts Care is brought up: installing the container runtime and Kubernetes packages on each node, initializing the control plane with <code>kubeadm</code>, labelling nodes, creating the application namespaces, and installing the cluster-level add-ons (the Cilium CNI and Longhorn storage). Once the cluster and these foundations are in place, the application stack is rolled out as described in [[Deployment/Kubernetes/Deploying Care onto the cluster|Deploying Care onto the cluster]].

{{Warning|1=Exact node names, IP addresses, internal DNS names, public domains, TLS certificates, and object-store endpoints are environment-specific and live in the [<nowiki/>https://github.com/ohcnetwork/deployment-k8 <code>ohcnetwork/deployment-k8</code>] repository. The commands below use neutral placeholders such as <code><primary-node></code>, <code><secondary-node></code>, and <code>kube-api.internal</code>. Do not copy real secrets or hostnames out of one environment into another — substitute the values for the cluster you are building.}}

== Prerequisites ==

The reference cluster is two Ubuntu nodes on the same network — one '''primary''' (control plane) and one '''secondary''' failover node — provisioned with <code>kubeadm</code>. Each node needs the same base packages installed before it can join the cluster:

<syntaxhighlight lang="bash">sudo apt-get install -y apt-transport-https ca-certificates curl gpg</syntaxhighlight>
== 1. Install the container runtime ==

Care uses '''containerd''' as the container runtime, installed from the Docker apt repository. Run the following on every node.

Add the Docker apt repository and install the runtime packages:

<syntaxhighlight lang="bash"># Add Docker's official GPG key
sudo apt update
sudo apt install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to apt sources
sudo tee /etc/apt/sources.list.d/docker.sources <<EOF
Types: deb
URIs: https://download.docker.com/linux/ubuntu
Suites: $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}")
Components: stable
Signed-By: /etc/apt/keyrings/docker.asc
EOF

sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin</syntaxhighlight>
Enable the systemd cgroup driver in containerd so it matches the kubelet's cgroup management:

<syntaxhighlight lang="bash">containerd config default > config.toml
sudo mv config.toml /etc/containerd/config.toml
sudo sed -i "s/SystemdCgroup = false/SystemdCgroup = true/g" /etc/containerd/config.toml
sudo systemctl restart containerd</syntaxhighlight>
Raise the inotify limits. Kubernetes and the workloads that run on it open a large number of file watches, and the defaults are easily exhausted:

<syntaxhighlight lang="bash">sudo sysctl fs.inotify.max_user_instances=1280
sudo sysctl fs.inotify.max_user_watches=655360</syntaxhighlight>
{{Tip|1=The <code>sysctl</code> commands above take effect immediately but are not persistent. To survive reboots, write the same <code>fs.inotify.*</code> keys into a file under <code>/etc/sysctl.d/</code>.}}

== 2. Install the Kubernetes packages ==

Install <code>kubelet</code>, <code>kubeadm</code>, and <code>kubectl</code> from the upstream Kubernetes apt repository. The cluster is pinned to the '''v1.34''' package line — keep the repository path and the installed version on the same line across all nodes.

Add the Kubernetes signing key and apt source:

<syntaxhighlight lang="bash">curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.34/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.34/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list</syntaxhighlight>
Install the packages and hold them so they are not upgraded out of band:

<syntaxhighlight lang="bash">sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl</syntaxhighlight>
{{Note|1=The reference cluster runs '''with swap enabled'''. The <code>kubeadm</code> configuration sets <code>failSwapOn: false</code> and enables the <code>NodeSwap</code> feature gate with <code>LimitedSwap</code> behaviour, so the kubelet tolerates swap rather than refusing to start. This is intentional for these nodes — see the <code>KubeletConfiguration</code> block in <code>kubeadm.yaml</code>.}}

== 3. Initialize the control plane ==

<code>kubeadm</code> is driven by the checked-in <code>kubeadm.yaml</code>, which defines the control-plane endpoint (<code>controlPlaneEndpoint</code>), the API server certificate SANs, the Kubernetes version, and the pod and service subnets. On the primary node, initialize the cluster:

<syntaxhighlight lang="bash">sudo kubeadm init --config kubeadm.yaml</syntaxhighlight>
When <code>kubeadm init</code> finishes it prints a <code>kubeadm join</code> command containing the API server address, token, and CA certificate hash. Run that printed command on each worker/secondary node to join it to the cluster:

<syntaxhighlight lang="bash"># Example shape only — use the exact command printed by `kubeadm init`
sudo kubeadm join <control-plane-endpoint>:6443 --token <token> \
--discovery-token-ca-cert-hash sha256:<hash></syntaxhighlight>
{{Note|1=The pod and service CIDRs are defined in <code>kubeadm.yaml</code> (<code>networking.podSubnet</code> and <code>networking.serviceSubnet</code>). The CNI installed in step 6 must agree with these values, which is why Cilium is configured with <code>ipam.mode=kubernetes</code> — it uses the pod CIDR that <code>kubeadm</code> allocated.}}

== 4. Make the control-plane node schedulable ==

By default <code>kubeadm</code> taints the control-plane node so no application pods land on it. Because the reference cluster is small and the primary node is expected to run workloads, remove the control-plane and not-ready taints to make it schedulable. The trailing <code>-</code> removes the taint:

<syntaxhighlight lang="bash">kubectl taint node <primary-node> node-role.kubernetes.io/control-plane:NoSchedule-
kubectl taint node <primary-node> node.kubernetes.io/not-ready:NoSchedule-</syntaxhighlight>
== 5. Label the nodes ==

The deployer pins workloads and ingress to specific nodes using labels. Apply role and ingress labels so scheduling and ingress placement are deterministic. Use <code>--overwrite=True</code> to make the commands idempotent:

<syntaxhighlight lang="bash"># Ingress placement
kubectl label node <primary-node> ingress=private
kubectl label node <secondary-node> ingress=private

kubectl label node <primary-node> onlyingress=nope --overwrite=True
kubectl label node <secondary-node> onlyingress=nope --overwrite=True

# Node roles
kubectl label node <primary-node> ohn/role=primary --overwrite=True
kubectl label node <secondary-node> ohn/role=secondary --overwrite=True</syntaxhighlight>
{{Note|1=A cloud-only ingress node (a node labelled <code>ingress=public</code> / <code>onlyingress=yes</code>) is part of the intended topology but is commented out in the source today. Treat the public-ingress node as environment-specific: add it only if your topology includes a dedicated cloud ingress node.}}

== 6. Create the application namespaces ==

Create one namespace per application area. The deployer expects these to exist before it applies any resources:

<syntaxhighlight lang="bash">kubectl create namespace care-be
kubectl create namespace care-fe
kubectl create namespace odoo
kubectl create namespace metabase
kubectl create namespace rustfs</syntaxhighlight>
{| class="wikitable"
|-
! Namespace
! Holds
|-
| <code>care-be</code>
| Care backend (API, workers)
|-
| <code>care-fe</code>
| Care frontend
|-
| <code>odoo</code>
| Odoo
|-
| <code>metabase</code>
| Metabase analytics
|-
| <code>rustfs</code>
| RustFS object store
|}

== 7. Install the Cilium CNI ==

The cluster has no pod networking until a CNI is installed. Care uses '''Cilium''', installed with the Cilium CLI.

The cluster API host is reached through an internal DNS name rather than a raw IP, so that name must resolve on every node. Add an entry mapping the API name to the primary node's IP in <code>/etc/hosts</code> on '''each''' node before installing Cilium:

<syntaxhighlight lang="text"># /etc/hosts on every node
<primary-node-ip> kube-api.internal</syntaxhighlight>
Install the Cilium CLI (architecture-aware) and then install Cilium itself, pointing it at the internal API name and the pod CIDR allocated by <code>kubeadm</code>:

<syntaxhighlight lang="bash">CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
CLI_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}

cilium install \
--set k8sServiceHost=kube-api.internal \
--set k8sServicePort=6443 \
--set ipam.mode=kubernetes</syntaxhighlight>
{{Warning|1=<code>kube-api.internal</code> is a placeholder for the cluster's real internal API DNS name, which is defined in the deployment-k8 repo (and listed among the API server certificate SANs in <code>kubeadm.yaml</code>). Use the actual name for your environment, and make sure the matching <code>/etc/hosts</code> entry exists on every node — Cilium pods will not come up if the API host does not resolve.}}

== 8. Install Longhorn for storage ==

Persistent volumes are provided by '''Longhorn''', installed through the deployer. The deployer splits work into a <code>apply-tofu</code> phase (provision the underlying resources with OpenTofu) and an <code>apply-k8</code> phase (apply the Kubernetes manifests).

Provision Longhorn:

<syntaxhighlight lang="bash">python deployer.py apply-tofu volumes</syntaxhighlight>
Once Longhorn is installed, configure the nodes and storage classes as needed in the Longhorn UI or manifests. Longhorn can also back up volumes to an S3-compatible target; if you use that, create the backup-target credentials as a secret in the <code>longhorn-system</code> namespace (generalize the endpoint, region, and keys for your environment):

<syntaxhighlight lang="bash">kubectl create secret generic <backup-secret-name> -n longhorn-system \
--from-literal=AWS_ACCESS_KEY_ID=XXX \
--from-literal=AWS_SECRET_ACCESS_KEY=XXX \
--from-literal=AWS_ENDPOINTS=<s3-endpoint> \
--from-literal=AWS_REGION=<s3-region></syntaxhighlight>
Then apply the remaining Longhorn Kubernetes resources:

<syntaxhighlight lang="bash">python deployer.py apply-k8 volumes</syntaxhighlight>
{{Tip|1=After installing Longhorn, wait a couple of minutes before relying on it. Longhorn needs time to detect the nodes' disks and build the additional metadata it requires before it can serve volumes reliably.}}

== Next steps ==

At this point you have a running cluster with networking and storage: nodes joined and labelled, namespaces created, Cilium providing pod networking, and Longhorn providing persistent volumes. The remaining work — ingress controllers, the image registry, PostgreSQL and Redis/Valkey, the RustFS object store, monitoring, and the Care backend and frontend themselves — is covered in [[Deployment/Kubernetes/Deploying Care onto the cluster|Deploying Care onto the cluster]].

{{Related}}

Deployment/Kubernetes/Provisioning the cluster - Revision history

Admin: Add Navbox deployment template before Related (via update-page on MediaWiki MCP Server)

Admin: OHC identity seed