TELEICU/Path B: NixOS

guideteleicuCARE 3.0+

Full architecture and workflow for the NixOS migration path.

Architecture

Every hospital PC runs NixOS with identical, pinned system configuration. A flake repository defines the full OS state — packages, services, users, firewall, K3s agent — and nodes converge to the declared state via comin (pull-based, primary) or deploy-rs (push-based, backup).

{{#mermaid:flowchart TB
    subgraph Repo["Flake Repository"]
        FLAKE["flake.nix<br/>all node configs"]
    end

    subgraph Primary["Primary Path"]
        COMIN["comin (pull)<br/>each node polls<br/>for new generation"]
    end

    subgraph Backup["Backup Path"]
        DEPLOY["deploy-rs (push)<br/>triggered from CI<br/>for urgent deploys"]
    end

    FLAKE --> COMIN
    FLAKE --> DEPLOY

    COMIN --> N1["Spoke Node<br/>NixOS + K3s agent"]
    COMIN --> N2["Hub Node<br/>NixOS + K3s agent"]
    COMIN --> NM["... (200 more)"]

    DEPLOY --> N1
    DEPLOY --> N2
}}

Provisioning (zero-touch)

New sites are provisioned using a custom ISO generated with nixos-generator:

The ISO boots with a pre-configured Cloudflare Tunnel or Tailscale connection already active
A bootstrap script on the ISO initiates nixos-anywhere over the tunnel
nixos-anywhere partitions the disk (ZFS via disko) and installs NixOS
The node reboots into its NixOS generation and registers as a K3s agent
comin starts polling for configuration updates

What NixOS manages

Every system package, kernel parameter, and service
Tailscale configuration and bringup
K3s agent installation, config, and auto-join
Cloudflare tunnel service (cloudflared)
ZFS dataset layout and mount points
OpenEBS localPV provisioner for K3s
Monitoring agent, log shipper, downtime classifier
Firewall rules (strict — only Tailscale + K3s + tunnel outbound)
sops-nix secret decryption at build time

What K3s manages

Middleware container deployments
Service discovery, ingress, workload scaling
Identity as a Kubernetes node (identical to Path A at this layer)

Storage

ZFS + OpenEBS localPV:

Root pool: mirrored (if multiple disks) or single-disk with regular snapshots
LocalPV datasets for stateful middleware workloads
ZFS snapshots via sanoid/syncoid for fast backup
Off-site backup via restic or zfs send to S3

Rollback

Instant: nixos-rebuild switch --rollback reverts to the previous generation
At boot: hold Shift/space during systemd-boot to select a previous generation
Every NixOS generation is an atomic, bootable unit — rollback is always available even if the OS is partially broken

Update mechanism

comin is the primary path (pull-based, suited for unreliable connectivity):

- Each node polls the flake repository on a configurable interval - When a new generation is available, it builds and activates it atomically - If the build fails, the node stays on its current generation — no partial state

deploy-rs is the backup path:

- Push-based, triggered from CI for urgent/canary deploys - Requires the node to be reachable at deploy time - Used for staging and canary nodes where immediate rollout is needed

Repository structure

{{#mermaid:flowchart LR
    subgraph Flake["teleicu-nixos/"]
        FLAKE_NIX["flake.nix<br/>flake.lock"]
        HOSTS["hosts/<br/>cp-1/<br/>cp-2/<br/>cp-3/<br/>spoke-template/<br/>hub-template/"]
        MODULES["modules/<br/>tailscale.nix<br/>k3s-agent.nix<br/>cloudflared.nix<br/>zfs.nix<br/>openebs.nix<br/>monitoring.nix<br/>downtime-classifier.nix"]
        PROFILES["profiles/<br/>base.nix<br/>spoke.nix<br/>hub.nix"]
        SECRETS["secrets/"]
        ISO["iso/<br/>bootstrap.nix"]
        COMIN_CFG["comin/<br/>configuration.nix"]
    end
}}

Flake structure

flake.nix exposes:

- nixosConfigurations for each host (CP VMs + spoke/hub templates) - cominConfigurations for comin-based auto-updates - packages for the bootstrap ISO - devShells for development tooling