TELEICU/Architecture overview - Revision history

Admin: Fix Mermaid diagram: remove

 wrapping so parser function executes (via update-page on MediaWiki MCP Server)

2026-07-06T03:18:15Z

Fix Mermaid diagram: remove <pre> wrapping so parser function executes (via update-page on MediaWiki MCP Server)

← Older revision		Revision as of 08:48, 6 July 2026
Line 34:		Line 34:
	== Target topology ==		== Target topology ==

	<pre>
	{{#mermaid:flowchart TB		{{#mermaid:flowchart TB
	subgraph CP["Cloud Control Plane (1-3 VMs)"]		subgraph CP["Cloud Control Plane (1-3 VMs)"]
Line 61:		Line 60:
	TS --- S3		TS --- S3
	}}		}}
	</pre>

	== Key design decisions ==		== Key design decisions ==

Admin: Automated edit (via update-page on MediaWiki MCP Server)

2026-07-06T03:09:24Z

Automated edit (via update-page on MediaWiki MCP Server)

← Older revision		Revision as of 08:39, 6 July 2026
Line 73:		Line 73:

	See the [[TELEICU/Migration strategy\|Migration strategy]] for the full decision framework.		See the [[TELEICU/Migration strategy\|Migration strategy]] for the full decision framework.

			{{Navbox TELEICU}}

	{{Related}}		{{Related}}

Admin: Replace ASCII topology with Mermaid diagram (via update-page on MediaWiki MCP Server)

2026-07-06T02:55:01Z

Replace ASCII topology with Mermaid diagram (via update-page on MediaWiki MCP Server)

← Older revision		Revision as of 08:25, 6 July 2026
Line 31:		Line 31:

	Both paths share the same target topology: all hospital PCs become K3s agent nodes, managed from 1–3 cloud-based control plane VMs. Inter-node communication runs over Tailscale.		Both paths share the same target topology: all hospital PCs become K3s agent nodes, managed from 1–3 cloud-based control plane VMs. Inter-node communication runs over Tailscale.

			== Target topology ==

			<pre>
			{{#mermaid:flowchart TB
			subgraph CP["Cloud Control Plane (1-3 VMs)"]
			CP1["CP VM 1<br/>(etcd member)"]
			CP2["CP VM 2<br/>(etcd member)"]
			CP3["CP VM 3<br/>(etcd member)"]
			end

			subgraph TS["Tailscale Mesh"]
			CP1 --- TS
			CP2 --- TS
			CP3 --- TS
			end

			subgraph Sites["Hospital Sites"]
			direction TB
			HUB["Hub Site<br/>K3s agent + middleware"]
			S1["Spoke Site<br/>K3s agent + middleware"]
			S2["Spoke Site<br/>K3s agent + middleware"]
			S3["Spoke Site<br/>K3s agent + middleware"]
			end

			TS --- HUB
			TS --- S1
			TS --- S2
			TS --- S3
			}}
			</pre>

	== Key design decisions ==		== Key design decisions ==

Admin: Create TELEICU architecture overview concept page (via create-page on MediaWiki MCP Server)

2026-07-06T02:52:06Z

Create TELEICU architecture overview concept page (via create-page on MediaWiki MCP Server)

New page

{{Doc header
|type=concept
|domain=teleicu
|title=Architecture overview
|order=1
}}

The TELEICU middleware layer connects 200+ hospitals (spokes) to central specialist hubs across 7 states. This page describes the current architecture, the reliability problems it faces, and the upgrade strategy being undertaken.

== Scope ==

* 200+ hospital sites (spokes) providing tele-ICU services
* Designated hub hospitals with specialist coverage
* 7 states across the network
* Currently running on heterogeneous hardware with mixed Ubuntu versions (20.04, 22.04, 24.04)

== Current pain points ==

Three reliability factors degrade the current deployment:

# '''Frequent power and internet disruption''' — sites in semi-urban and rural areas experience intermittent connectivity. Nodes go offline unpredictably.
# '''Tampering by unauthorized personnel''' — physical access to site hardware has led to configuration drift: Cloudflare tunnel credentials altered, docker-compose files modified, system settings changed without tracking.
# '''Gap in OS security and maintenance updates''' — 200 devices are out of sync across Ubuntu 20.04, 22.04, and 24.04. No consistent patch cadence.

== Upgrade strategy ==

The goal is to consolidate all nodes into a single K3s cluster with consistent, reproducible system state. Two paths are under evaluation:

* '''Path A — Ansible Pull''': Manage the OS layer and k3s agent configuration through a pull-based ansible workflow. Minimal new tooling, existing ops team skills apply.
* '''Path B — NixOS''': Full OS immutability via NixOS, provisioned with <code>nixos-anywhere</code>, updated via <code>comin</code> (pull) with <code>deploy-rs</code> as backup.

Both paths share the same target topology: all hospital PCs become K3s agent nodes, managed from 1–3 cloud-based control plane VMs. Inter-node communication runs over Tailscale.

== Key design decisions ==

* '''Storage''' — NixOS path uses ZFS + OpenEBS for localPV; Ansible path uses Longhorn atop existing filesystems (no repartitioning possible).
* '''Updates''' — Both paths are pull-based (comin / ansible-pull) to handle unreliable connectivity.
* '''Monitoring''' — VictoriaMetrics for metrics, with on-device agents that classify downtime by cause (power vs network vs other).
* '''Incident management''' — Hub-admin notification chain with escalation based on downtime classification.
* '''Secrets''' — sops-nix for NixOS, Ansible Vault + sops for Ansible.
* '''Networking''' — Tailscale mesh with ACL-based tagging for all inter-node traffic.

See the [[TELEICU/Migration strategy|Migration strategy]] for the full decision framework.

{{Related}}