TELEICU/Canary rollout: Difference between revisions
From OHC Network Wiki
More languages
More actions
Content deleted Content added
Create TELEICU canary rollout flow page (via create-page on MediaWiki MCP Server) |
Fix 3 Mermaid diagrams: remove <pre> wrapping (via update-page on MediaWiki MCP Server) |
||
| (2 intermediate revisions by the same user not shown) | |||
| Line 10: | Line 10: | ||
== Stages == |
== Stages == |
||
{{#mermaid:flowchart LR |
|||
# '''Staging''' — a small set of test VMs or spare hardware that mirrors production. Every change lands here first. |
|||
STAGING["Staging<br/>(test VMs/hardware)"] |
|||
# '''Canary (1–2 sites)''' — real hospital PCs, selected from sites with tolerant operations. Changes soak here for a validation period. |
|||
CANARY["Canary<br/>(1-2 real sites)"] |
|||
# '''Regional rollout''' — one state or hub region at a time, with a gating check after each region. |
|||
REGIONAL["Regional Rollout<br/>(one state at a time)"] |
|||
# '''Fleet-wide''' — remaining sites after all regions have passed. |
|||
FLEET["Fleet-wide<br/>(remaining sites)"] |
|||
STAGING -->|"validate →"| CANARY |
|||
CANARY -->|"gate →"| REGIONAL |
|||
REGIONAL -->|"gate →"| FLEET |
|||
}} |
|||
== Gating criteria == |
== Gating criteria == |
||
| Line 29: | Line 35: | ||
=== Ansible path === |
=== Ansible path === |
||
{{#mermaid:flowchart LR |
|||
# Push change to ansible repo <code>staging</code> branch → staging nodes converge on next pull |
|||
STAGING["staging branch"] |
|||
# Merge to <code>canary</code> branch → canary nodes pull and apply |
|||
CANARY["canary branch"] |
|||
# Merge to <code>regional/<state></code> branch → that state's nodes converge |
|||
REGIONAL["regional/<state> branch"] |
|||
# Merge to <code>main</code> → fleet-wide rollout |
|||
MAIN["main branch"] |
|||
STAGING -->|"merge"| CANARY |
|||
CANARY -->|"merge"| REGIONAL |
|||
REGIONAL -->|"merge"| MAIN |
|||
}} |
|||
=== NixOS path === |
=== NixOS path === |
||
{{#mermaid:flowchart LR |
|||
STAGING["Update flake → rebuild in staging"] |
|||
CANARY["deploy-rs .#canary-node<br/>or merge to canary branch"] |
|||
REGIONAL["Tag release → comin picks up tag"] |
|||
# For fleet: update <code>main</code> branch of flake → all comin agents pull new generation |
|||
FLEET["Merge to main → comin pulls"] |
|||
STAGING -->|"validate →"| CANARY |
|||
== Branch/tag structure == |
|||
CANARY -->|"gate →"| REGIONAL |
|||
REGIONAL -->|"gate →"| FLEET |
|||
}} |
|||
{{Navbox TELEICU}} |
|||
syntaxhighlight lang="text" |
|||
main ────────────── fleet-wide |
|||
├── regional/kerala |
|||
├── regional/maharashtra |
|||
├── … |
|||
└── canary |
|||
└── staging |
|||
syntaxhighlight |
|||
{{Related}} |
{{Related}} |
||
Latest revision as of 08:52, 6 July 2026
Both migration paths use the same staged rollout process to minimize fleet-wide risk.
Stages
Gating criteria
A stage must pass all checks before promoting to the next:
- All nodes report healthy to K3s control plane
- VictoriaMetrics shows no anomaly in key metrics
- Downtime classifier reports expected status (no false positives)
- Hub admin confirms no degradation in clinical workflow
- Rollback tested and confirmed working