Several central servers

One central server is enough for most sites: nodes keep routing while it is down, and only the console and configuration changes wait. For a console that must always be available, or one central server per region, run several on one shared database.

How it works

• All are active. Browsers and nodes can use any of them; the console shows every node whichever server it is opened on. Configuration changes, drain and resume, commands, node status and the live monitoring grid are shared through the database.
• Nodes fail over. Each node is given every central server’s address, nearest first. It uses the first that answers, moves to the next within seconds when it stops answering, and returns to its first choice once that is back (checked every five minutes).
• One leader. One server holds a leader lease and runs the work that must happen once: evaluating alerts, deleting old history, sending scheduled reports and HL7 status updates, tracking AI jobs. If it stops, another takes over within 30 seconds.
• A stopped server raises a central server not running alert. Remove a retired one from the list on the Nodes tab.

Setting it up

1. Make the database highly available (see The database), since every central server depends on it.

2. Choose how secrets are protected, before entering any. Secrets (the SMTP password, webhook URL, enrollment key, DICOMweb credentials, de-identification keys) are encrypted with a key every central server must be able to read:

   SecretProtection	Use
   Machine (default)	One central server on Windows. Only that server can read them.
   Account	Several Windows servers running the service as one domain account (a gMSA is ideal).
   SID=S-1-5-21-…	Several Windows servers whose service accounts are members of one AD group (a DPAPI-NG descriptor).
   Certificate	Linux, Docker, or mixed: every server uses the same secret-protection.pfx (copy it from the first).
   HashiCorpVault	A key in HashiCorp Vault’s Transit engine protects them. See Secrets in a vault.
   AzureKeyVault	A key in Azure Key Vault protects them. See Secrets in a vault.

   After changing it, re-enter the SMTP password and webhook, and generate a new enrollment key.

3. Install the central service on each server, with the same database and the same SECRETPROTECTION:

   msiexec /i Routes.Central.msi /qn CONNECTIONSTRING="Server=router-ag;Database=Routes;MultiSubnetFailover=True;Integrated Security=true;TrustServerCertificate=true" SECRETPROTECTION=Account SERVICEACCOUNT="CONTOSO\svc-routes" SERVICEPASSWORD="<password>" INSTANCENAME="central-east" ADMINGROUPS="CONTOSO\PACS-Admins"

   INSTANCENAME is the server’s name in the console (default: the computer name).

4. Give every node all the addresses, nearest first, separated by ;:

   msiexec /i Routes.Node.msi /qn CENTRALURL="https://central-east:5080/;https://central-west:5080/" APIKEY="<enrollment key>"

   With self-signed certificates, give each server’s thumbprint too, separated by ;.

5. For the console, put the servers behind a load balancer with one name, or open any of them directly. Sign-in sessions are accepted by every server, so a load balancer needs no session affinity. With an identity provider, register the load-balanced name’s redirect URI. If the load balancer or a reverse proxy ends TLS or hides the users’ addresses, list it under TrustedProxies, so that sign-in limits and the audit log see each user’s address rather than the proxy’s.

Upgrading

Upgrade the central servers one at a time; nodes and browsers use the others meanwhile. See Upgrades and backups.

Secrets in a vault

With SecretProtection set to HashiCorpVault or AzureKeyVault, the key that protects the stored secrets is held in the vault and never leaves it: the secrets stay in the database, encrypted with keys that only the vault can unwrap.

• A copy of the database, or a backup of it, reveals no secret without access to the vault; revoking that access locks them.
• The vault’s audit log shows each use of the key. Central servers use it when they start (and when Data Protection makes a new key, every 10 years), not for every secret, so the vault being briefly away does not stop them.
• Every central server uses the same vault and key; none needs a file copied to it.
• Moving an existing installation: set the vault settings and restart the central servers. Keys protected the old way (DPAPI, the certificate) move to the vault as each server starts; the old protection is then no longer needed. A key one server cannot read (another server’s Machine DPAPI) waits for that server.

HashiCorp Vault

Enable the Transit engine and make a key (vault secrets enable transit, vault write -f transit/keys/routes), and a policy allowing update on transit/encrypt/routes and transit/decrypt/routes. Then:

Setting
VaultAddress	https://vault.example.org:8200
VaultTransitMount, VaultTransitKey	transit and routes by default.
VaultNamespace	Vault Enterprise or HCP.
VaultRoleId, VaultSecretId	AppRole sign-in (VaultAppRoleMount, default approle).
VaultKubernetesRole	In Kubernetes: the pod’s service account signs in (VaultKubernetesMount, default kubernetes).
VaultToken	A token instead (or the VAULT_TOKEN environment variable); it must be renewed outside Routes.

Azure Key Vault

Make an RSA key (2048 bits or more), and give the central servers’ identity the Key Vault Crypto Service Encryption User role on it (wrap and unwrap). Then:

Setting
AzureKeyVaultKey	The key’s identifier, https://<vault>.vault.azure.net/keys/<name>, without a version: the newest wraps new keys, and each key’s own version reads it back (keep old versions enabled).
(none)	On Azure, the machine’s or app’s managed identity signs in; AzureClientId picks a user-assigned one.
AzureTenantId, AzureClientId, AzureClientSecret	Elsewhere: a service principal.
AzureAuthorityHost	Another cloud, such as https://login.microsoftonline.us.

Give the sign-in secrets (VaultSecretId, VaultToken, AzureClientSecret) as environment variables or in the registry, readable by the service only; they are never in the database or the console.