forked from EduCraft/curriculum-project-hub
feat: add deployable alpha silo
This commit is contained in:
+126
-76
@@ -1,101 +1,151 @@
|
||||
# Hub service installation
|
||||
# Alpha Silo service installation
|
||||
|
||||
The initial production target is a supported Linux host with systemd,
|
||||
PostgreSQL, Node.js 20 or newer, `pg_isready`, `runuser`, `setpriv`, bubblewrap,
|
||||
`socat` and `cph`. The Hub runs as the dedicated non-login `cph-hub` user; do not
|
||||
create or run the unit as root.
|
||||
The supervised alpha runs one Organization per named Silo. The supported host
|
||||
has systemd, PostgreSQL, Node.js 24+, `pg_isready`, `runuser`, `setpriv`,
|
||||
bubblewrap, `socat`, `pg_dump`, `tar`, `sha256sum`, and a compatible `cph`.
|
||||
|
||||
Build or deploy the Hub under `/srv/curriculum-project-hub`, then run:
|
||||
Each Silo needs its own logical database/role, instance id, service user,
|
||||
environment, keyring, workspace, port, domain, and host filesystem quota.
|
||||
Application code lives in immutable versioned directories under
|
||||
`/srv/curriculum-project-hub/releases/`; releases contain no customer state.
|
||||
|
||||
## Install an instance
|
||||
|
||||
```sh
|
||||
sudo BASE=/srv/curriculum-project-hub \
|
||||
HUB_DIR=/srv/curriculum-project-hub/hub \
|
||||
bash /srv/curriculum-project-hub/hub/deploy/install_service.sh
|
||||
HUB_DIR=/srv/curriculum-project-hub/releases/<release-id>/hub \
|
||||
INSTANCE_ID=org-a \
|
||||
PORT=8788 \
|
||||
MEMORY_MAX=16G CPU_QUOTA=400% TASKS_MAX=512 \
|
||||
bash /srv/curriculum-project-hub/releases/<release-id>/hub/deploy/install_service.sh
|
||||
```
|
||||
|
||||
On a clean host the first invocation creates
|
||||
`/srv/curriculum-project-hub/.secrets/platform.env` and a random root-owned
|
||||
`secret-keyring.json`, then exits with status 78. Copy the keyring to a
|
||||
separately protected recovery location before continuing; it must not live only
|
||||
on this host or in the ordinary database/workspace backup. Fill every required
|
||||
environment blank, set `CPH_BIN` to an executable compatible `cph`, keep both
|
||||
source files root-only at mode `0600`, and run the same command again. No
|
||||
systemd unit is installed until configuration,
|
||||
paths, Node, bubblewrap, `socat`, `cph --version`, PostgreSQL connectivity and directory
|
||||
ownership all pass preflight.
|
||||
|
||||
The service account cannot read the keyring source. The installed unit uses
|
||||
systemd `LoadCredential` to materialize a read-only `cph-secret-keyring` for the
|
||||
running service. Hub refuses production startup if this credential is missing,
|
||||
malformed, or does not contain its declared active key. Provider credentials
|
||||
are configured through the Organization admin connection API, never through
|
||||
process-global `ANTHROPIC_*` variables; preflight rejects those legacy settings.
|
||||
|
||||
Rotate the local KEK only from the controlled host console. The command first
|
||||
atomically adds a new active key while retaining every previous key, then
|
||||
authenticates and rewraps every stored Provider and Feishu Application DEK,
|
||||
writes redacted org audit events, and verifies the complete set again:
|
||||
The first invocation creates root-owned `0600` files below
|
||||
`/srv/curriculum-project-hub/.secrets/org-a/` and exits 78. Copy the keyring to
|
||||
a separately protected recovery destination and fill every blank in
|
||||
`platform.env`. Before rerunning the installer, migrate the empty Silo database:
|
||||
|
||||
```sh
|
||||
sudo bash -c '
|
||||
set -euo pipefail
|
||||
systemctl stop cph-hub.service
|
||||
set -a
|
||||
. /srv/curriculum-project-hub/.secrets/platform.env
|
||||
set +a
|
||||
node /srv/curriculum-project-hub/hub/dist/deployment/rotate-secret-kek.js \
|
||||
--keyring-file /srv/curriculum-project-hub/.secrets/secret-keyring.json
|
||||
systemctl start cph-hub.service
|
||||
set -a; . /srv/curriculum-project-hub/.secrets/org-a/platform.env; set +a
|
||||
node /srv/curriculum-project-hub/releases/<release-id>/hub/node_modules/prisma/build/index.js \
|
||||
migrate deploy \
|
||||
--schema /srv/curriculum-project-hub/releases/<release-id>/hub/prisma/schema.prisma
|
||||
'
|
||||
```
|
||||
|
||||
The CLI refuses to rotate while `cph-hub.service` is active; stopping the only
|
||||
writer prevents a concurrent BYOK version from being appended under the old
|
||||
in-memory keyring. A PostgreSQL transaction advisory lock excludes a second
|
||||
rotation and is released automatically if the CLI or host crashes. The
|
||||
operation is therefore rerunnable after interruption: old and new keys remain
|
||||
in the atomically replaced keyring, so partially rewrapped rows stay
|
||||
decryptable. It also refuses success while any envelope still names a
|
||||
non-active KEK. Back up the updated keyring to the separate recovery location
|
||||
and complete a restore preflight before considering old-key retirement. The
|
||||
pilot deliberately does not auto-delete previous keys.
|
||||
Then rerun the installer so it can complete preflight and install the stopped
|
||||
unit. Resource numbers above are examples, not defaults; the operator must set
|
||||
measured ceilings explicitly.
|
||||
|
||||
The deployment path also runs `npm run audit:production` from the locked clean
|
||||
install before building or restarting the service. A current high or critical
|
||||
production advisory therefore blocks deployment instead of becoming a warning
|
||||
buried in CI output.
|
||||
The installer refuses overlapping release/persistent paths, root service users,
|
||||
missing Node/cph/PostgreSQL/bubblewrap prerequisites, invalid credentials, and
|
||||
unreachable database roles. The installed unit is `cph-hub-org-a.service` and
|
||||
uses systemd `LoadCredential`; the service user cannot read the source keyring.
|
||||
|
||||
An existing service account is accepted only when its primary group, home and
|
||||
non-login shell match the requested configuration and it has no supplementary
|
||||
groups. After provisioning, the installer executes path-access, built Hub,
|
||||
Prisma, an authenticated database query, `cph`, `socat` and bubblewrap namespace probes
|
||||
as that account with `no_new_privs` set. Inaccessible parents, UID-specific
|
||||
database/network policy, or a host that forbids unprivileged user namespaces
|
||||
therefore fail before unit installation.
|
||||
|
||||
The default persistent layout is deliberately outside every source/release
|
||||
tree:
|
||||
Default state paths are:
|
||||
|
||||
```text
|
||||
/var/lib/cph-hub/home
|
||||
/var/lib/cph-hub/state
|
||||
/var/lib/cph-hub/workspaces
|
||||
/var/cache/cph-hub
|
||||
/var/lib/cph-hub/org-a/home
|
||||
/var/lib/cph-hub/org-a/state
|
||||
/var/lib/cph-hub/org-a/workspaces
|
||||
/var/cache/cph-hub/org-a
|
||||
```
|
||||
|
||||
Custom paths are supported with `SERVICE_HOME`, `STATE_DIR`, `CACHE_DIR` and
|
||||
`WORKSPACE_ROOT`, but all must be absolute. The installer rejects any persistent
|
||||
path that is equal to, above, or below `BASE`; this keeps rsync, release
|
||||
switching, rollback and pruning unable to traverse customer workspaces. Set the
|
||||
same custom workspace in `HUB_PROJECT_WORKSPACE_ROOT` inside `platform.env`.
|
||||
## Bootstrap the only Organization
|
||||
|
||||
After a successful installation:
|
||||
Prepare a root-owned `0600` JSON file containing
|
||||
`organization`, `owner`, `feishu`, `provider`, and optionally `teams`. Secrets
|
||||
never appear in command-line arguments.
|
||||
|
||||
```json
|
||||
{
|
||||
"organization": { "id": "org_a", "slug": "org-a", "name": "Org A" },
|
||||
"owner": { "openId": "ou_owner", "displayName": "Owner" },
|
||||
"feishu": {
|
||||
"appId": "cli_xxx",
|
||||
"appSecret": "...",
|
||||
"botOpenId": "ou_bot"
|
||||
},
|
||||
"provider": {
|
||||
"providerId": "openrouter",
|
||||
"baseUrl": "https://openrouter.ai/api",
|
||||
"authToken": "..."
|
||||
},
|
||||
"teams": [{ "slug": "teachers", "name": "Teachers" }]
|
||||
}
|
||||
```
|
||||
|
||||
```sh
|
||||
sudo systemctl start cph-hub.service
|
||||
sudo systemctl status cph-hub.service
|
||||
sudo bash -c '
|
||||
set -euo pipefail
|
||||
set -a; . /srv/curriculum-project-hub/.secrets/org-a/platform.env; set +a
|
||||
node /srv/curriculum-project-hub/releases/<release-id>/hub/dist/deployment/bootstrap-silo-cli.js \
|
||||
--config-file /root/org-a-bootstrap.json \
|
||||
--keyring-file /srv/curriculum-project-hub/.secrets/org-a/secret-keyring.json
|
||||
'
|
||||
```
|
||||
|
||||
`HOST` and `PORT` in `platform.env` are passed to the real HTTP listener. The
|
||||
default `127.0.0.1:8788` expects a local TLS reverse proxy; the public OAuth URL
|
||||
must be an HTTPS `HUB_PUBLIC_BASE_URL`.
|
||||
Bootstrap encrypts and activates both connection records and is rerunnable when
|
||||
provider configuration fails after Organization creation. Hub startup refuses
|
||||
to become healthy unless the sole Organization, active provider, active Feishu
|
||||
application, and Feishu listener are ready. Delete the bootstrap input after
|
||||
the recovery copy is current. Production has no process-global Feishu/provider
|
||||
credential fallback.
|
||||
|
||||
## Start and rotate
|
||||
|
||||
```sh
|
||||
sudo systemctl start cph-hub-org-a.service
|
||||
sudo systemctl status cph-hub-org-a.service
|
||||
```
|
||||
|
||||
KEK rotation requires the named unit to be stopped. Source its instance env so
|
||||
`HUB_SYSTEMD_UNIT` and `DATABASE_URL` identify the same Silo:
|
||||
|
||||
```sh
|
||||
sudo bash -c '
|
||||
set -euo pipefail
|
||||
systemctl stop cph-hub-org-a.service
|
||||
set -a; . /srv/curriculum-project-hub/.secrets/org-a/platform.env; set +a
|
||||
node /srv/curriculum-project-hub/hub/dist/deployment/rotate-secret-kek.js \
|
||||
--keyring-file /srv/curriculum-project-hub/.secrets/org-a/secret-keyring.json
|
||||
systemctl start cph-hub-org-a.service
|
||||
'
|
||||
```
|
||||
|
||||
## Backup and restore drill (recommended after the demo is running)
|
||||
|
||||
Stop the unit and run `backup_silo.sh` with distinct root-owned destinations:
|
||||
|
||||
```sh
|
||||
sudo INSTANCE_ID=org-a \
|
||||
ENV_FILE=/srv/curriculum-project-hub/.secrets/org-a/platform.env \
|
||||
KEYRING_FILE=/srv/curriculum-project-hub/.secrets/org-a/secret-keyring.json \
|
||||
BUSINESS_BACKUP_DIR=/backup/business \
|
||||
RECOVERY_BACKUP_DIR=/separate-recovery \
|
||||
bash hub/deploy/backup_silo.sh
|
||||
```
|
||||
|
||||
The business set contains the PostgreSQL custom dump and workspace archive. The
|
||||
separate recovery set contains the keyring and environment. Both include
|
||||
checksums; neither destination may be the live host's only disk.
|
||||
|
||||
Restore into a separate drill database/workspace, verify checksums, then run:
|
||||
|
||||
```sh
|
||||
set -a; . /path/to/restored/platform.env; set +a
|
||||
node hub/dist/deployment/restore-preflight.js \
|
||||
--keyring-file /path/to/restored/secret-keyring.json
|
||||
```
|
||||
|
||||
Traffic stays disabled until the sole Organization and every Feishu/provider
|
||||
envelope authenticate, the workspace exists, and an end-to-end test run passes.
|
||||
|
||||
For the first supervised demo, install → bootstrap → start → health check is the
|
||||
deployment gate. A completed off-host restore drill is an Alpha hardening item,
|
||||
not a prerequisite for bringing up that first controlled Silo.
|
||||
|
||||
The default bind remains loopback; expose it only through a TLS reverse proxy.
|
||||
The platform admin surface is not part of the alpha and must not be exposed.
|
||||
|
||||
Reference in New Issue
Block a user