curriculum-project-hub/hub/deploy/README.md

# Alpha Silo service installation

For a brand-new Organization, start with the repeatable wizard and end-to-end
operator runbook:

```sh
bash hub/deploy/new_silo.sh
```

See [NEW_SILO_RUNBOOK.md](NEW_SILO_RUNBOOK.md). The remainder of this document
describes the individual installer and maintenance primitives used by the
generated bundle.

## CI: roll Hub + admin SPA by fleet (`educraft` / `educraft-dev`)

`.gitea/workflows/deploy-admin.yml` deploys Hub releases (including the org-admin
SPA) to the managed Alpha host via `deploy_fleet_release.sh`. Fleets share the
host and are selected by the middle DNS label of `HUB_PUBLIC_BASE_URL`:

| Fleet | Domain | CI trigger |
|-------|--------|------------|
| **dev** | `https://<slug>.educraft-dev.paradigm-edu.net` | every push + PR (paths under `hub/`) |
| **prod** | `https://<slug>.educraft.paradigm-edu.net` | push to `main` + tags |

Example prod tenant:
`https://para-26071100.educraft.paradigm-edu.net/auth/feishu/para-26071100`.

Required Gitea secret: `DEPLOY_SSH_KEY` (private key for `root@39.107.254.4`).
Optional: `DEPLOY_HOST` / `DEPLOY_USER` / `DEPLOY_SSH_PORT` / `DEPLOY_BASE`.
Set repository var `ALLOW_EMPTY_FLEET=1` until the first silo exists for a fleet.

Local dry-run against the host:

```sh
CPH_FLEET=dev \
  PLATFORM_DEPLOY_SSH_KEY=$HOME/.ssh/id_ed25519 \
  bash hub/deploy/deploy_fleet_release.sh
```

The supervised alpha runs one Organization per named Silo. The supported host
has systemd, PostgreSQL, Node.js 24+, `pg_isready`, `runuser`, `setpriv`,
bubblewrap, `socat`, `pg_dump`, `tar`, `sha256sum`, and a compatible `cph`.

Each Silo needs its own logical database/role, instance id, service user,
environment, keyring, workspace, port, domain, and host filesystem quota.
Application code lives in immutable versioned directories under
`/srv/curriculum-project-hub/releases/`; releases contain no customer state.

## Install an instance

```sh
sudo BASE=/srv/curriculum-project-hub \
  HUB_DIR=/srv/curriculum-project-hub/releases/<release-id>/hub \
  INSTANCE_ID=org-a \
  WORKSPACE_ROOT=/w/997 \
  PORT=8788 \
  MEMORY_MAX=16G CPU_QUOTA=400% TASKS_MAX=512 \
  bash /srv/curriculum-project-hub/releases/<release-id>/hub/deploy/install_service.sh
```

The first invocation creates root-owned `0600` files below
`/srv/curriculum-project-hub/.secrets/org-a/` and exits 78. Copy the keyring to
a separately protected recovery destination and fill every blank in
`platform.env`. Before rerunning the installer, migrate the empty Silo database:

```sh
sudo bash -c '
  set -euo pipefail
  set -a; . /srv/curriculum-project-hub/.secrets/org-a/platform.env; set +a
  node /srv/curriculum-project-hub/releases/<release-id>/hub/node_modules/prisma/build/index.js \
    migrate deploy \
    --schema /srv/curriculum-project-hub/releases/<release-id>/hub/prisma/schema.prisma
'
```

Then rerun the installer so it can complete preflight and install the stopped
unit. Resource numbers above are examples, not defaults; the operator must set
measured ceilings explicitly.

The installer refuses overlapping release/persistent paths, root service users,
missing Node/cph/PostgreSQL/bubblewrap prerequisites, invalid credentials, and
unreachable database roles. The installed unit is `cph-hub-org-a.service` and
uses systemd `LoadCredential`; the service user cannot read the source keyring.

Default state paths are:

```text
/var/lib/cph-hub/org-a/home
/var/lib/cph-hub/org-a/state
/w/997
/var/cache/cph-hub/org-a
```

Organization Agent roles and skills are runtime configuration. Skill versions
are stored below the Silo state directory (`state/skills`) and are included in
`backup_silo.sh` as `agent-skills.tar`; PostgreSQL stores role bundles, skill
metadata and role-to-skill selection. Operate them as the Silo service user so
content ownership remains correct:

```sh
sudo INSTANCE_ID=org-a \
  ENV_FILE=/srv/curriculum-project-hub/.secrets/org-a/platform.env \
  bash hub/deploy/agent_config.sh install-skill \
  --organization org-a --source /staging/typst --version 1
sudo INSTANCE_ID=org-a \
  ENV_FILE=/srv/curriculum-project-hub/.secrets/org-a/platform.env \
  bash hub/deploy/agent_config.sh upsert-role \
  --organization org-a --role draft --label 草稿 --tools-json null
sudo INSTANCE_ID=org-a \
  ENV_FILE=/srv/curriculum-project-hub/.secrets/org-a/platform.env \
  bash hub/deploy/agent_config.sh set-role-skills \
  --organization org-a --role draft --skills outline,lesson-project,typst
sudo INSTANCE_ID=org-a \
  ENV_FILE=/srv/curriculum-project-hub/.secrets/org-a/platform.env \
  bash hub/deploy/agent_config.sh list --organization org-a
```

`--tools-json null` means the full registered tool surface; `[]` means no
ordinary tools. SDK-bundled skills and workspace/user setting sources remain
disabled regardless of runtime configuration.

## Bootstrap the only Organization

Prepare a root-owned `0600` JSON file containing
`organization`, `owner`, `feishu`, `provider`, and optionally `teams`. Secrets
never appear in command-line arguments.

```json
{
  "organization": { "id": "org_a", "slug": "org-a", "name": "Org A" },
  "owner": { "openId": "ou_owner", "displayName": "Owner" },
  "feishu": {
    "appId": "cli_xxx",
    "appSecret": "...",
    "botOpenId": "ou_bot"
  },
  "provider": {
    "providerId": "openrouter",
    "baseUrl": "https://openrouter.ai/api",
    "authToken": "..."
  },
  "teams": [{ "slug": "teachers", "name": "Teachers" }]
}
```

```sh
sudo bash -c '
  set -euo pipefail
  set -a; . /srv/curriculum-project-hub/.secrets/org-a/platform.env; set +a
  node /srv/curriculum-project-hub/releases/<release-id>/hub/dist/deployment/bootstrap-silo-cli.js \
    --config-file /root/org-a-bootstrap.json \
    --keyring-file /srv/curriculum-project-hub/.secrets/org-a/secret-keyring.json
'
```

Bootstrap encrypts and activates both connection records and is rerunnable when
provider configuration fails after Organization creation. Hub startup refuses
to become healthy unless the sole Organization, active provider, active Feishu
application, and Feishu listener are ready. Delete the bootstrap input after
the recovery copy is current. Production has no process-global Feishu/provider
credential fallback.

## Start and rotate

```sh
sudo systemctl start cph-hub-org-a.service
sudo systemctl status cph-hub-org-a.service
```

KEK rotation requires the named unit to be stopped. Source its instance env so
`HUB_SYSTEMD_UNIT` and `DATABASE_URL` identify the same Silo:

```sh
sudo bash -c '
  set -euo pipefail
  systemctl stop cph-hub-org-a.service
  set -a; . /srv/curriculum-project-hub/.secrets/org-a/platform.env; set +a
  node /srv/curriculum-project-hub/hub/dist/deployment/rotate-secret-kek.js \
    --keyring-file /srv/curriculum-project-hub/.secrets/org-a/secret-keyring.json
  systemctl start cph-hub-org-a.service
'
```

## Backup and restore drill (recommended after the demo is running)

Stop the unit and run `backup_silo.sh` with distinct root-owned destinations:

```sh
sudo INSTANCE_ID=org-a \
  ENV_FILE=/srv/curriculum-project-hub/.secrets/org-a/platform.env \
  KEYRING_FILE=/srv/curriculum-project-hub/.secrets/org-a/secret-keyring.json \
  BUSINESS_BACKUP_DIR=/backup/business \
  RECOVERY_BACKUP_DIR=/separate-recovery \
  bash hub/deploy/backup_silo.sh
```

The business set contains the PostgreSQL custom dump, workspace archive and
`agent-skills.tar`. The
separate recovery set contains the keyring and environment. Both include
checksums; neither destination may be the live host's only disk.

Restore into a separate drill database, workspace and skill-store directory;
verify checksums before extracting both tar archives, then run:

```sh
set -a; . /path/to/restored/platform.env; set +a
mkdir -p "$HUB_PROJECT_WORKSPACE_ROOT" "$HUB_SKILL_STORE_ROOT"
tar -xf /path/to/business/workspaces.tar -C "$HUB_PROJECT_WORKSPACE_ROOT"
tar -xf /path/to/business/agent-skills.tar -C "$HUB_SKILL_STORE_ROOT"
node hub/dist/deployment/restore-preflight.js \
  --keyring-file /path/to/restored/secret-keyring.json
sudo INSTANCE_ID=org-a ENV_FILE=/path/to/restored/platform.env \
  HUB_DIR=/path/to/restored/release/hub \
  bash hub/deploy/agent_config.sh verify-store --organization org-a
```

Traffic stays disabled until the sole Organization and every Feishu/provider
envelope authenticate, the workspace exists, and an end-to-end test run passes.

For the first supervised demo, install → bootstrap → start → health check is the
deployment gate. A completed off-host restore drill is an Alpha hardening item,
not a prerequisite for bringing up that first controlled Silo.

The default bind remains loopback; expose it only through a TLS reverse proxy.
The platform admin surface is not part of the alpha and must not be exposed.