docs: define SaaS capacity controls

.scratch/saas-production-readiness/assets/initial-abuse-capacity-controls.md

@@ -0,0 +1,125 @@
+# Initial abuse and capacity controls
+
+## Verdict
+
+The initial production boundary requires hard safety controls before customer
+traffic. The current project lock and small in-memory trigger queue are useful
+local protections, but they do not provide multi-tenant admission, durable
+backpressure, storage safety, request isolation, bounded execution, or an
+operator stop mechanism.
+
+The accepted contract is recorded in
+[ADR-0022](../../../docs/adr/0022-layered-capacity-admission.md) and
+[`Spec.System.Capacity`](../../../spec/Spec/System/Capacity.lean). Exact values
+remain deliberately open until production-like calibration; the required
+dimensions and failure behaviors do not.
+
+## Current implementation inventory
+
+| Surface | Existing protection | Production gap |
+| --- | --- | --- |
+| HTTP | Fastify's default 1 MiB body limit | No route-class, user, Org, IP, or administrative-write rate limits; request/connection/handler timeouts are unset |
+| Project execution | Database lock permits one active run per Project | No platform or Org concurrency ceiling and no fair scheduler |
+| Busy Project | Process-local FIFO, default five items and five-minute wait | Restart loses accepted work; expiry is logged and dropped without a durable state or user notification |
+| Agent execution | Global max-turn default of 25 | No wall-clock, tool-call, output, memory, CPU, process, or Org-specific budget |
+| Feishu messages | Message batching limits text count/characters | File/post attachment count and size are unbounded; downloads may buffer whole files |
+| Storage | Workspace path boundary | No Project or Org byte quota, archive-expansion limit, or entity-count ceiling |
+| Usage | Provider-reported token/cost fields and Org aggregation | Missing cost can disappear from totals; no soft-threshold notification or provider-mode attribution contract |
+| Host | systemd lifecycle | No service CPU, memory, or task ceiling and no audited workload brake |
+
+Most Organization member/team/project lists are also unpaginated, and usage
+aggregation can load all matching runs. Those query shapes must be bounded as
+part of enforcement rather than hidden behind larger process memory.
+
+## Accepted contract
+
+### Layered limits
+
+- Every mandatory dimension has a versioned, non-overridable platform ceiling.
+- An Organization OWNER/ADMIN may configure a lower policy limit.
+- Effective limit is `min(platform, organization)`; absent Organization policy
+  means the platform ceiling, never unlimited.
+- Missing or invalid mandatory production ceilings fail startup.
+
+### Admission and fairness
+
+- Project exclusivity remains one active run.
+- Platform and Organization concurrency limits apply before starting work.
+- Available slots are scheduled fairly across Organizations, FIFO within an
+  Organization, so one tenant cannot monopolize a global FIFO.
+- Accepted work is durable and queryable. Queue length and wait are bounded.
+- Overflow is explicit `capacity_exhausted` (`429` plus `Retry-After` for HTTP;
+  a clear busy response for Feishu). Nothing is silently dropped.
+- Wait timeout produces terminal `EXPIRED`, not a later automatic start.
+- The initiator may withdraw their queued request. Project `MANAGE` or an
+  Organization admin may cancel queued requests in their scope. Actor, reason,
+  and notification are recorded.
+
+### Retry and idempotency
+
+- Feishu `event_id` and API idempotency identify one admission across retries.
+- Once a run starts, the whole run is never automatically replayed because tool
+  side effects may already exist.
+- Only bounded transport retries before model output or tool side effects are
+  automatic. Later failure/timeouts require an explicit new request.
+
+### Request, file, storage, and entity safety
+
+- Authenticated Web requests are limited by user, Organization, and endpoint
+  class; signed Feishu events by Organization app, chat, and sender;
+  administrative writes use stricter independent buckets.
+- Admin surfaces require both a valid user session and the corresponding
+  platform/Organization administrator authorization. OAuth callbacks and
+  signed Feishu ingress remain necessary non-session entry points.
+- Hard limits cover request body, single-file bytes, attachment count, archive
+  expanded bytes/file count/depth, Project/Organization storage, and per-type
+  Organization entity counts.
+- Transfer is streamed and counted; partial files are removed after failure.
+- Crossing a ceiling blocks new growth without auto-deleting or hiding existing
+  resources. Read, export, delete, and administrator remediation stay usable.
+
+### Run budget
+
+Every run is bounded by wall time, model turns, tool-call count, per-tool time,
+output/log/event bytes, and process memory/CPU/child-process count. A breached
+budget ends explicitly as `TIMED_OUT` or `LIMIT_EXCEEDED`, records the dimension
+and observed usage, releases capacity, and never masquerades as success.
+
+### Usage and cost
+
+- Token usage, provider-reported cost, run count, and duration are attributed by
+  Organization, Project, Run, model, and Provider Connection.
+- Missing provider cost is `unknown`, never zero.
+- Thresholds are soft alerts in the pilot; they do not automatically stop runs
+  and are not payment settlement.
+- Org admins receive Organization alerts. Platform admins additionally receive
+  anomalous-cost alerts for platform-managed connections; BYOK does not trigger
+  platform financial alerts.
+
+### Emergency control
+
+An audited Platform Administrator control supports Organization or global
+`DRAIN` (reject new work, pause queue starts, finish active runs) and `STOP_NOW`
+(also cancel active runs). Both require an actor, reason, timestamp, explicit
+recovery, and user-visible outcomes. This is workload control, not Organization
+deletion or account suspension.
+
+## Required production evidence
+
+1. Production configuration validation fails closed for every missing or
+   invalid platform ceiling.
+2. Concurrent noisy-neighbor tests prove platform/Org/Project concurrency and
+   Organization-fair scheduling without starvation.
+3. Restart and crash tests prove accepted queue entries, idempotency, expiry,
+   cancellation, and notifications survive process loss.
+4. HTTP and Feishu load tests prove every rate-limit identity and typed overload
+   response, including retry guidance.
+5. File/archive/storage probes prove streaming enforcement, cleanup, and
+   continued read/delete access while over limit.
+6. Hung provider/tool and resource-exhaustion probes prove every run/process
+   budget terminates, records cause, and releases locks/capacity.
+7. Usage tests prove provider-mode attribution, unknown-cost handling, alert
+   routing, and absence of hard monetary blocking.
+8. `DRAIN` and `STOP_NOW` drills prove scoped, audited, reversible behavior.
+9. Production-like calibration publishes the tested numerical defaults and
+   capacity headroom used by the release gate.

.scratch/saas-production-readiness/assets/tenant-auth-security-audit.md

@@ -29,8 +29,10 @@ The audit used these upstream requirements:
   run's project workspace.
 - ADR-0019: authorization is evaluated against principals resolved at request
   time so membership changes take effect immediately.
-- ADR-0021: customer Feishu and model-provider credentials are org-scoped
+- ADR-0021: Feishu and model-provider credentials are org-scoped secrets,
-  secrets, encrypted at rest, and accessed through a resolver.
+  encrypted at rest, and accessed through a resolver. Provider ownership may be
+  Organization-managed BYOK or a distinct platform-managed connection for that
+  Organization; neither mode permits a process-global shared key.
 
 ## Confirmed protections
 
@@ -130,10 +132,10 @@ encrypted credentials "deferred", although ADR-0021 says they **must** be
 org-scoped, encrypted at rest, and accessed through a resolver.
 
 `EnvRuntimeSettings.provider` similarly discards its `scope` argument and uses
-one global OpenRouter token for every project. This both leaks tenant ownership
+one global OpenRouter token for every project. This erases the accepted
-of credentials and makes the accepted "customers supply their own provider
+per-Organization provider mode and makes both BYOK and platform-managed cost
-keys" accounting model false. The schema has no encrypted credential store,
+attribution false. The schema has no encrypted credential store, key version,
-key version, rotation state, or resolver seam.
+rotation state, or resolver seam.
 
 This is also a product reachability blocker: one customer-owned Feishu app
 cannot receive bot/OAuth traffic for unrelated customer tenants under the
@@ -248,7 +250,9 @@ Production readiness requires automated proof of all of the following:
    organization, project, run, and pending action consistently.
 4. ACTIVE Organization status is enforced in the shared authorization seam.
 5. Feishu and provider credentials resolve by Organization, are encrypted at
-   rest with versioned keys, never enter logs/agent tools, and can rotate.
+   rest with versioned keys, never enter logs/agent tools, and can rotate;
+   provider resolution honors either Organization-managed BYOK or a distinct
+   platform-managed connection for that Organization.
 6. Cross-org HTTP and Prisma service tests remain green, while new negative
    tests cover every finding above.
 7. Production Node and Rust dependency audits pass with no ignored

.scratch/saas-production-readiness/issues/05-define-abuse-capacity-controls.md

@@ -1,7 +1,7 @@
 # Define initial abuse and capacity controls
 
 Type: grilling
-Status: open
+Status: resolved
 Blocked by: 02, 03
 
 ## Question
@@ -9,3 +9,35 @@ Blocked by: 02, 03
 Which request limits, file limits, concurrency limits, agent budgets, tenant
 quotas, and backpressure policies are mandatory for the accepted initial
 production boundary, and which values or behaviors remain product decisions?
+
+## Answer
+
+The initial production service requires layered, fail-closed capacity controls:
+every mandatory dimension has a versioned non-overridable platform ceiling,
+while an Organization may configure only a lower policy limit. Agent work uses
+a durable bounded queue, fair scheduling across Organizations, FIFO within an
+Organization, explicit overload/expiry/cancellation states, and no automatic
+whole-run replay after execution starts.
+
+Hard controls cover request rate/body size, concurrency and queueing, files and
+archive expansion, Project/Organization storage and entity counts, run/tool
+budgets, and process resources. Reaching a limit blocks new growth without
+deleting or hiding existing data. Token and provider cost remain attributed
+soft-alert signals in the pilot, not hard spend controls; unknown cost is not
+zero. Platform administrators also receive audited `DRAIN` and `STOP_NOW`
+workload brakes.
+
+The full current-state inventory, accepted behavior, and release evidence are
+recorded in [Initial abuse and capacity controls](../assets/initial-abuse-capacity-controls.md),
+with the durable decision in ADR-0022 and `Spec.System.Capacity`. Numerical
+ceilings remain open until production-like calibration.
+
+The implementation frontier is:
+
+- [Implement layered limits and multi-dimensional request rate limiting](32-layered-limits-rate-limiting.md)
+- [Build the durable Organization-fair admission scheduler](33-durable-fair-admission-scheduler.md)
+- [Enforce file, archive, storage, and entity budgets](34-file-storage-budgets.md)
+- [Enforce Agent Run and process resource budgets](35-run-runtime-budgets.md)
+- [Implement provider-mode usage attribution and soft alerts](36-usage-soft-alerts.md)
+- [Add audited emergency workload brakes](37-emergency-workload-brakes.md)
+- [Calibrate and prove production capacity ceilings](38-calibrate-production-capacity.md)

.scratch/saas-production-readiness/issues/07-audit-product-surface-completeness.md

@@ -6,7 +6,8 @@ Status: open
 ## Question
 
 Against ADR-0020 and ADR-0021, which required platform-admin, org-admin,
-project-onboarding, team/access, session, usage, and Feishu flows are absent,
+project-onboarding, team/access, session, usage, provider-mode, Organization
+policy-limit, emergency workload-control, and Feishu flows are absent,
 unreachable, or only test-facing rather than usable in the initial production
-service?
+service? Treat ADR-0022's capacity control surfaces as part of the accepted
-
+product boundary.

.scratch/saas-production-readiness/issues/08-prove-production-release-gate.md

@@ -2,7 +2,7 @@
 
 Type: task
 Status: open
-Blocked by: 01, 02, 03, 04, 05, 06, 07, 09, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31
+Blocked by: 01, 02, 03, 04, 05, 06, 07, 09, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38
 
 ## Question
 

.scratch/saas-production-readiness/issues/19-design-org-scoped-secret-control-plane.md

@@ -7,6 +7,7 @@ Status: open
 
 Which key-management, envelope-encryption, rotation, connection identity, and
 runtime resolver design will implement ADR-0021's org-scoped Feishu and model
-provider credentials, support one customer-owned Feishu app per tenant, and
+provider credentials; support one customer-owned Feishu app per Organization;
-ensure plaintext credentials never reach business records, logs, or agent
+support Organization-managed BYOK and a distinct platform-managed provider
-tools?
+connection per Organization; and ensure plaintext credentials never reach
+business records, logs, or agent tools?

.scratch/saas-production-readiness/issues/23-durable-feishu-inbox-and-queue.md

@@ -2,6 +2,7 @@
 
 Type: task
 Status: open
+Blocked by: 18
 
 ## Question
 
@@ -10,4 +11,6 @@ durable inbound work lifecycle that atomically claims duplicate events, records
 processing/completion/failure, survives restart, retries recoverable failures,
 and proves concurrent redelivery cannot duplicate a run or silently lose an
 accepted prompt. Preserve execution-time authorization from ticket 18 when
-deferred work is finally started.
+deferred work is finally started, and expose the durable identity/state needed
+by ADR-0022's Organization-fair admission scheduler rather than recreating a
+second queue lifecycle.

.scratch/saas-production-readiness/issues/29-decide-telemetry-slos-alert-routing.md

@@ -11,4 +11,6 @@ redaction/access/retention policy, initial service-level indicators and
 objectives, error budgets, alert thresholds, on-call destinations, escalation
 ownership, and telemetry-blind detection. Prefer the smallest operable control
 plane, but record every domain-visible availability or retry assumption in the
-appropriate ADR/spec rather than hiding it in dashboard configuration.
+appropriate ADR/spec rather than hiding it in dashboard configuration. Consume
+ADR-0022's already-decided usage/cost soft-alert ownership as input; do not
+reopen its hard-limit or provider-mode financial-alert semantics here.

.scratch/saas-production-readiness/issues/32-layered-limits-rate-limiting.md

@@ -0,0 +1,13 @@
+# Implement layered limits and multi-dimensional request rate limiting
+
+Type: task
+Status: open
+Blocked by: 20, 22
+
+## Question
+
+Implement ADR-0022's fail-closed platform ceilings, Organization-lowerable
+policy limits, bounded list/query behavior, and typed multi-dimensional rate
+limits for authenticated Web, signed Feishu, OAuth, and administrative writes.
+Prove effective limits never exceed platform ceilings and every overload result
+is observable, scoped, and carries retry guidance where applicable.

.scratch/saas-production-readiness/issues/33-durable-fair-admission-scheduler.md

@@ -0,0 +1,13 @@
+# Build the durable Organization-fair admission scheduler
+
+Type: task
+Status: open
+Blocked by: 18, 23, 24, 32
+
+## Question
+
+Extend the durable inbound lifecycle into ADR-0022's bounded Agent Admission
+Queue with platform/Organization/Project concurrency, fair scheduling across
+Organizations, FIFO within each Organization, durable position/state,
+expiration notification, scoped cancellation, idempotent admission, explicit
+capacity rejection, and no automatic replay after a run has started.

.scratch/saas-production-readiness/issues/34-file-storage-budgets.md

@@ -0,0 +1,13 @@
+# Enforce file, archive, storage, and entity budgets
+
+Type: task
+Status: open
+Blocked by: 17, 23, 32
+
+## Question
+
+Enforce ADR-0022's request/file/attachment/archive-expansion,
+Project/Organization storage, and Organization entity-count limits with
+streaming accounting, atomic reservation, cleanup of incomplete artifacts,
+bounded queries, explicit errors, and continued read/export/delete/admin
+remediation while an Organization is at or above a ceiling.

.scratch/saas-production-readiness/issues/35-run-runtime-budgets.md

@@ -0,0 +1,13 @@
+# Enforce Agent Run and process resource budgets
+
+Type: task
+Status: open
+Blocked by: 16, 24, 32
+
+## Question
+
+Implement ADR-0022's wall-time, turn, tool-count, per-tool-time,
+output/log/event-size, memory, CPU, and child-process budgets. Termination must
+be fail-fast and observable, persist `TIMED_OUT` or `LIMIT_EXCEEDED` with the
+triggered dimension and actual usage, cancel the provider/process safely, and
+release run locks and admission capacity under every injected failure.

.scratch/saas-production-readiness/issues/36-usage-soft-alerts.md

@@ -0,0 +1,13 @@
+# Implement provider-mode usage attribution and soft alerts
+
+Type: task
+Status: open
+Blocked by: 19, 24, 29, 32
+
+## Question
+
+Attribute token usage, provider-reported cost, run count, and duration by
+Organization, Project, Run, model, and Provider Connection; preserve missing
+cost as unknown; expose Organization-configurable soft thresholds; and route
+Organization alerts plus platform-managed anomalous-cost alerts without hard
+spend blocking, BYOK financial alerts, or payment-settlement semantics.

.scratch/saas-production-readiness/issues/37-emergency-workload-brakes.md

@@ -0,0 +1,13 @@
+# Add audited emergency workload brakes
+
+Type: task
+Status: open
+Blocked by: 22, 23, 24, 27, 30, 32, 33
+
+## Question
+
+Implement reversible, reason-bearing, audited `DRAIN` and `STOP_NOW` controls at
+Organization and platform scope. Prove new admissions, queued work, active runs,
+locks, user notifications, authorization, telemetry, restart persistence, and
+explicit recovery follow ADR-0022 without deleting or silently suspending
+Organization data.

.scratch/saas-production-readiness/issues/38-calibrate-production-capacity.md

@@ -0,0 +1,14 @@
+# Calibrate and prove production capacity ceilings
+
+Type: research
+Status: open
+Blocked by: 30, 32, 33, 34, 35, 36, 37
+
+## Question
+
+On the accepted production-like host and provider path, measure safe numerical
+defaults and headroom for every ADR-0022 platform ceiling. Exercise normal and
+noisy-neighbor traffic, provider stalls and rate limits, queue saturation,
+large files/archives, storage pressure, process budgets, and both workload
+brakes; publish reproducible evidence and the checked-in configuration values
+required by the production release gate.

.scratch/saas-production-readiness/map.md

@@ -5,18 +5,25 @@ Label: wayfinder:map
 ## Destination
 
 The repository can be deployed from a clean checkout as the initial production
-multi-tenant Curriculum Project Hub, operated safely on the ADR-0018/0020/0021
+multi-tenant Curriculum Project Hub: platform staff manage Organizations; every
-single-host Linux topology, with repeatable release evidence, rollback and
+Organization authenticates through and operates its own customer-owned Feishu
-recovery procedures, and no known critical security or data-integrity gaps.
+app/bot, and selects either an Organization-managed BYOK provider connection or
+a distinct platform-managed provider connection; Organization owners/admins
+manage their own members, projects and settings. The ADR-0018/0020/0021
+single-host Linux topology is operated with repeatable release evidence,
+rollback and recovery, and no known critical security or data-integrity gaps.
 
 ## Notes
 
 - `spec/` remains the semantic source of truth; implementation drift must be
   surfaced rather than silently resolved.
 - The initial product boundary follows the accepted ADRs: Organization is the
-  tenant root, customer onboarding may be manual, and full billing, limits,
+  tenant root and customer onboarding may be manual. Full billing, tenant
-  tenant suspension, and self-service platform administration remain deferred
+  suspension, and self-service platform administration remain deferred;
-  unless a readiness investigation proves they are required for safe operation.
+  ADR-0022's safety limits and workload brakes are required for production.
+- Manual guided setup may provision each Organization's customer-owned Feishu
+  app during the pilot; one process-global Feishu/OAuth credential shared by
+  unrelated Organizations is not an acceptable destination state.
 - Use the `diagnosing-bugs` loop for every reproduced defect. Prefer explicit
   failures and observable critical paths over fallback behavior.
 - Work on `saas-production-readiness`; make small conventional commits after
@@ -32,6 +39,7 @@ recovery procedures, and no known critical security or data-integrity gaps.
 - [Audit tenant, authentication, and request security boundaries](issues/02-audit-tenant-auth-security.md) — retain the working org-admin/application authorization core, but production is blocked on agent/MCP isolation, execution-time object binding, org-scoped encrypted credentials, tenant status, browser sessions, and clean dependency audits.
 - [Audit run lifecycle and data integrity under failure](issues/03-audit-run-lifecycle-integrity.md) — retain the database-enforced project-lock uniqueness and normal happy path, but replace receipt-before-success, process-local queues, independent run/session/lock effects, silent delivery/audit loss, and partial startup reset with durable, recoverable state machines; explicitly decide failed-run workspace semantics.
 - [Define production observability and incident recovery](issues/04-define-observability-recovery.md) — keep the existing Fastify/Prisma/systemd signal fragments, but production requires dependency-aware readiness, typed and correlated errors, durable-work telemetry, an explicit SLO/retention/alerting control plane, and recovery runbooks proven by fault drills.
+- [Define initial abuse and capacity controls](issues/05-define-abuse-capacity-controls.md) — require layered fail-closed limits, durable Organization-fair admission, explicit backpressure, bounded files/storage/runs, soft usage alerts, and audited emergency workload brakes; calibrate numerical ceilings under production-like load.
 
 ## Fog
 

AGENTS.md

@@ -11,6 +11,12 @@
 - org 后台 project explorer 里 `Folder` 是透明组织节点,不是权限资源;project 仍是权限边界。
   普通老师可在飞书群自助建 project 但受 org policy 控制(见 ADR-0021 /
   `Spec.System.ProjectWorkspace`)。
+- 每个 org 自选 BYOK 或平台托管 model provider connection;平台托管也必须是该 org
+  独享的 key/base URL,不得让无关 org 共用 process-global provider key(见 ADR-0021 /
+  `Spec.System.Organization`)。
+- 生产容量按不可突破的 platform ceiling 与 org 可下调 policy 分层;有效限制取两者较低值。
+  Agent admission 必须持久、有界、跨 org 公平且显式背压(见 ADR-0022 /
+  `Spec.System.Capacity`)。
 
 ## 纪律
 

CONTEXT.md

@@ -0,0 +1,77 @@
+# Curriculum Project Hub
+
+The shared language for the multi-tenant service that lets customer organizations manage curriculum projects and operate agents through their own Feishu connection.
+
+## Language
+
+**Organization**:
+The customer tenant root that owns its memberships, projects, teams, external connections, and settings.
+_Avoid_: Account, workspace
+
+**Platform Administrator**:
+An internal operator who creates and operates Organizations without becoming a member of them.
+_Avoid_: Organization admin, super member
+
+**Organization Administrator**:
+An Organization OWNER or ADMIN who manages only that Organization's members, projects, connections, and settings.
+_Avoid_: Platform admin
+
+**Admin Surface**:
+The private web control plane available only to an authenticated user with the corresponding Platform Administrator or Organization Administrator authority.
+_Avoid_: Public management page, member portal
+
+**Customer-owned Feishu Application**:
+The Organization-scoped Feishu application used for that Organization's login, bot messages, and directory integration; unrelated Organizations never share it.
+_Avoid_: Global Feishu app, platform bot
+
+**Provider Credential Mode**:
+The Organization's choice between a BYOK Provider Connection and a Platform-managed Provider Connection.
+_Avoid_: Global provider setting
+
+**BYOK Provider Connection**:
+An Organization-scoped model-provider connection whose key and base URL are supplied and managed by an Organization Administrator.
+_Avoid_: Platform key
+
+**Platform-managed Provider Connection**:
+An Organization-scoped model-provider connection whose distinct key and base URL are supplied and managed by a Platform Administrator for that Organization.
+_Avoid_: Shared platform key, BYOK
+
+**Platform Capacity Ceiling**:
+A non-overridable upper bound imposed by the platform to protect shared service capacity.
+_Avoid_: Organization quota
+
+**Organization Policy Limit**:
+An optional lower bound configured by an Organization Administrator to govern that Organization's own usage.
+_Avoid_: Platform limit
+
+**Effective Limit**:
+The lower of the Platform Capacity Ceiling and the Organization Policy Limit; when no Organization Policy Limit exists, the Platform Capacity Ceiling applies.
+_Avoid_: Unlimited default
+
+**Organization-fair Scheduling**:
+The policy that allocates available agent execution capacity fairly across Organizations, while preserving first-in-first-out order within each Organization.
+_Avoid_: Global FIFO
+
+**Agent Admission Queue**:
+The durable, bounded set of accepted agent run requests waiting for execution capacity.
+_Avoid_: In-memory trigger queue, unlimited backlog
+
+**Expired Run Request**:
+An accepted agent run request that reached its maximum queue wait before execution; it is terminal and will never execute automatically.
+_Avoid_: Delayed request, automatic retry
+
+**Run Request Initiator**:
+The Organization member whose action created an agent run request.
+_Avoid_: Queue owner, Project owner
+
+**Explicit Run Retry**:
+A new agent run request deliberately created after an earlier run failed or timed out.
+_Avoid_: Automatic replay, implicit rerun
+
+**Run Budget**:
+The deterministic resource boundary for one agent run, independent of monetary or token usage alerts.
+_Avoid_: Cost budget, unlimited run
+
+**Emergency Workload Brake**:
+An audited Platform Administrator control that prevents new agent work for one Organization or the whole platform and may explicitly stop active work during an incident.
+_Avoid_: Organization deletion, service restart

docs/adr/0018-agent-execution-surface-bounded-by-workspace.md

@@ -158,9 +158,12 @@ as it upholds `AgentFileOp.Authorized`.
   *effects* are then confined, is a policy choice left to implementation. The
   invariant governs effects either way; current choice is arbitrary commands
   (sandbox confines effects).
-- **Per-user rate limiting and token budgets.** Orthogonal to the surface
+
-  boundary (this ADR is about *what* the agent may touch, not *how often* it
+## Related Decisions
-  may run). Deferred to a separate concern.
+
-- **Inbound file size / MIME limits.** The inbox lives inside the boundary,
+- **Rate limiting and token usage.** ADR-0022 requires multi-dimensional hard
-  but unbounded file size is a DoS vector distinct from the boundary
+  request limits while keeping token thresholds as attributed soft alerts in
-  invariant. Deferred.
+  the initial release.
+- **Inbound file limits.** ADR-0022 requires hard file, attachment, archive,
+  Project-storage, and Organization-storage limits with explicit failure and
+  partial-file cleanup.

docs/adr/0020-organization-is-saas-tenant-root.md

@@ -83,8 +83,10 @@ separate control plane and must not be modeled as project `MANAGE`.
 
 ## Open Questions / Deferred
 
-- Full platform admin UI, billing, limits, and tenant suspension workflows are
+- Full platform admin UI, billing, and tenant suspension workflows are
-  deferred.
+  deferred. Production safety limits and workload brakes are no longer
+  deferred; ADR-0022 defines them without treating a workload brake as tenant
+  suspension.
 - Whether direct `USER` project grants should require active
   `OrganizationMembership` is deferred; explicit direct grants remain allowed
   for now.

docs/adr/0021-org-admin-project-onboarding.md

@@ -11,8 +11,8 @@ surface is not only the Feishu bot trigger path, but also the control planes
 around it:
 
 - platform staff need a private platform admin area to create and operate orgs;
-- customer org owners/admins need an org admin area for teams, roles, model
+- customer org owners/admins need an org admin area for teams, roles, provider
-  provider configuration, projects, folders, sessions, and usage accounting;
+  mode and BYOK configuration, projects, folders, sessions, and usage accounting;
 - ordinary teachers should be able to start work from the natural Feishu group
   flow without entering the web backend.
 
@@ -46,6 +46,19 @@ OAuth login, bot messages, and directory sync. App secrets are org-scoped
 secrets and must be encrypted at rest; business code should access them through
 a secret/connection resolver, not raw plaintext columns.
 
+Each Organization chooses one model-provider credential mode:
+
+- `BYOK`: an Organization OWNER/ADMIN supplies and manages that Organization's
+  provider key and base URL in the org admin area;
+- `PLATFORM_MANAGED`: a platform admin supplies and manages one distinct key
+  and base URL for that Organization in the platform admin area. Organization
+  users cannot read the plaintext credential.
+
+Provider connections remain Organization-scoped in both modes. Unrelated
+Organizations never share a process-global provider key. Provider secrets must
+be encrypted at rest and resolved at runtime through the same connection/secret
+boundary as Feishu credentials.
+
 Project management has two creation paths:
 
 - org owner/admin creates projects in the org web backend;
@@ -77,9 +90,10 @@ aggregation:
 Project permissions remain on `PROJECT` resources. Moving a project between
 folders does not change grants.
 
-Usage accounting is org-wise and project/folder aggregatable. It is not payment
+Usage accounting is org-wise and project/folder aggregatable for both provider
-collection in the pilot because customers supply their own model provider API
+modes. It is not payment collection in the pilot: BYOK spend belongs to the
-keys and base URLs.
+customer's provider account, while platform-managed spend is attributed to the
+Organization for operational reporting; commercial billing remains deferred.
 
 ## Consequences
 
@@ -89,8 +103,8 @@ keys and base URLs.
   explorer without creating a second ACL system.
 - Service code should expose project creation and chat binding as reusable
   backend operations so Feishu cards and future web APIs call the same rules.
-- Org role/model/provider/billing panels can be added on top of the org tenant
+- Org role/model/provider-mode panels and the platform-managed connection panel
-  root without changing project authorization.
+  can be added on top of the org tenant root without changing project authorization.
 
 ## Open Questions / Deferred
 

docs/adr/0022-layered-capacity-admission.md

@@ -0,0 +1,88 @@
+# ADR 0022: Layered Capacity Admission And Workload Safety
+
+## Status
+
+Accepted.
+
+## Context
+
+The initial SaaS deployment shares one production host across Organizations.
+The current Hub only has a one-run-per-project lock, a process-local five-item
+trigger queue, a five-minute queue wait, a global max-turn setting, and
+Fastify's default request-body limit. It has no global or Organization agent
+capacity, durable backpressure, fair scheduling, storage quotas, run wall-clock
+budget, multi-dimensional rate limiting, or emergency workload control.
+
+Leaving these limits implicit lets one Organization monopolize the host, lets
+accepted work disappear on restart, and makes overload look like success. A
+single global FIFO would preserve arrival order while still permitting one
+tenant to occupy every execution slot. Unlimited queues or automatic whole-run
+replay would hide overload and can duplicate tool side effects.
+
+## Decision
+
+Every mandatory capacity dimension has a non-optional **Platform Capacity
+Ceiling**. An Organization OWNER/ADMIN may configure a lower **Organization
+Policy Limit**; the **Effective Limit** is the lower value, or the platform
+ceiling when the Organization value is absent. There is no unlimited fallback.
+Production refuses to start when a mandatory platform ceiling is absent or
+invalid. Platform ceilings are changed through versioned deployment
+configuration in the initial release; live platform-admin editing is deferred
+until audited approval and rollback exist.
+
+Agent work uses a durable, bounded admission queue. Scheduling is fair across
+Organizations and FIFO within an Organization, while the existing one-active-
+run-per-Project rule remains. Capacity saturation queues only within effective
+length and wait limits. Queue overflow is an explicit `capacity_exhausted`
+response with retry guidance; an accepted request that waits too long becomes
+`EXPIRED`, is never started later, and notifies its initiator. Initiators may
+withdraw their own queued requests; Project `MANAGE` and Organization admins
+may cancel queued requests in their scope. Every transition is auditable.
+
+The platform never automatically replays a whole Agent Run after execution has
+started. Limited transport retries are allowed only before model output or tool
+side effects. Otherwise the run ends explicitly and a retry is a new user-
+initiated request. Feishu event identity and API idempotency prevent delivery
+retries from creating duplicate admissions.
+
+The first production gate requires hard limits for request rate/body size,
+agent concurrency, queue length/wait, files and attachment counts, archive
+expansion, Project and Organization storage, Organization entity counts, run
+wall time/turns/tool calls/output, individual tool duration, and execution
+process memory/CPU/process count. Reaching an entity or storage ceiling blocks
+new growth but does not delete, hide, or lock existing data; read, export,
+delete, and administrator remediation remain available.
+
+Admin surfaces require an authenticated user plus the corresponding platform
+or Organization administrator authorization. Authenticated Web traffic is
+limited by user, Organization, and endpoint class; signed Feishu traffic is
+limited by Organization application, chat, and sender; administrative writes
+use separate stricter buckets. Rejections and truncation are never silent.
+
+Token, provider-reported cost, run count, and duration are attributed by
+Organization, Project, Run, model, and Provider Connection. They are statistics
+and soft alerts in the initial release, not hard spend controls or settlement.
+Missing provider cost remains unknown rather than zero. Organization admins
+receive their alerts; platform admins additionally receive anomalous-cost
+alerts for platform-managed Provider Connections, not BYOK financial alerts.
+
+Platform administrators have audited, reason-bearing, reversible workload
+brakes at Organization and platform scope. `DRAIN` rejects new work and pauses
+queue starts while allowing active runs to finish. `STOP_NOW` also cancels
+active runs. These controls do not delete or suspend the Organization.
+
+Exact numerical ceilings remain open until production-like capacity tests
+measure the supported host and provider behavior. Defaults must be checked in,
+validated, observable, and proven at the release gate rather than guessed as
+domain constants.
+
+## Consequences
+
+- The current in-memory `TriggerQueue` and global-only agent settings are
+  implementation divergences, not acceptable production fallbacks.
+- Durable ingress, run/lock recovery, authorization, telemetry, and capacity
+  enforcement must share one Organization-scoped admission identity.
+- HTTP overload uses typed `429` responses with `Retry-After`; Feishu reports a
+  clear busy/expired/canceled outcome to the initiating user.
+- Capacity calibration is a release prerequisite and must include noisy-neighbor,
+  restart, queue expiry, file/archive, provider-stall, and workload-brake tests.

hub/src/org/usage.ts

@@ -1,6 +1,8 @@
 /**
  * Org / project usage rollups from AgentRun cost facts (ADR-0021).
- * Not payment collection — customers may supply their own provider keys.
+ * Operational usage accounting, not payment collection. Organizations may use
+ * either their own provider credentials or a distinct platform-managed
+ * connection; commercial billing remains outside the pilot scope.
  */
 import type { Prisma, PrismaClient } from "@prisma/client";
 

spec/Spec/System.lean

@@ -1,6 +1,7 @@
 import Spec.System.ProjectGroup
 import Spec.System.Organization
 import Spec.System.ProjectWorkspace
+import Spec.System.Capacity
 import Spec.System.Run
 import Spec.System.Lock
 import Spec.System.Memory
@@ -19,6 +20,8 @@ import Spec.System.Audit
 - `Organization` —— SaaS tenant root(ADR-0020);project/team 单归属,TEAM grant 不跨 org。
 - `ProjectWorkspace` —— org 后台 project explorer:folder 是透明组织节点,project 仍是权限边界
   (ADR-0021)。
+- `Capacity` —— platform ceiling 与 org policy 的分层限制、持久 admission request 状态和
+  平台紧急工作负载制动(ADR-0022)。
 - `Run` —— AgentRun 状态与终止判定(状态集合完整性 OPEN)。
 - `Lock` —— 锁 owner=run(ADR-0002),及"持锁者必为非终止 run"的核心不变式。
 - `Memory` —— 按需上下文:锚点类别(ADR-0003)+ MCP 工具按 run/project 上下文授权的不变式。
@@ -29,5 +32,5 @@ import Spec.System.Audit
   (ADR-0004);role-capability 与 settings-policy 的组合规则 OPEN。
 - `Audit` —— 有意从简(内容多为 plumbing,OPEN)。
 
-标识符见 `Spec.Prelude`。决策出处:ADR-0001..0004, 0018, 0020, 0021。
+标识符见 `Spec.Prelude`。决策出处:ADR-0001..0004, 0018, 0020..0022。
 -/

spec/Spec/System/Capacity.lean

@@ -0,0 +1,89 @@
+import Spec.Prelude
+
+/-!
+# Capacity —— SaaS capacity admission and abuse controls (ADR-0022)
+
+初始生产服务共享有限的单机资源,但不能让一个 Organization 垄断容量或让无界输入拖垮
+其他租户。ADR-0022 钉死分层限制、持久 admission、显式背压和紧急制动的领域语义;
+具体数值必须由生产式容量测试校准,因此保持 `OPEN`,不得把未经验证的数字冒充契约。
+-/
+
+namespace Spec.System
+
+/-- 首个生产版本必须有硬边界的容量维度(`PINNED`, ADR-0022)。金额与 token 用量是
+统计和软告警,不在本硬限制集合中;各维度的具体数值为 `OPEN`,须由容量测试决定。 -/
+inductive CapacityDimension where
+  | requestRate
+  | requestBodySize
+  | agentConcurrency
+  | admissionQueueLength
+  | admissionQueueWait
+  | fileSize
+  | attachmentCount
+  | archiveExpansion
+  | projectStorage
+  | organizationStorage
+  | memberCount
+  | projectCount
+  | teamCount
+  | folderCount
+  | sessionCount
+  | runWallTime
+  | runTurns
+  | runToolCalls
+  | toolWallTime
+  | runOutputSize
+  | processMemory
+  | processCpu
+  | processCount
+
+/-- 一个容量维度的分层限制(`PINNED`, ADR-0022):platform ceiling 永远存在且不可被
+Organization 突破;Organization policy 可以缺省,也只能配置更低的限制。 -/
+structure LayeredLimit where
+  /-- 由版本化生产部署配置提供的不可突破上限(`PINNED`, ADR-0022)。 -/
+  platformCeiling : Nat
+  /-- Organization 管理员可选的更低策略限制(`PINNED`, ADR-0022)。 -/
+  organizationLimit : Option Nat
+
+/-- 分层限制是良构的 iff Organization policy 未设置或不高于 platform ceiling
+(`PINNED`, ADR-0022)。不允许用 `none` 表示 platform unlimited。 -/
+def LayeredLimit.Valid (limit : LayeredLimit) : Prop :=
+  match limit.organizationLimit with
+  | none => True
+  | some organizationLimit => organizationLimit ≤ limit.platformCeiling
+
+/-- 有效限制(`PINNED`, ADR-0022)取 platform ceiling 与 Organization policy 中较低者;
+Organization policy 缺省时直接采用 platform ceiling,永不退化为 unlimited。 -/
+def LayeredLimit.effective (limit : LayeredLimit) : Nat :=
+  match limit.organizationLimit with
+  | none => limit.platformCeiling
+  | some organizationLimit => min limit.platformCeiling organizationLimit
+
+/-- 已被服务接受的 agent run request 状态(`PINNED`, ADR-0022)。`expired` 和
+`canceled` 都是可审计终态且不得自动执行;`started` 表示已从 admission queue 移交给
+一个 AgentRun。被容量规则拒绝的输入从未被接受,因此不伪装成 queued request。 -/
+inductive RunRequestState where
+  | queued
+  | started
+  | expired
+  | canceled
+
+/-- 平台紧急工作负载制动模式(`PINNED`, ADR-0022)。`drain` 停止新 admission 与队列
+启动但允许当前 run 完成;`stopNow` 还会取消当前 run。它们不等同于删除或封禁 org。 -/
+inductive WorkloadBrakeMode where
+  | open
+  | drain
+  | stopNow
+
+/-- 紧急制动是否允许接收新的 agent 工作(`PINNED`, ADR-0022):只有 `open` 允许。 -/
+def WorkloadBrakeMode.AcceptsNew : WorkloadBrakeMode → Prop
+  | .open => True
+  | .drain | .stopNow => False
+
+/-- 紧急制动是否允许当前 agent run 继续(`PINNED`, ADR-0022):`drain` 允许收尾,
+`stopNow` 要求取消。 -/
+def WorkloadBrakeMode.AllowsActive : WorkloadBrakeMode → Prop
+  | .open | .drain => True
+  | .stopNow => False
+
+end Spec.System

spec/Spec/System/Organization.lean

@@ -9,7 +9,8 @@ ADR-0020:Hub 长期按 SaaS 形态演进,`Organization` 是客户侧 tenant root
 不复用 project 的 `read/edit/manage` 角色格,而是另一个控制面。
 
 本模块只钉死会造成实现分歧的不变量:tenant root 存在、project/team 单归属、team grant
-不得跨 org。用户身份、外部目录 provider 细节、平台控制面角色集合仍为 `OPEN`。
+不得跨 org,以及 model provider connection 的凭据归属模式。用户身份、外部目录
+provider 细节、平台控制面角色集合仍为 `OPEN`。
 -/
 
 namespace Spec.System
@@ -48,4 +49,20 @@ def TeamProjectGrantScope.WellScoped
     (teamOrg : I.TeamId → Option I.OrganizationId) : Prop :=
   ∃ o, projectOrg grant.project = some o ∧ teamOrg grant.team = some o
 
+/-- Organization 的 model provider 凭据归属模式(`PINNED`, ADR-0021):BYOK 由 org
+管理员提供和管理;platform-managed 由平台管理员为该 org 单独提供和管理。两种模式
+都不允许无关 org 共用 process-global provider key。模式切换过程仍为 `OPEN`。 -/
+inductive ProviderCredentialMode where
+  | byok
+  | platformManaged
+
+/-- Organization 的 model provider connection(`PINNED`, ADR-0021):connection 必须
+归属一个且仅一个 organization,并明确采用哪一种凭据模式。key/base URL 的加密表示、
+轮换与 resolver 机制由后续 secret-control-plane 决策规定。 -/
+structure OrganizationProviderConnection where
+  /-- connection 所属 organization(`PINNED`, ADR-0021)。 -/
+  organization : I.OrganizationId
+  /-- connection 的凭据归属模式(`PINNED`, ADR-0021)。 -/
+  mode : ProviderCredentialMode
+
 end Spec.System

spec/Spec/System/Run.lean

@@ -8,21 +8,23 @@
 
 namespace Spec.System
 
-/-- AgentRun 运行状态(状态名 `PINNED`, ADR-0001..0003 + likec4;**完整性 `OPEN`**
+/-- AgentRun 运行状态(状态名 `PINNED`, ADR-0001..0003, ADR-0022 + likec4;
-——散文从未声明"状态恰好这些";实现若需新状态(如 pending)须 surface,不得默认
+**完整性 `OPEN`**——散文从未声明"状态恰好这些";实现若需新状态(如 pending)须
-本枚举已穷尽)。终止态见 `RunState.Terminal`。 -/
+surface,不得默认本枚举已穷尽)。终止态见 `RunState.Terminal`。 -/
 inductive RunState where
   | active
   | waitingForUser
   | completed
   | failed
   | timedOut
+  | limitExceeded
   | canceled
 
-/-- run 处于**终止态**(`PINNED`, ADR-0002:锁在 completes/fails/timesOut/canceled
+/-- run 处于**终止态**(`PINNED`, ADR-0002, ADR-0022:锁在 completes/fails/
-时释放)。`active`/`waitingForUser` 非终止——后者仍占用项目(锁未释放)。 -/
+timesOut/limitExceeded/canceled 时释放)。`active`/`waitingForUser` 非终止——后者仍
+占用项目(锁未释放)。 -/
 def RunState.Terminal : RunState → Prop
-  | .completed | .failed | .timedOut | .canceled => True
+  | .completed | .failed | .timedOut | .limitExceeded | .canceled => True
   | .active | .waitingForUser => False
 
 end Spec.System