curriculum-project-hub/docs/adr/0027-external-capability-registry.md

# ADR 0027: External Capability Registry

## Status

Accepted.

## Context

ADR-0026 introduced the `UsageFact` ledger with a `kind = external_capability`
fact and a `capabilityId` field, but deferred the capability registry itself.
Two concrete needs now force the issue:

- **PDF→Markdown bundle** conversion (and, imminently, audio/video→text)
  must run as a side effect of an `AgentRun`, bill in non-token units
  (pages, seconds), use a different provider than the model loop, and report
  cost through a different channel. It is not a sub-run (ADR-0026 rejected
  that) and not a model-provider call (it does not speak the Anthropic/
  OpenRouter protocol).
- The Agent already has a Bash tool. Without a first-class capability seam,
  the path of least resistance is for the agent to shell out to ad-hoc
  scripts that embed API keys, write to arbitrary paths, and report nothing
  to the ledger. That is exactly the unattributed, uncontained external
  consumption ADR-0022/0026 exist to prevent.

The model-provider connection (`OrganizationProviderConnection`, ADR-0024)
is the wrong seam for these services:

- Its payload schema (`baseUrl` + `authToken` + `anthropicApiKey`) and
  readiness probe (`/v1/models?supported_parameters=tools`) are specific to
  OpenRouter/Anthropic. MinerU, Whisper, and future OCR/ASR services have
  different auth shapes (an API token, optionally a project id) and no
  `/v1/models` endpoint.
- Its uniqueness key is `(organizationId, providerId)` where `providerId`
  is an OpenRouter-style model-routing id. A capability provider id
  (`mineru`) names a *service*, not a model.
- Coupling capability credentials into the model-provider table would force
  every capability's auth shape through `ProviderSecretPayloadV1` and every
  readiness probe through `probeOpenRouterCredential`.

The Feishu Application Connection (`OrganizationFeishuApplicationConnection`)
is the right structural precedent: it reuses the ADR-0024 envelope (KEK →
DEK → AES-256-GCM, AAD-bound to purpose/org/connection/version) but has its
own connection table, its own payload schema, its own readiness probe, and
its own per-org uniqueness. Capability connections follow the same pattern.

## Decision

### External Capability

An **External Capability** is a platform-registered, org-enabled document or
media transform service invoked as a side effect of an `AgentRun`. It is
identified by a stable `capabilityId` (e.g. `pdf_to_md_bundle`,
`audio_video_to_text`). A capability:

- Has an **input kind** (PDF, image, audio, video, …) and an **output
  contract** (markdown bundle with extracted images, transcript text, …).
- Bills in **non-token units** (pages, audio-seconds) recorded on a
  `UsageFact` with `kind = external_capability`, or in tokens when the
  backing service reports them.
- Writes its output **into the invoking run's workspace** (ADR-0018
  `AgentSurface` — no escapes).
- Is invoked through a **capability adapter** in Hub, never by the Agent
  shelling out with embedded credentials.

### Capability Connection

Credentials for a capability live in an **`OrganizationCapabilityConnection`**,
structurally identical to the Feishu Application Connection:

- Belongs to exactly one Organization.
- Unique by `(organizationId, capabilityId)`.
- `DRAFT` / `ACTIVE` / `DISABLED`; resolution accepts only `ACTIVE` with a
  valid active secret version.
- Secret material is an immutable, AAD-bound, KEK-wrapped envelope version
  (`CapabilityCredentialVersion`), reusing the ADR-0024 encryption
  machinery with `purpose = "capability"`.
- Its payload schema is capability-specific (`CapabilitySecretPayloadV1`:
  `baseUrl`, `apiToken`, optional `projectId`). New capability types extend
  the payload, not the connection table.
- A capability-specific **readiness probe** validates the credential before
  activation (e.g. MinerU: a trivial authenticated GET). The probe is
  injectable, matching the Feishu/provider pattern, so tests never hit the
  network.

### Capability Adapter

The adapter is the seam between the Agent and the external service. It:

- Resolves the org's active capability connection (fail-closed, no
  process-global fallback — ADR-0024).
- Accepts a workspace-relative input path and an output directory.
- Calls the backing service (MinerU, Whisper, …) via an injectable
  `Client` interface so the real HTTP client is swappable and mockable.
- Writes the produced markdown + image assets into the run's workspace.
- Writes one `UsageFact` (or more, if the service reports per-stage
  consumption) with `kind = external_capability`, `capabilityId`,
  `provider` (the service id), `quantity + unit` (pages / seconds), and
  `costUsd + costSource = provider_reported` when the service reports cost.

### Registry

The platform maintains a **registry** of known capabilities: their id,
input kind, output contract, metering unit, and adapter. This is
code-level registration (like `ToolRegistry`), not a database table — a
capability is available to an Organization only when (a) the platform
knows the adapter and (b) the Organization has an `ACTIVE` connection for
it. Both gates are required.

### What is NOT in this ADR

- The capability invocation is **not** a first-class persisted record
  (`CapabilityInvocation` table) in this ADR. The `UsageFact` row with
  `capabilityId` + `correlationId` is the durable trace. If we later need
  a richer invocation log (retries, partial output, multi-stage status),
  that is a follow-up; for now the fact is enough.
- **Pricebook** remains deferred (ADR-0026). Capability facts use
  `provider_reported` when the service returns cost; otherwise `unknown`.
- **Org-scoped enable/disable policy** beyond connection status is
  deferred. An org with an `ACTIVE` connection has the capability; one
  without does not. A finer "enabled but no credential" toggle is not
  needed yet.
- **Agent-facing tool exposure** (how the Agent discovers and calls the
  capability — MCP tool, Bash wrapper, or built-in) is an implementation
  detail of the adapter wiring, not a contract concern. The contract pins
  that the Agent never receives the capability credential.

## Consequences

- Adding a new external capability (e.g. `image_ocr`) is: register an
  adapter, add a `capabilityId` constant, optionally extend the secret
  payload — no schema change to `UsageFact` or `AgentRun`.
- The model-provider connection table stays focused on model routing;
  capability credentials do not pollute its payload or readiness probe.
- Three connection types now share the ADR-0024 envelope: model-provider,
  Feishu application, and capability. Each has its own table, payload
  schema, and probe, but the same encryption, rotation, and resolver
  boundary.
- The Agent's Bash tool remains available, but the intended path for
  document/media transforms is the capability adapter. Whether to narrow
  Bash for capability-shaped tasks is an operational policy decision,
  not a contract one.
- Tests prove: workspace containment of capability output, fail-closed
  credential resolution, `UsageFact` attribution with non-token metering,
  and that the Agent process never receives the capability credential.

## Deferred

- `CapabilityInvocation` as a first-class durable record (status, retries,
  partial output) — currently the `UsageFact` row is the only trace.
- Pricebook derivation for capability costs (ADR-0026 deferred).
- Org-scoped capability enable/disable policy finer than connection status.
- Agent-facing tool discovery (MCP vs built-in) for capabilities.