curriculum-project-hub/docs/adr/0027-external-capability-registry.md

ADR 0027: External Capability Registry

Status

Accepted.

Context

ADR-0026 introduced the UsageFact ledger with a kind = external_capability fact and a capabilityId field, but deferred the capability registry itself. Two concrete needs now force the issue:

• PDF→Markdown bundle conversion (and, imminently, audio/video→text) must run as a side effect of an AgentRun, bill in non-token units (pages, seconds), use a different provider than the model loop, and report cost through a different channel. It is not a sub-run (ADR-0026 rejected that) and not a model-provider call (it does not speak the Anthropic/ OpenRouter protocol).
• The Agent already has a Bash tool. Without a first-class capability seam, the path of least resistance is for the agent to shell out to ad-hoc scripts that embed API keys, write to arbitrary paths, and report nothing to the ledger. That is exactly the unattributed, uncontained external consumption ADR-0022/0026 exist to prevent.

The model-provider connection (OrganizationProviderConnection, ADR-0024) is the wrong seam for these services:

• Its payload schema (baseUrl + authToken + anthropicApiKey) and readiness probe (/v1/models?supported_parameters=tools) are specific to OpenRouter/Anthropic. MinerU, Whisper, and future OCR/ASR services have different auth shapes (an API token, optionally a project id) and no /v1/models endpoint.
• Its uniqueness key is (organizationId, providerId) where providerId is an OpenRouter-style model-routing id. A capability provider id (mineru) names a service, not a model.
• Coupling capability credentials into the model-provider table would force every capability's auth shape through ProviderSecretPayloadV1 and every readiness probe through probeOpenRouterCredential.

The Feishu Application Connection (OrganizationFeishuApplicationConnection) is the right structural precedent: it reuses the ADR-0024 envelope (KEK → DEK → AES-256-GCM, AAD-bound to purpose/org/connection/version) but has its own connection table, its own payload schema, its own readiness probe, and its own per-org uniqueness. Capability connections follow the same pattern.

Decision

External Capability

An External Capability is a platform-registered, org-enabled document or media transform service invoked as a side effect of an AgentRun. It is identified by a stable capabilityId (e.g. pdf_to_md_bundle, audio_video_to_text). A capability:

• Has an input kind (PDF, image, audio, video, …) and an output contract (markdown bundle with extracted images, transcript text, …).
• Bills in non-token units (pages, audio-seconds) recorded on a UsageFact with kind = external_capability, or in tokens when the backing service reports them.
• Writes its output into the invoking run's workspace (ADR-0018 AgentSurface — no escapes).
• Is invoked through a capability adapter in Hub, never by the Agent shelling out with embedded credentials.

Capability Connection

Credentials for a capability live in an OrganizationCapabilityConnection, structurally identical to the Feishu Application Connection:

• Belongs to exactly one Organization.
• Unique by (organizationId, capabilityId).
• DRAFT / ACTIVE / DISABLED; resolution accepts only ACTIVE with a valid active secret version.
• Secret material is an immutable, AAD-bound, KEK-wrapped envelope version (CapabilityCredentialVersion), reusing the ADR-0024 encryption machinery with purpose = "capability".
• Its payload schema is capability-specific (CapabilitySecretPayloadV1: baseUrl, apiToken, optional projectId). New capability types extend the payload, not the connection table.
• A capability-specific readiness probe validates the credential before activation (e.g. MinerU: a trivial authenticated GET). The probe is injectable, matching the Feishu/provider pattern, so tests never hit the network.

Capability Adapter

The adapter is the seam between the Agent and the external service. It:

• Resolves the org's active capability connection (fail-closed, no process-global fallback — ADR-0024).
• Accepts a workspace-relative input path and an output directory.
• Calls the backing service (MinerU, Whisper, …) via an injectable Client interface so the real HTTP client is swappable and mockable.
• Writes the produced markdown + image assets into the run's workspace.
• Writes one UsageFact (or more, if the service reports per-stage consumption) with kind = external_capability, capabilityId, provider (the service id), quantity + unit (pages / seconds), and costUsd + costSource = provider_reported when the service reports cost.

Registry

The platform maintains a registry of known capabilities: their id, input kind, output contract, metering unit, and adapter. This is code-level registration (like ToolRegistry), not a database table — a capability is available to an Organization only when (a) the platform knows the adapter and (b) the Organization has an ACTIVE connection for it. Both gates are required.

What is NOT in this ADR

• The capability invocation is not a first-class persisted record (CapabilityInvocation table) in this ADR. The UsageFact row with capabilityId + correlationId is the durable trace. If we later need a richer invocation log (retries, partial output, multi-stage status), that is a follow-up; for now the fact is enough.
• Pricebook remains deferred (ADR-0026). Capability facts use provider_reported when the service returns cost; otherwise unknown.
• Org-scoped enable/disable policy beyond connection status is deferred. An org with an ACTIVE connection has the capability; one without does not. A finer "enabled but no credential" toggle is not needed yet.
• Agent-facing tool exposure (how the Agent discovers and calls the capability — MCP tool, Bash wrapper, or built-in) is an implementation detail of the adapter wiring, not a contract concern. The contract pins that the Agent never receives the capability credential.

Consequences

• Adding a new external capability (e.g. image_ocr) is: register an adapter, add a capabilityId constant, optionally extend the secret payload — no schema change to UsageFact or AgentRun.
• The model-provider connection table stays focused on model routing; capability credentials do not pollute its payload or readiness probe.
• Three connection types now share the ADR-0024 envelope: model-provider, Feishu application, and capability. Each has its own table, payload schema, and probe, but the same encryption, rotation, and resolver boundary.
• The Agent's Bash tool remains available, but the intended path for document/media transforms is the capability adapter. Whether to narrow Bash for capability-shaped tasks is an operational policy decision, not a contract one.
• Tests prove: workspace containment of capability output, fail-closed credential resolution, UsageFact attribution with non-token metering, and that the Agent process never receives the capability credential.

Deferred

• CapabilityInvocation as a first-class durable record (status, retries, partial output) — currently the UsageFact row is the only trace.
• Pricebook derivation for capability costs (ADR-0026 deferred).
• Org-scoped capability enable/disable policy finer than connection status.
• Agent-facing tool discovery (MCP vs built-in) for capabilities.