feat(hub): usage fact ledger for run-scoped cost attribution (ADR-0026)

Replace the single-scalar cost model on AgentRun with an append-only
UsageFact ledger. One AgentRun owns zero or more UsageFact rows; each
records one billable consumption event (model completion, external
capability, or tool proxy) with its own provider/model/tokens/quantity/
cost. AgentRun.costUsd/inputTokens/outputTokens become a derived rollup
cache.

This unblocks external capabilities (PDF->MD bundle, audio/video->text)
that bill in non-token units (pages, seconds) through a different
provider than the main agent loop, without per-capability schema changes
or nested AgentRuns (which would pollute lock/admission/session
semantics).

Contract:
- spec/Spec/System/Agent/Usage.lean pins UsageFact, UsageFactKind,
  CostSource and three invariants: append-only; belongs to one run,
  never holds a lock; missing cost != zero (ADR-0022).
- ADR-0026 records the decision, the rejected nested-Run alternative,
  the rollup cache strategy, and the deferred capability registry /
  pricebook / post-hoc correction flows.

Schema:
- UsageFact model with indexes on (runId, occurredAt), (runId, kind),
  (provider, model, occurredAt), (capabilityId, occurredAt).
- Migration backfills one synthetic model_completion fact per existing
  run with recorded cost/tokens (correlationId = runId marks backfill);
  truly unrecorded runs stay runsWithoutCost per ADR-0022.

Write path (trigger finish):
- Write the UsageFact first, then mirror it onto AgentRun as two
  separate statements (not one transaction). The fact is the truth so it
  goes first; the cache is derived so it goes second. A crash between
  them leaves the cache stale but the usage service re-reads facts
  directly, so this is recoverable; the reverse order would lose the
  truth. Separate statements also avoid an AgentRun row lock held across
  the insert's FK ShareLock, which deadlocked concurrent workspace
  teardown under the Organization->Project->AgentRun->UsageFact cascade.

Read paths:
- org/usage.ts aggregates from UsageFact, ignoring the AgentRun cache.
- slash /usage buckets by (fact.provider, fact.model); a run with a
  main loop + an external call lands in two buckets.
- session detail exposes usageFacts[] + costSource for future per-run
  cost-breakdown UI.

Tests:
- usage.test.ts: 6 integration tests pin fact aggregation, missing-cost-
  !=-zero, multi-fact-per-run, empty-run, project-level, empty-org.
- trigger.test.ts: existing /usage assertion ($0.0023,
  openrouter / mock-model) passes on the fact path.
- feishu-reactions mock prisma gains usageFact.create.
This commit is contained in:
2026-07-18 14:44:23 +08:00
parent 97f7972cc5
commit f3b087371a
11 changed files with 760 additions and 46 deletions
+146
View File
@@ -0,0 +1,146 @@
# ADR 0026: Usage Fact Ledger
## Status
Accepted.
## Context
ADR-0022 pins the cost-attribution contract for the platform: token,
provider-reported cost, run count, and duration are attributed by Organization,
Project, Run, model, and Provider Connection; "missing provider cost remains
unknown rather than zero." ADR-0021 scopes usage accounting as operational
reporting, not payment collection (commercial billing stays deferred).
The implementation today stores this as **one scalar per `AgentRun`**:
`inputTokens`, `outputTokens`, `costUsd?`, `costSource?`, written once from the
Claude Agent SDK's `result.total_cost_usd` when the run finishes. This is
adequate for a single provider-reported model loop, but it cannot represent:
- **External capability consumption** inside a run. PDF→Markdown bundle
conversion, audio/video transcription, OCR and similar media transforms are
not the main agent loop. They run as side effects of a run, may use a
different provider/model, may bill in non-token units (pages, seconds,
invocations), and may report cost through a different channel than the
OpenRouter gateway. Today there is no row to write that cost to — it would
either disappear or silently corrupt the run's single scalar.
- **Multiple model calls within one run** (e.g. a sub-model invoked by a
tool, a gateway-side reroute). The scalar collapses them into one number.
- **Pricebook derivation** after the fact. With only a final USD figure and no
`(provider, model, occurredAt, tokens)` fact, an operator cannot re-derive
cost from a price table when the provider did not report it.
Treating each external call as a nested `AgentRun` was considered and
rejected: `AgentRun` carries lock ownership (ADR-0002), session/provider/role
binding (ADR-0017), admission/capacity semantics (ADR-0022), and the
user-visible task boundary. External calls hold none of those. Making them
`AgentRun`s would pollute run counts, admission, lock semantics, and session
continuity, and would still not solve non-token metering.
## Decision
Introduce **`UsageFact`** as the single source of truth for billable
consumption inside an `AgentRun`. An `AgentRun` owns zero or more
append-only `UsageFact` rows; each row records one billable consumption event:
- `kind``model_completion` (the main agent loop) | `external_capability`
| `tool_proxy`. The kind set is `OPEN`; new kinds must be surfaced, not
silently folded into an existing one.
- `provider` — e.g. `openrouter`, `mineru`, `openai_whisper`.
- `model?`, `inputTokens?`, `outputTokens?` — token metering, optional
because non-token capabilities have none.
- `quantity?` + `unit?` — non-token metering (pages, audio_seconds,
invocations), coexisting with tokens rather than replacing them.
- `costUsd?` + `costSource``provider_reported` | `pricebook_derived` |
`unknown`. `costUsd = null` means **unknown, not zero** (ADR-0022). When the
provider reported a cost, `costSource = provider_reported` and that value
wins. When only tokens are known, a later pricebook pass may derive
`costUsd` with `costSource = pricebook_derived`. When neither is possible,
`costSource = unknown` and `costUsd` stays null.
- `occurredAt` — when the consumption happened; the pricebook derivation
depends on this, not on `AgentRun.finishedAt`, because an external
capability may complete before the run finishes.
- `capabilityId?` — for `external_capability` facts, the registered capability
id (e.g. `pdf_to_md_bundle`, `audio_video_to_text`).
- `correlationId?` — external request id for reconciliation / idempotency;
not part of the aggregation key.
### Invariants
1. **Append-only.** A `UsageFact` row is never updated or deleted. A cost
correction is a new row; the old row stays. `onDelete: Cascade` exists only
so a hard run delete (itself not a normal path) cleans up its facts.
2. **Belongs to exactly one Run; never holds a lock.** A fact is a side-effect
ledger of one run, not a sub-run. External capability calls obey the same
boundary: their identity is `capabilityId + correlationId`, not `RunId`.
3. **Missing cost ≠ zero.** Aggregation MUST NOT sum `null` `costUsd` as 0.
A run whose facts all have `costUsd = null` is "cost unknown" — reported as
`runsWithoutCost`, exactly as the pre-migration `costUsd = null` runs are
today. A run with at least one `costUsd`-bearing fact contributes its sum.
### Rollup cache
`AgentRun.costUsd / inputTokens / outputTokens / costSource` columns are kept
as a **derived rollup cache**, not dropped:
- Existing non-`/usage` readers (slash `/usage`, session detail, integration
tests asserting `run.costUsd`) continue to work without code changes for
historical runs.
- On run finish, the writer writes the `UsageFact` row first and then mirrors
it onto `AgentRun` as two separate statements, not one transaction. The
fact is the truth, so it is written first; the cache is derived, so it is
written second. A crash between the two leaves the cache stale, but the
usage service re-reads `UsageFact` directly, so staleness is recoverable
(and the reverse order would lose the truth, which is not). Keeping the
fact insert and the run update in separate statements also avoids holding
the `AgentRun` row lock across the FK ShareLock taken by the insert, which
deadlocks against concurrent workspace teardown under the cascade path
`Organization → Project → AgentRun → UsageFact`.
- The org/project usage service and the slash `/usage` command read from
`UsageFact` directly — they are the canonical aggregation path.
### Migration
A new `UsageFact` table is added. For every existing `AgentRun` with a
non-null `costUsd` or non-null `inputTokens`/`outputTokens`, the migration
inserts **one synthetic `model_completion` fact** carrying the run's
provider, model, tokens, cost, and `costSource`. Its `correlationId` is the
run id, so the row is identifiable as a backfill artefact. This keeps
historical reporting correct under the new aggregation path. Pre-migration
runs with no recorded cost remain `runsWithoutCost` by design, matching
ADR-0022's "missing cost ≠ zero" rule and the existing migration
`20260709143000_agent_run_cost_tracking`'s "no backfill" stance for the
truly unrecorded.
## Consequences
- The billing model now supports external capabilities (PDF→MD, ASR, OCR, …)
without per-capability schema changes: a new capability is one new
`capabilityId` value and one or more `external_capability` facts.
- `AgentRun.costUsd` is no longer the truth; it is a convenience cache.
Readers that need the truth (cost breakdown, per-capability attribution)
must read `UsageFact`. The cache MUST be kept consistent on the write path.
- The capability registry, pricebook, and any commercial settlement remain
`OPEN` and out of pilot scope (ADR-0021). This ADR only pins the ledger
shape and invariants.
- `usage.ts` and `/usage` now perform a join against `UsageFact` rather than a
single-table scan of `AgentRun`; the index on `(runId, occurredAt)` and
`(provider, model, occurredAt)` keeps the existing org/project/report
queries bounded.
- Cost corrections (e.g. a provider rebills a run) produce a new fact row; the
rollup cache must be recomputed. The initial release writes facts once at
run finish and does not support post-hoc correction flows — that remains
`OPEN`.
## Deferred
- **Capability registry** as a first-class spec entity (`Spec.System.Capability`)
with org-scoped enable/disable, input/output contracts, metering schemas,
and org-exclusive credentials (ADR-0024 alignment). This ADR only reserves
the `capabilityId` field and the `external_capability` fact kind.
- **Pricebook** — a versioned price table keyed by `(provider, model, unit)`
with time validity. Required to actually produce `pricebook_derived` costs;
until then, facts without provider-reported cost stay `unknown`.
- **Post-hoc cost correction flow** — append-only today; correction UI and
rollup recompute are `OPEN`.
- **Commercial billing, invoicing, settlement** — still deferred per ADR-0021.