feat(hub): usage fact ledger for run-scoped cost attribution (ADR-0026) (#4)

Co-authored-by: Hong Jiarong <me@jrhim.com>
Co-committed-by: Hong Jiarong <me@jrhim.com>
This commit is contained in:
2026-07-18 14:46:41 +08:00
committed by 洪佳荣
parent 97f7972cc5
commit aaa098bb8b
11 changed files with 760 additions and 46 deletions
+146
View File
@@ -0,0 +1,146 @@
# ADR 0026: Usage Fact Ledger
## Status
Accepted.
## Context
ADR-0022 pins the cost-attribution contract for the platform: token,
provider-reported cost, run count, and duration are attributed by Organization,
Project, Run, model, and Provider Connection; "missing provider cost remains
unknown rather than zero." ADR-0021 scopes usage accounting as operational
reporting, not payment collection (commercial billing stays deferred).
The implementation today stores this as **one scalar per `AgentRun`**:
`inputTokens`, `outputTokens`, `costUsd?`, `costSource?`, written once from the
Claude Agent SDK's `result.total_cost_usd` when the run finishes. This is
adequate for a single provider-reported model loop, but it cannot represent:
- **External capability consumption** inside a run. PDF→Markdown bundle
conversion, audio/video transcription, OCR and similar media transforms are
not the main agent loop. They run as side effects of a run, may use a
different provider/model, may bill in non-token units (pages, seconds,
invocations), and may report cost through a different channel than the
OpenRouter gateway. Today there is no row to write that cost to — it would
either disappear or silently corrupt the run's single scalar.
- **Multiple model calls within one run** (e.g. a sub-model invoked by a
tool, a gateway-side reroute). The scalar collapses them into one number.
- **Pricebook derivation** after the fact. With only a final USD figure and no
`(provider, model, occurredAt, tokens)` fact, an operator cannot re-derive
cost from a price table when the provider did not report it.
Treating each external call as a nested `AgentRun` was considered and
rejected: `AgentRun` carries lock ownership (ADR-0002), session/provider/role
binding (ADR-0017), admission/capacity semantics (ADR-0022), and the
user-visible task boundary. External calls hold none of those. Making them
`AgentRun`s would pollute run counts, admission, lock semantics, and session
continuity, and would still not solve non-token metering.
## Decision
Introduce **`UsageFact`** as the single source of truth for billable
consumption inside an `AgentRun`. An `AgentRun` owns zero or more
append-only `UsageFact` rows; each row records one billable consumption event:
- `kind``model_completion` (the main agent loop) | `external_capability`
| `tool_proxy`. The kind set is `OPEN`; new kinds must be surfaced, not
silently folded into an existing one.
- `provider` — e.g. `openrouter`, `mineru`, `openai_whisper`.
- `model?`, `inputTokens?`, `outputTokens?` — token metering, optional
because non-token capabilities have none.
- `quantity?` + `unit?` — non-token metering (pages, audio_seconds,
invocations), coexisting with tokens rather than replacing them.
- `costUsd?` + `costSource``provider_reported` | `pricebook_derived` |
`unknown`. `costUsd = null` means **unknown, not zero** (ADR-0022). When the
provider reported a cost, `costSource = provider_reported` and that value
wins. When only tokens are known, a later pricebook pass may derive
`costUsd` with `costSource = pricebook_derived`. When neither is possible,
`costSource = unknown` and `costUsd` stays null.
- `occurredAt` — when the consumption happened; the pricebook derivation
depends on this, not on `AgentRun.finishedAt`, because an external
capability may complete before the run finishes.
- `capabilityId?` — for `external_capability` facts, the registered capability
id (e.g. `pdf_to_md_bundle`, `audio_video_to_text`).
- `correlationId?` — external request id for reconciliation / idempotency;
not part of the aggregation key.
### Invariants
1. **Append-only.** A `UsageFact` row is never updated or deleted. A cost
correction is a new row; the old row stays. `onDelete: Cascade` exists only
so a hard run delete (itself not a normal path) cleans up its facts.
2. **Belongs to exactly one Run; never holds a lock.** A fact is a side-effect
ledger of one run, not a sub-run. External capability calls obey the same
boundary: their identity is `capabilityId + correlationId`, not `RunId`.
3. **Missing cost ≠ zero.** Aggregation MUST NOT sum `null` `costUsd` as 0.
A run whose facts all have `costUsd = null` is "cost unknown" — reported as
`runsWithoutCost`, exactly as the pre-migration `costUsd = null` runs are
today. A run with at least one `costUsd`-bearing fact contributes its sum.
### Rollup cache
`AgentRun.costUsd / inputTokens / outputTokens / costSource` columns are kept
as a **derived rollup cache**, not dropped:
- Existing non-`/usage` readers (slash `/usage`, session detail, integration
tests asserting `run.costUsd`) continue to work without code changes for
historical runs.
- On run finish, the writer writes the `UsageFact` row first and then mirrors
it onto `AgentRun` as two separate statements, not one transaction. The
fact is the truth, so it is written first; the cache is derived, so it is
written second. A crash between the two leaves the cache stale, but the
usage service re-reads `UsageFact` directly, so staleness is recoverable
(and the reverse order would lose the truth, which is not). Keeping the
fact insert and the run update in separate statements also avoids holding
the `AgentRun` row lock across the FK ShareLock taken by the insert, which
deadlocks against concurrent workspace teardown under the cascade path
`Organization → Project → AgentRun → UsageFact`.
- The org/project usage service and the slash `/usage` command read from
`UsageFact` directly — they are the canonical aggregation path.
### Migration
A new `UsageFact` table is added. For every existing `AgentRun` with a
non-null `costUsd` or non-null `inputTokens`/`outputTokens`, the migration
inserts **one synthetic `model_completion` fact** carrying the run's
provider, model, tokens, cost, and `costSource`. Its `correlationId` is the
run id, so the row is identifiable as a backfill artefact. This keeps
historical reporting correct under the new aggregation path. Pre-migration
runs with no recorded cost remain `runsWithoutCost` by design, matching
ADR-0022's "missing cost ≠ zero" rule and the existing migration
`20260709143000_agent_run_cost_tracking`'s "no backfill" stance for the
truly unrecorded.
## Consequences
- The billing model now supports external capabilities (PDF→MD, ASR, OCR, …)
without per-capability schema changes: a new capability is one new
`capabilityId` value and one or more `external_capability` facts.
- `AgentRun.costUsd` is no longer the truth; it is a convenience cache.
Readers that need the truth (cost breakdown, per-capability attribution)
must read `UsageFact`. The cache MUST be kept consistent on the write path.
- The capability registry, pricebook, and any commercial settlement remain
`OPEN` and out of pilot scope (ADR-0021). This ADR only pins the ledger
shape and invariants.
- `usage.ts` and `/usage` now perform a join against `UsageFact` rather than a
single-table scan of `AgentRun`; the index on `(runId, occurredAt)` and
`(provider, model, occurredAt)` keeps the existing org/project/report
queries bounded.
- Cost corrections (e.g. a provider rebills a run) produce a new fact row; the
rollup cache must be recomputed. The initial release writes facts once at
run finish and does not support post-hoc correction flows — that remains
`OPEN`.
## Deferred
- **Capability registry** as a first-class spec entity (`Spec.System.Capability`)
with org-scoped enable/disable, input/output contracts, metering schemas,
and org-exclusive credentials (ADR-0024 alignment). This ADR only reserves
the `capabilityId` field and the `external_capability` fact kind.
- **Pricebook** — a versioned price table keyed by `(provider, model, unit)`
with time validity. Required to actually produce `pricebook_derived` costs;
until then, facts without provider-reported cost stay `unknown`.
- **Post-hoc cost correction flow** — append-only today; correction UI and
rollup recompute are `OPEN`.
- **Commercial billing, invoicing, settlement** — still deferred per ADR-0021.
@@ -0,0 +1,64 @@
-- ADR-0026: append-only UsageFact ledger. AgentRun.costUsd/inputTokens/
-- outputTokens become a derived rollup cache; the truth is in UsageFact.
CREATE TABLE "UsageFact" (
"id" TEXT NOT NULL,
"runId" TEXT NOT NULL,
"occurredAt" TIMESTAMP(3) NOT NULL,
"kind" TEXT NOT NULL,
"provider" TEXT NOT NULL,
"model" TEXT,
"inputTokens" INTEGER,
"outputTokens" INTEGER,
"quantity" DECIMAL(18, 6),
"unit" TEXT,
"costUsd" DECIMAL(18, 8),
"costSource" TEXT NOT NULL,
"capabilityId" TEXT,
"correlationId" TEXT,
"metadata" JSONB NOT NULL,
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT "UsageFact_pkey" PRIMARY KEY ("id")
);
CREATE INDEX "UsageFact_runId_occurredAt_idx" ON "UsageFact"("runId", "occurredAt");
CREATE INDEX "UsageFact_runId_kind_idx" ON "UsageFact"("runId", "kind");
CREATE INDEX "UsageFact_provider_model_occurredAt_idx"
ON "UsageFact"("provider", "model", "occurredAt");
CREATE INDEX "UsageFact_capabilityId_occurredAt_idx"
ON "UsageFact"("capabilityId", "occurredAt");
ALTER TABLE "UsageFact"
ADD CONSTRAINT "UsageFact_runId_fkey"
FOREIGN KEY ("runId") REFERENCES "AgentRun"("id")
ON DELETE CASCADE ON UPDATE CASCADE;
-- Backfill: one synthetic model_completion fact per AgentRun that has any
-- recorded usage (cost or tokens). correlationId = run id marks these as
-- backfill artefacts (real facts use an external correlation id or null).
-- Runs with no recorded usage stay fact-less and remain runsWithoutCost,
-- matching ADR-0022's "missing cost ≠ zero" rule and the pre-existing
-- migration 20260709143000_agent_run_cost_tracking's "no backfill for the
-- truly unrecorded" stance.
INSERT INTO "UsageFact" (
"id", "runId", "occurredAt", "kind", "provider", "model",
"inputTokens", "outputTokens", "costUsd", "costSource",
"correlationId", "metadata"
)
SELECT
'usagefact_backfill_' || "AgentRun"."id",
"AgentRun"."id",
COALESCE("AgentRun"."finishedAt", "AgentRun"."startedAt", CURRENT_TIMESTAMP),
'model_completion',
"AgentRun"."provider",
"AgentRun"."model",
"AgentRun"."inputTokens",
"AgentRun"."outputTokens",
"AgentRun"."costUsd",
COALESCE("AgentRun"."costSource", 'unknown'),
"AgentRun"."id",
'{}'::jsonb
FROM "AgentRun"
WHERE "AgentRun"."costUsd" IS NOT NULL
OR "AgentRun"."inputTokens" IS NOT NULL
OR "AgentRun"."outputTokens" IS NOT NULL;
+54 -2
View File
@@ -596,9 +596,13 @@ model AgentRun {
summary String? summary String?
inputTokens Int? inputTokens Int?
outputTokens Int? outputTokens Int?
/// Provider/gateway-reported cost in USD. Null means this run has no trusted cost fact. /// ADR-0026: derived rollup cache of UsageFact rows for this run. The truth
/// is in UsageFact; this column is convenience for existing readers. Null
/// means this run has no trusted cost fact (ADR-0022: missing cost ≠ zero).
costUsd Decimal? @db.Decimal(18, 8) costUsd Decimal? @db.Decimal(18, 8)
/// Semantic source of costUsd, e.g. provider_reported. Not a pricing-estimate fallback. /// ADR-0026: semantic source of the rolled-up costUsd (provider_reported |
/// pricebook_derived | unknown). Mirrors the dominant CostSource of the
/// run's facts; not a pricing-estimate fallback.
costSource String? costSource String?
metadata Json metadata Json
error String? error String?
@@ -612,6 +616,7 @@ model AgentRun {
projectLock ProjectAgentLock? projectLock ProjectAgentLock?
messages AgentMessage[] @relation("runMessages") messages AgentMessage[] @relation("runMessages")
fileChanges AgentFileChange[] @relation("runFileChanges") fileChanges AgentFileChange[] @relation("runFileChanges")
usageFacts UsageFact[] @relation("runUsageFacts")
@@index([projectId, status]) @@index([projectId, status])
@@index([projectId, finishedAt]) @@index([projectId, finishedAt])
@@ -817,3 +822,50 @@ model AgentFileChange {
@@index([projectId]) @@index([projectId])
@@index([path]) @@index([path])
} }
// --- Usage fact ledger (ADR-0026) ----------------------------------------
/// ADR-0026: one billable consumption event inside an AgentRun. Append-only;
/// the run's AgentRun.costUsd/inputTokens/outputTokens are a derived rollup
/// of these rows, not the truth. kind=external_capability carries capabilityId
/// for media transforms (PDF→MD, audio/video→text, …). costUsd=null means
/// unknown, not zero (ADR-0022). The kind set is OPEN: new kinds must be
/// surfaced, not silently folded in.
model UsageFact {
id String @id @default(cuid())
runId String
/// When the consumption happened. Pricebook derivation uses this, not
/// AgentRun.finishedAt, because an external capability may finish before
/// the run ends.
occurredAt DateTime
/// model_completion | external_capability | tool_proxy. OPEN set.
kind String
/// e.g. openrouter, mineru, openai_whisper.
provider String
model String?
inputTokens Int?
outputTokens Int?
/// Non-token meter (pages, audio_seconds, invocations). Coexists with tokens.
quantity Decimal? @db.Decimal(18, 6)
unit String?
/// USD. Null = unknown, NOT zero (ADR-0022). When null, costSource=unknown.
costUsd Decimal? @db.Decimal(18, 8)
/// provider_reported | pricebook_derived | unknown. Required even when
/// costUsd is null, so a reader can distinguish "reported zero" from
/// "no report at all".
costSource String
/// For kind=external_capability, the registered capability id
/// (e.g. pdf_to_md_bundle, audio_video_to_text). Null for model_completion.
capabilityId String?
/// External request id for reconciliation / idempotency. Not an aggregation key.
correlationId String?
metadata Json
createdAt DateTime @default(now())
run AgentRun @relation("runUsageFacts", fields: [runId], references: [id], onDelete: Cascade)
@@index([runId, occurredAt])
@@index([runId, kind])
@@index([provider, model, occurredAt])
@@index([capabilityId, occurredAt])
}
+54 -24
View File
@@ -92,7 +92,18 @@ export function createSlashCommandRegistry(
finishedAt: { not: null }, finishedAt: { not: null },
}, },
orderBy: { finishedAt: "asc" }, orderBy: { finishedAt: "asc" },
select: { model: true, provider: true, inputTokens: true, outputTokens: true, costUsd: true }, select: {
usageFacts: {
select: {
provider: true,
model: true,
inputTokens: true,
outputTokens: true,
costUsd: true,
},
orderBy: { occurredAt: "asc" },
},
},
}); });
await sendText(rt, chatId, formatUsageReport(runs, scope), sendOptions); await sendText(rt, chatId, formatUsageReport(runs, scope), sendOptions);
}, },
@@ -118,18 +129,24 @@ async function currentRoleSessionIds(
return sessions.map((session) => session.id); return sessions.map((session) => session.id);
} }
interface UsageRun { interface UsageRunWithFacts {
readonly model: string; readonly usageFacts: readonly {
readonly provider: string; readonly provider: string;
readonly inputTokens: number | null; readonly model: string | null;
readonly outputTokens: number | null; readonly inputTokens: number | null;
readonly costUsd: unknown; readonly outputTokens: number | null;
readonly costUsd: unknown;
}[];
} }
function formatUsageReport(runs: readonly UsageRun[], scope: "current" | "project"): string { function formatUsageReport(runs: readonly UsageRunWithFacts[], scope: "current" | "project"): string {
if (runs.length === 0) return `${scope === "current" ? "当前角色会话" : "当前项目"}还没有已结束的 Agent run。`; if (runs.length === 0) return `${scope === "current" ? "当前角色会话" : "当前项目"}还没有已结束的 Agent run。`;
// Bucket by (fact.provider, fact.model) — ADR-0026: external capabilities
// carry their own provider/model and contribute their own meter, so a run
// with a main loop + an external call lands in two buckets. A run with no
// cost-bearing fact is "unrecorded" (ADR-0022: missing cost ≠ zero).
const buckets = new Map<string, { const buckets = new Map<string, {
provider: string; model: string; runs: number; inputTokens: number; outputTokens: number; costUsd: number; provider: string; model: string; facts: number; inputTokens: number; outputTokens: number; costUsd: number;
}>(); }>();
let recordedRuns = 0; let recordedRuns = 0;
let unrecordedRuns = 0; let unrecordedRuns = 0;
@@ -137,21 +154,34 @@ function formatUsageReport(runs: readonly UsageRun[], scope: "current" | "projec
let totalOutputTokens = 0; let totalOutputTokens = 0;
let totalCostUsd = 0; let totalCostUsd = 0;
for (const run of runs) { for (const run of runs) {
const costUsd = decimalToNumberOrNull(run.costUsd); const facts = run.usageFacts;
if (costUsd === null) { unrecordedRuns++; continue; } if (facts.length === 0 || !facts.some((f) => f.costUsd !== null && f.costUsd !== undefined)) {
unrecordedRuns++;
// Tokens from unrecorded runs still count toward totals (matches pre-0026).
for (const f of facts) {
totalInputTokens += f.inputTokens ?? 0;
totalOutputTokens += f.outputTokens ?? 0;
}
continue;
}
recordedRuns++; recordedRuns++;
const key = `${run.provider}\u0000${run.model}`; for (const f of facts) {
const bucket = buckets.get(key) ?? { const costUsd = decimalToNumberOrNull(f.costUsd);
provider: run.provider, model: run.model, runs: 0, inputTokens: 0, outputTokens: 0, costUsd: 0, if (costUsd === null) continue;
}; const model = f.model ?? "(unknown model)";
bucket.runs++; const key = `${f.provider}\u0000${model}`;
bucket.inputTokens += run.inputTokens ?? 0; const bucket = buckets.get(key) ?? {
bucket.outputTokens += run.outputTokens ?? 0; provider: f.provider, model, facts: 0, inputTokens: 0, outputTokens: 0, costUsd: 0,
bucket.costUsd += costUsd; };
buckets.set(key, bucket); bucket.facts += 1;
totalInputTokens += run.inputTokens ?? 0; bucket.inputTokens += f.inputTokens ?? 0;
totalOutputTokens += run.outputTokens ?? 0; bucket.outputTokens += f.outputTokens ?? 0;
totalCostUsd += costUsd; bucket.costUsd += costUsd;
buckets.set(key, bucket);
totalInputTokens += f.inputTokens ?? 0;
totalOutputTokens += f.outputTokens ?? 0;
totalCostUsd += costUsd;
}
} }
const lines = [ const lines = [
`${scope === "current" ? "当前角色会话" : "当前项目"}用量`, `${scope === "current" ? "当前角色会话" : "当前项目"}用量`,
@@ -160,7 +190,7 @@ function formatUsageReport(runs: readonly UsageRun[], scope: "current" | "projec
]; ];
if (unrecordedRuns > 0) lines.push(`另有 ${formatInteger(unrecordedRuns)} 个 run 未记录成本。`); if (unrecordedRuns > 0) lines.push(`另有 ${formatInteger(unrecordedRuns)} 个 run 未记录成本。`);
for (const bucket of buckets.values()) { for (const bucket of buckets.values()) {
lines.push(`- ${bucket.provider} / ${bucket.model}: ${formatInteger(bucket.runs)} runs, ${formatUsd(bucket.costUsd)}`); lines.push(`- ${bucket.provider} / ${bucket.model}: ${formatInteger(bucket.facts)} , ${formatUsd(bucket.costUsd)}`);
} }
return lines.join("\n"); return lines.join("\n");
} }
+29 -2
View File
@@ -604,6 +604,32 @@ export function makeTriggerHandler(deps: TriggerDeps): TriggerHandler {
data: { metadata: mergeSessionMetadata(session.metadata, metadataPatch) }, data: { metadata: mergeSessionMetadata(session.metadata, metadataPatch) },
}); });
} }
// ADR-0026: the UsageFact ledger is the truth; AgentRun.costUsd /
// inputTokens / outputTokens are a derived rollup cache for existing
// readers. We write the fact first, then mirror it onto the run.
// They are separate statements (not one transaction) so the AgentRun
// row lock is held for the shortest possible window and concurrent
// workspace teardown cannot deadlock on the FK ShareLock. A crash
// between the two leaves the cache stale, but the cache is derived
// (the usage service re-reads facts), so staleness is recoverable;
// the reverse order would lose the truth and is unrecoverable.
const finishedAt = new Date();
const reportedCost = result.costUsd;
const costSource = reportedCost !== undefined ? "provider_reported" : "unknown";
await deps.prisma.usageFact.create({
data: {
runId: run.id,
occurredAt: finishedAt,
kind: "model_completion",
provider: providerId,
model,
inputTokens: result.usage.inputTokens,
outputTokens: result.usage.outputTokens,
costUsd: reportedCost ?? null,
costSource,
metadata: {},
},
});
await deps.prisma.agentRun.update({ await deps.prisma.agentRun.update({
where: { id: run.id }, where: { id: run.id },
data: { data: {
@@ -618,9 +644,10 @@ export function makeTriggerHandler(deps: TriggerDeps): TriggerHandler {
: "FAILED", : "FAILED",
inputTokens: result.usage.inputTokens, inputTokens: result.usage.inputTokens,
outputTokens: result.usage.outputTokens, outputTokens: result.usage.outputTokens,
...(result.costUsd !== undefined ? { costUsd: result.costUsd, costSource: "provider_reported" } : {}), costUsd: reportedCost ?? null,
costSource,
error: result.error ?? null, error: result.error ?? null,
finishedAt: new Date(), finishedAt,
}, },
}); });
await writeAudit(deps.prisma, { await writeAudit(deps.prisma, {
+35
View File
@@ -73,9 +73,28 @@ export async function getSessionDetail(
inputTokens: true, inputTokens: true,
outputTokens: true, outputTokens: true,
costUsd: true, costUsd: true,
costSource: true,
startedAt: true, startedAt: true,
finishedAt: true, finishedAt: true,
error: true, error: true,
usageFacts: {
select: {
id: true,
occurredAt: true,
kind: true,
provider: true,
model: true,
inputTokens: true,
outputTokens: true,
quantity: true,
unit: true,
costUsd: true,
costSource: true,
capabilityId: true,
correlationId: true,
},
orderBy: { occurredAt: "asc" },
},
}, },
orderBy: { startedAt: "desc" }, orderBy: { startedAt: "desc" },
take: 100, take: 100,
@@ -106,9 +125,25 @@ export async function getSessionDetail(
inputTokens: r.inputTokens, inputTokens: r.inputTokens,
outputTokens: r.outputTokens, outputTokens: r.outputTokens,
costUsd: r.costUsd === null ? null : Number(r.costUsd), costUsd: r.costUsd === null ? null : Number(r.costUsd),
costSource: r.costSource,
startedAt: r.startedAt.toISOString(), startedAt: r.startedAt.toISOString(),
finishedAt: r.finishedAt?.toISOString() ?? null, finishedAt: r.finishedAt?.toISOString() ?? null,
error: r.error, error: r.error,
usageFacts: r.usageFacts.map((f) => ({
id: f.id,
occurredAt: f.occurredAt.toISOString(),
kind: f.kind,
provider: f.provider,
model: f.model,
inputTokens: f.inputTokens,
outputTokens: f.outputTokens,
quantity: f.quantity === null ? null : Number(f.quantity),
unit: f.unit,
costUsd: f.costUsd === null ? null : Number(f.costUsd),
costSource: f.costSource,
capabilityId: f.capabilityId,
correlationId: f.correlationId,
})),
})), })),
}; };
} }
+87 -17
View File
@@ -1,8 +1,14 @@
/** /**
* Org / project usage rollups from AgentRun cost facts (ADR-0021). * Org / project usage rollups from the UsageFact ledger (ADR-0021, ADR-0026).
* Operational usage accounting, not payment collection. Organizations may use * Operational usage accounting, not payment collection. Organizations may use
* either their own provider credentials or a distinct platform-managed * either their own provider credentials or a distinct platform-managed
* connection; commercial billing remains outside the pilot scope. * connection; commercial billing remains outside the pilot scope.
*
* The truth is in `UsageFact`; `AgentRun.costUsd / inputTokens / outputTokens`
* are a derived rollup cache. This service reads `UsageFact` directly so that
* external-capability consumption (PDF→MD, ASR, …) is attributed correctly,
* not just the main model loop. A run with no cost-bearing fact is
* `runsWithoutCost` — ADR-0022: missing cost ≠ zero.
*/ */
import type { Prisma, PrismaClient } from "@prisma/client"; import type { Prisma, PrismaClient } from "@prisma/client";
@@ -32,6 +38,53 @@ export interface UsageReport {
}; };
} }
type FactRow = {
readonly inputTokens: number | null;
readonly outputTokens: number | null;
readonly costUsd: unknown;
};
type RunWithFacts = {
readonly projectId: string;
readonly usageFacts: readonly FactRow[];
};
/** A run is "recorded with cost" if any of its facts carries a known costUsd. */
function runHasRecordedCost(facts: readonly FactRow[]): boolean {
return facts.some((f) => f.costUsd !== null && f.costUsd !== undefined);
}
/** Decimal→number for Prisma Decimal; null/undefined → null. */
function factCostUsdToNumber(value: unknown): number | null {
if (value === null || value === undefined) return null;
const n = typeof value === "number" ? value : Number(value);
return Number.isFinite(n) ? n : null;
}
interface RunRollup {
readonly hasCost: boolean;
readonly inputTokens: number;
readonly outputTokens: number;
readonly costUsd: number | null;
}
function rollupRun(facts: readonly FactRow[]): RunRollup {
let inputTokens = 0;
let outputTokens = 0;
let costUsd: number | null = null;
let hasCost = false;
for (const f of facts) {
inputTokens += f.inputTokens ?? 0;
outputTokens += f.outputTokens ?? 0;
const c = factCostUsdToNumber(f.costUsd);
if (c !== null) {
hasCost = true;
costUsd = (costUsd ?? 0) + c;
}
}
return { hasCost, inputTokens, outputTokens, costUsd };
}
export async function getOrgUsage( export async function getOrgUsage(
prisma: PrismaClient, prisma: PrismaClient,
input: { input: {
@@ -54,8 +107,9 @@ export async function getOrgUsage(
return emptyReport(input.from, input.to); return emptyReport(input.from, input.to);
} }
const projectIds = projects.map((p) => p.id);
const runWhere: Prisma.AgentRunWhereInput = { const runWhere: Prisma.AgentRunWhereInput = {
projectId: { in: projects.map((p) => p.id) }, projectId: { in: projectIds },
...(input.from !== undefined || input.to !== undefined ...(input.from !== undefined || input.to !== undefined
? { ? {
startedAt: { startedAt: {
@@ -66,20 +120,25 @@ export async function getOrgUsage(
: {}), : {}),
}; };
// Date range filters by run.startedAt, matching the pre-ADR-0026 semantics:
// a run is in or out of the window based on when it started, and all of its
// facts come along. Filtering facts by occurredAt independently would let a
// run contribute partial cost to a window it doesn't belong to.
const runs = await prisma.agentRun.findMany({ const runs = await prisma.agentRun.findMany({
where: runWhere, where: runWhere,
select: { select: {
projectId: true, projectId: true,
inputTokens: true, usageFacts: {
outputTokens: true, select: { inputTokens: true, outputTokens: true, costUsd: true },
costUsd: true, orderBy: { occurredAt: "asc" },
},
}, },
}); });
type MutableRow = { type MutableRow = {
projectId: string; readonly projectId: string;
projectName: string; readonly projectName: string;
folderId: string | null; readonly folderId: string | null;
runCount: number; runCount: number;
runsWithCost: number; runsWithCost: number;
runsWithoutCost: number; runsWithoutCost: number;
@@ -103,22 +162,33 @@ export async function getOrgUsage(
}); });
} }
for (const run of runs) { for (const run of runs as readonly RunWithFacts[]) {
const row = byProject.get(run.projectId); const row = byProject.get(run.projectId);
if (row === undefined) continue; if (row === undefined) continue;
const roll = rollupRun(run.usageFacts);
row.runCount += 1; row.runCount += 1;
row.inputTokens += run.inputTokens ?? 0; if (roll.hasCost) {
row.outputTokens += run.outputTokens ?? 0;
if (run.costUsd === null) {
row.runsWithoutCost += 1;
} else {
row.runsWithCost += 1; row.runsWithCost += 1;
const cost = Number(run.costUsd); row.costUsd = (row.costUsd ?? 0) + (roll.costUsd ?? 0);
row.costUsd = (row.costUsd ?? 0) + cost; } else {
row.runsWithoutCost += 1;
} }
row.inputTokens += roll.inputTokens;
row.outputTokens += roll.outputTokens;
} }
const projectsOut: ProjectUsageRow[] = [...byProject.values()]; const projectsOut: ProjectUsageRow[] = [...byProject.values()].map((r) => ({
projectId: r.projectId,
projectName: r.projectName,
folderId: r.folderId,
runCount: r.runCount,
runsWithCost: r.runsWithCost,
runsWithoutCost: r.runsWithoutCost,
inputTokens: r.inputTokens,
outputTokens: r.outputTokens,
costUsd: r.costUsd,
}));
let runCount = 0; let runCount = 0;
let runsWithCost = 0; let runsWithCost = 0;
let runsWithoutCost = 0; let runsWithoutCost = 0;
+191
View File
@@ -0,0 +1,191 @@
import { describe, it, expect, beforeEach } from "vitest";
import { prisma, resetDb, seedTestOrganization, DEFAULT_ORG_ID } from "./helpers.js";
import { getOrgUsage, getProjectUsage } from "../../src/org/usage.js";
/**
* ADR-0026: org/project usage rolls up from the UsageFact ledger, not from the
* AgentRun scalar cache. These tests pin the fact-based aggregation contract:
* - a run with a cost-bearing fact → runsWithCost, costUsd summed
* - a run whose facts all have null costUsd → runsWithoutCost (≠ zero)
* - non-token meter (quantity/unit) is stored but does not corrupt token sums
* - multiple facts per run (main loop + external capability) are summed together
* - the AgentRun rollup cache is NOT consulted by getOrgUsage
*/
describe("usage rollup from UsageFact (ADR-0026)", () => {
beforeEach(async () => {
await resetDb();
await seedTestOrganization();
});
async function seedRun(opts: {
readonly projectId: string;
readonly projectName: string;
readonly folderId?: string;
readonly facts: ReadonlyArray<{
readonly kind?: string;
readonly provider?: string;
readonly model?: string;
readonly inputTokens?: number;
readonly outputTokens?: number;
readonly costUsd?: number | null;
readonly costSource?: string;
readonly capabilityId?: string;
readonly quantity?: number | null;
readonly unit?: string | null;
}>;
}): Promise<void> {
const project = await prisma.project.create({
data: {
id: opts.projectId,
organizationId: DEFAULT_ORG_ID,
name: opts.projectName,
workspaceDir: `/tmp/test-${opts.projectId}`,
...(opts.folderId !== undefined ? { folderId: opts.folderId } : {}),
},
});
const startedAt = new Date("2026-07-18T10:00:00Z");
const finishedAt = new Date("2026-07-18T10:05:00Z");
const run = await prisma.agentRun.create({
data: {
projectId: project.id,
entrypoint: "FEISHU",
provider: "openrouter",
model: "mock-model",
metadata: {},
status: "COMPLETED",
prompt: "test",
startedAt,
finishedAt,
// Deliberately set the rollup cache to values that differ from the
// facts, to prove getOrgUsage reads facts, not the cache.
inputTokens: 999,
outputTokens: 999,
costUsd: 999,
costSource: "provider_reported",
},
});
for (const [i, f] of opts.facts.entries()) {
await prisma.usageFact.create({
data: {
runId: run.id,
occurredAt: new Date(startedAt.getTime() + i * 1000),
kind: f.kind ?? "model_completion",
provider: f.provider ?? "openrouter",
model: f.model ?? "mock-model",
inputTokens: f.inputTokens ?? null,
outputTokens: f.outputTokens ?? null,
costUsd: f.costUsd ?? null,
costSource: f.costSource ?? (f.costUsd !== undefined && f.costUsd !== null ? "provider_reported" : "unknown"),
capabilityId: f.capabilityId ?? null,
quantity: f.quantity ?? null,
unit: f.unit ?? null,
metadata: {},
},
});
}
}
it("sums cost and tokens from facts, ignoring the AgentRun rollup cache", async () => {
await seedRun({
projectId: "proj-a",
projectName: "A",
facts: [{ inputTokens: 10, outputTokens: 5, costUsd: 0.0023 }],
});
await seedRun({
projectId: "proj-b",
projectName: "B",
facts: [{ inputTokens: 20, outputTokens: 10, costUsd: 0.01 }],
});
const report = await getOrgUsage(prisma, { organizationId: DEFAULT_ORG_ID });
expect(report.totals.runCount).toBe(2);
expect(report.totals.runsWithCost).toBe(2);
expect(report.totals.runsWithoutCost).toBe(0);
expect(report.totals.inputTokens).toBe(30);
expect(report.totals.outputTokens).toBe(15);
expect(report.totals.costUsd).toBeCloseTo(0.0123, 8);
});
it("treats a run with only null-cost facts as runsWithoutCost, never zero", async () => {
await seedRun({
projectId: "proj-unknown",
projectName: "Unknown",
facts: [{ inputTokens: 7, outputTokens: 3, costUsd: null, costSource: "unknown" }],
});
await seedRun({
projectId: "proj-known",
projectName: "Known",
facts: [{ inputTokens: 4, outputTokens: 2, costUsd: 0.05 }],
});
const report = await getOrgUsage(prisma, { organizationId: DEFAULT_ORG_ID });
expect(report.totals.runCount).toBe(2);
expect(report.totals.runsWithCost).toBe(1);
expect(report.totals.runsWithoutCost).toBe(1);
// The unknown-cost run still contributes its tokens.
expect(report.totals.inputTokens).toBe(11);
expect(report.totals.outputTokens).toBe(5);
// And its cost is NOT counted as zero — only the known cost is summed.
expect(report.totals.costUsd).toBeCloseTo(0.05, 8);
});
it("sums multiple facts per run (main loop + external capability)", async () => {
await seedRun({
projectId: "proj-multi",
projectName: "Multi",
facts: [
{ kind: "model_completion", inputTokens: 100, outputTokens: 50, costUsd: 0.02 },
{
kind: "external_capability",
provider: "mineru",
model: null,
inputTokens: null,
outputTokens: null,
costUsd: 0.015,
costSource: "provider_reported",
capabilityId: "pdf_to_md_bundle",
quantity: 12,
unit: "pages",
},
],
});
const report = await getOrgUsage(prisma, { organizationId: DEFAULT_ORG_ID });
expect(report.totals.runCount).toBe(1);
expect(report.totals.runsWithCost).toBe(1);
expect(report.totals.inputTokens).toBe(100);
expect(report.totals.outputTokens).toBe(50);
expect(report.totals.costUsd).toBeCloseTo(0.035, 8);
});
it("a run with no facts at all is runsWithoutCost and contributes nothing", async () => {
await seedRun({ projectId: "proj-empty", projectName: "Empty", facts: [] });
const report = await getOrgUsage(prisma, { organizationId: DEFAULT_ORG_ID });
expect(report.totals.runCount).toBe(1);
expect(report.totals.runsWithCost).toBe(0);
expect(report.totals.runsWithoutCost).toBe(1);
expect(report.totals.inputTokens).toBe(0);
expect(report.totals.outputTokens).toBe(0);
expect(report.totals.costUsd).toBeNull();
});
it("getProjectUsage returns just that project's row", async () => {
await seedRun({ projectId: "proj-x", projectName: "X", facts: [{ costUsd: 0.1, inputTokens: 1, outputTokens: 1 }] });
await seedRun({ projectId: "proj-y", projectName: "Y", facts: [{ costUsd: 0.2, inputTokens: 2, outputTokens: 2 }] });
const row = await getProjectUsage(prisma, { organizationId: DEFAULT_ORG_ID, projectId: "proj-x" });
expect(row.projectId).toBe("proj-x");
expect(row.runCount).toBe(1);
expect(row.costUsd).toBeCloseTo(0.1, 8);
});
it("returns an empty report for an org with no projects", async () => {
const report = await getOrgUsage(prisma, { organizationId: DEFAULT_ORG_ID });
expect(report.projects).toHaveLength(0);
expect(report.totals.runCount).toBe(0);
expect(report.totals.costUsd).toBeNull();
});
});
+3
View File
@@ -290,6 +290,9 @@ function mockPrisma(): PrismaClient {
findUnique: vi.fn(async () => null), findUnique: vi.fn(async () => null),
update: vi.fn(async () => ({ id: "run-1" })), update: vi.fn(async () => ({ id: "run-1" })),
}, },
usageFact: {
create: vi.fn(async () => ({ id: "fact-1" })),
},
projectAgentLock: { projectAgentLock: {
findUnique: vi.fn(async () => (lock === null ? null : { runId: lock.runId })), findUnique: vi.fn(async () => (lock === null ? null : { runId: lock.runId })),
create: vi.fn(async (args: { data: { projectId: string; runId: string } }) => { create: vi.fn(async (args: { data: { projectId: string; runId: string } }) => {
+5 -1
View File
@@ -10,6 +10,7 @@ import Spec.System.Agent.Run
import Spec.System.Agent.AgentRole import Spec.System.Agent.AgentRole
import Spec.System.Agent.Memory import Spec.System.Agent.Memory
import Spec.System.Agent.AgentSurface import Spec.System.Agent.AgentSurface
import Spec.System.Agent.Usage
import Spec.System.Lock import Spec.System.Lock
import Spec.System.Permission import Spec.System.Permission
import Spec.System.PermissionGrant import Spec.System.PermissionGrant
@@ -39,11 +40,14 @@ likec4 已画出结构;这里补语义:
- `Memory` —— 按需上下文:锚点类别(ADR-0003)+ MCP 工具按 run/project 上下文授权。 - `Memory` —— 按需上下文:锚点类别(ADR-0003)+ MCP 工具按 run/project 上下文授权。
- `AgentSurface` —— agent 执行面被 run 的工作区所界定(ADR-0018);与 Lock 正交—— - `AgentSurface` —— agent 执行面被 run 的工作区所界定(ADR-0018);与 Lock 正交——
Lock 限定并发,Surface 限定波及面。机制 OPEN。 Lock 限定并发,Surface 限定波及面。机制 OPEN。
- `Usage` —— Run 内 append-only 用量计量事实账本(ADR-0026);主模型 completion、
外部能力调用、代理网关旁路共用同一账本。fact 不持锁、不跨 run;`costUsd = none`
表示未知而非零(ADR-0022)。Run 上的 cost/token 标量是其 rollup cache。
- `Permission` —— read⊂edit⊂manage 角色体系、能力推导、单调性;force-release 在格外。 - `Permission` —— read⊂edit⊂manage 角色体系、能力推导、单调性;force-release 在格外。
- `PermissionGrant` —— grant(resource×principal×role)与 settings(六 policy 旋钮) - `PermissionGrant` —— grant(resource×principal×role)与 settings(六 policy 旋钮)
(ADR-0004);组合规则 OPEN。 (ADR-0004);组合规则 OPEN。
- `Audit` —— customer Project/Run 审计从简(内容 OPEN);Platform Audit 由 - `Audit` —— customer Project/Run 审计从简(内容 OPEN);Platform Audit 由
`PlatformAdministration` 独立承载。 `PlatformAdministration` 独立承载。
标识符见 `Spec.Prelude`。决策出处:ADR-0001..0004, 0018, 0020..0024 标识符见 `Spec.Prelude`。决策出处:ADR-0001..0004, 0018, 0020..0026
-/ -/
+92
View File
@@ -0,0 +1,92 @@
import Spec.Prelude
/-!
# Usage —— Run 内用量计量事实账本(ADR-0026)
一次 `AgentRun` 内可能产生**多条**可计费消耗:主模型 completion、外部能力调用
(PDF→MD bundle、音视频转写等)、或代理网关侧旁路调用。这些消耗**不**是子 Run——
它们不持项目锁、不占 session 语义、不复用 `AgentRun` 状态机。它们是 Run 内的
append-only **计量事实**(`UsageFact`)。
`AgentRun` 上的 cost/token 标量是 facts 的派生 rollup cache,不是真相来源;真相在
`UsageFact` 行。这一层抽象让"主模型"与"外部能力"两类消耗共享同一账本,而非给后者
各开一种特化字段或 nested Run。
核心不变式(`PINNED`, ADR-0022 + ADR-0026):
- **append-only** —— fact 一旦写入只读不更不删;`costUsd` 修正走新 fact,不改旧行。
- **`costUsd = none` 表示"未知",不是"零成本"** —— ADR-0022:missing provider cost
remains unknown rather than zero。聚合时不得把 none 当 0 求和。
- **fact 不持锁、不跨 run** —— 它是 Run 内的副作用账本,不是又一次 `@bot` 生命周期。
外部能力调用亦同:它的语义边界是 `capabilityId` + `correlationId`,不是 `RunId`。
- **价目回算是派生** —— `costSource = pricebookDerived` 时,`costUsd` 由
`occurredAt × (provider, model)` 查价目表派生;provider 实报(`providerReported`)
优先于派生。无价目可查时 `costSource = unknown`,`costUsd = none`。
外部能力(capability)注册表、价目表(pricebook)、商业结算均 `OPEN`,pilot 不做
(ADR-0021 仍 defer commercial billing)。本模块只 pin **账本结构与不变式**。
-/
namespace Spec.System.Agent
variable (I : Identifiers) (Time Cost Quantity : Type)
/-- 计量事实类型(`PINNED`, ADR-0026)。集合完整性 `OPEN`——新增 kind 须 surface,
不得静默把外部消耗塞进既有 kind。 -/
inductive UsageFactKind where
/-- 主 agent loop 的模型 completion(`PINNED`)。 -/
| modelCompletion
/-- 外部能力调用(`PINNED`;如 pdf_to_md_bundle、audio_video_to_text)。 -/
| externalCapability
/-- 代理网关侧旁路调用(`PINNED`;非主 loop 的模型调用,如嵌入工具内的子请求)。 -/
| toolProxy
/-- 成本来源(`PINNED`, ADR-0026)。优先级:provider 实报 > 价目派生 > unknown。 -/
inductive CostSource where
/-- provider/gateway 实报(`PINNED`)。 -/
| providerReported
/-- 用 `occurredAt × (provider, model)` 价目表派生(`PINNED`)。 -/
| pricebookDerived
/-- 缺失,而非零(`PINNED`;ADR-0022 missing cost ≠ zero)。 -/
| unknown
/-- 一次可计费消耗的计量事实(`PINNED`, ADR-0026)。append-only;一次 run ≥0 条。
字段分两组:**归属与时间**(run/occurredAt/kind)用于聚合与价目回算;
**计量与成本**(tokens/quantity/unit/costUsd/costSource)用于账目本身。
非 token 计量(页//次)走 `quantity + unit`,与 token 并存而非取代。 -/
structure UsageFact where
/-- 所属 run(`PINNED`;fact 不跨 run,fact 不持锁,ADR-0026)。 -/
run : I.RunId
/-- 消耗发生时刻(`PINNED`);价目回算依赖此字段而非 run.finishedAt,
因为外部能力可能早于 run 结束。 -/
occurredAt : Time
/-- 事实类型(`PINNED`, ADR-0026)。 -/
kind : UsageFactKind
/-- provider 标识(`PINNED`;如 openrouter、mineru、openai_whisper)。 -/
provider : String
/-- 模型标识(`OPEN`;非模型计费可为 none)。 -/
model : Option String
/-- 输入 tokens(`OPEN`;非 token 计量可为 none)。 -/
inputTokens : Option Nat
/-- 输出 tokens(`OPEN`)。 -/
outputTokens : Option Nat
/-- 非 token 计量数值(`OPEN`;如页数、音频秒数)。与 tokens 并存。 -/
quantity : Option Quantity
/-- 计量单位(`OPEN`;如 "pages"、"audio_seconds"、"invocations")。 -/
unit : Option String
/-- 成本(`PINNED`;none = 未知,非零,ADR-0022)。 -/
costUsd : Option Cost
/-- 成本来源(`PINNED`;costUsd ≠ none 时必填,costUsd = none 时为 `unknown`)。 -/
costSource : CostSource
/-- 外部能力标识(`OPEN`;kind = externalCapability 时填,如 pdf_to_md_bundle)。 -/
capabilityId : Option String
/-- 外部请求关联 id(`OPEN`;对账/幂等用,不参与聚合)。 -/
correlationId : Option String
/-- Fact 的成本是否**已知**(`PINNED`, ADR-0026)。聚合 rollup 时,只对已知成本求和;
未知成本的 fact 不贡献 0,而是让汇总保持未知(若所有 fact 均未知)或部分未知。 -/
def UsageFact.HasKnownCost (fact : UsageFact I Time Cost Quantity) : Prop :=
fact.costUsd none
end Spec.System.Agent