forked from bai/curriculum-project-hub
feat(hub): usage fact ledger for run-scoped cost attribution (ADR-0026)
Replace the single-scalar cost model on AgentRun with an append-only UsageFact ledger. One AgentRun owns zero or more UsageFact rows; each records one billable consumption event (model completion, external capability, or tool proxy) with its own provider/model/tokens/quantity/ cost. AgentRun.costUsd/inputTokens/outputTokens become a derived rollup cache. This unblocks external capabilities (PDF->MD bundle, audio/video->text) that bill in non-token units (pages, seconds) through a different provider than the main agent loop, without per-capability schema changes or nested AgentRuns (which would pollute lock/admission/session semantics). Contract: - spec/Spec/System/Agent/Usage.lean pins UsageFact, UsageFactKind, CostSource and three invariants: append-only; belongs to one run, never holds a lock; missing cost != zero (ADR-0022). - ADR-0026 records the decision, the rejected nested-Run alternative, the rollup cache strategy, and the deferred capability registry / pricebook / post-hoc correction flows. Schema: - UsageFact model with indexes on (runId, occurredAt), (runId, kind), (provider, model, occurredAt), (capabilityId, occurredAt). - Migration backfills one synthetic model_completion fact per existing run with recorded cost/tokens (correlationId = runId marks backfill); truly unrecorded runs stay runsWithoutCost per ADR-0022. Write path (trigger finish): - Write the UsageFact first, then mirror it onto AgentRun as two separate statements (not one transaction). The fact is the truth so it goes first; the cache is derived so it goes second. A crash between them leaves the cache stale but the usage service re-reads facts directly, so this is recoverable; the reverse order would lose the truth. Separate statements also avoid an AgentRun row lock held across the insert's FK ShareLock, which deadlocked concurrent workspace teardown under the Organization->Project->AgentRun->UsageFact cascade. Read paths: - org/usage.ts aggregates from UsageFact, ignoring the AgentRun cache. - slash /usage buckets by (fact.provider, fact.model); a run with a main loop + an external call lands in two buckets. - session detail exposes usageFacts[] + costSource for future per-run cost-breakdown UI. Tests: - usage.test.ts: 6 integration tests pin fact aggregation, missing-cost- !=-zero, multi-fact-per-run, empty-run, project-level, empty-org. - trigger.test.ts: existing /usage assertion ($0.0023, openrouter / mock-model) passes on the fact path. - feishu-reactions mock prisma gains usageFact.create.
This commit is contained in:
@@ -0,0 +1,64 @@
|
||||
-- ADR-0026: append-only UsageFact ledger. AgentRun.costUsd/inputTokens/
|
||||
-- outputTokens become a derived rollup cache; the truth is in UsageFact.
|
||||
|
||||
CREATE TABLE "UsageFact" (
|
||||
"id" TEXT NOT NULL,
|
||||
"runId" TEXT NOT NULL,
|
||||
"occurredAt" TIMESTAMP(3) NOT NULL,
|
||||
"kind" TEXT NOT NULL,
|
||||
"provider" TEXT NOT NULL,
|
||||
"model" TEXT,
|
||||
"inputTokens" INTEGER,
|
||||
"outputTokens" INTEGER,
|
||||
"quantity" DECIMAL(18, 6),
|
||||
"unit" TEXT,
|
||||
"costUsd" DECIMAL(18, 8),
|
||||
"costSource" TEXT NOT NULL,
|
||||
"capabilityId" TEXT,
|
||||
"correlationId" TEXT,
|
||||
"metadata" JSONB NOT NULL,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
CONSTRAINT "UsageFact_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
CREATE INDEX "UsageFact_runId_occurredAt_idx" ON "UsageFact"("runId", "occurredAt");
|
||||
CREATE INDEX "UsageFact_runId_kind_idx" ON "UsageFact"("runId", "kind");
|
||||
CREATE INDEX "UsageFact_provider_model_occurredAt_idx"
|
||||
ON "UsageFact"("provider", "model", "occurredAt");
|
||||
CREATE INDEX "UsageFact_capabilityId_occurredAt_idx"
|
||||
ON "UsageFact"("capabilityId", "occurredAt");
|
||||
|
||||
ALTER TABLE "UsageFact"
|
||||
ADD CONSTRAINT "UsageFact_runId_fkey"
|
||||
FOREIGN KEY ("runId") REFERENCES "AgentRun"("id")
|
||||
ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- Backfill: one synthetic model_completion fact per AgentRun that has any
|
||||
-- recorded usage (cost or tokens). correlationId = run id marks these as
|
||||
-- backfill artefacts (real facts use an external correlation id or null).
|
||||
-- Runs with no recorded usage stay fact-less and remain runsWithoutCost,
|
||||
-- matching ADR-0022's "missing cost ≠ zero" rule and the pre-existing
|
||||
-- migration 20260709143000_agent_run_cost_tracking's "no backfill for the
|
||||
-- truly unrecorded" stance.
|
||||
INSERT INTO "UsageFact" (
|
||||
"id", "runId", "occurredAt", "kind", "provider", "model",
|
||||
"inputTokens", "outputTokens", "costUsd", "costSource",
|
||||
"correlationId", "metadata"
|
||||
)
|
||||
SELECT
|
||||
'usagefact_backfill_' || "AgentRun"."id",
|
||||
"AgentRun"."id",
|
||||
COALESCE("AgentRun"."finishedAt", "AgentRun"."startedAt", CURRENT_TIMESTAMP),
|
||||
'model_completion',
|
||||
"AgentRun"."provider",
|
||||
"AgentRun"."model",
|
||||
"AgentRun"."inputTokens",
|
||||
"AgentRun"."outputTokens",
|
||||
"AgentRun"."costUsd",
|
||||
COALESCE("AgentRun"."costSource", 'unknown'),
|
||||
"AgentRun"."id",
|
||||
'{}'::jsonb
|
||||
FROM "AgentRun"
|
||||
WHERE "AgentRun"."costUsd" IS NOT NULL
|
||||
OR "AgentRun"."inputTokens" IS NOT NULL
|
||||
OR "AgentRun"."outputTokens" IS NOT NULL;
|
||||
@@ -596,9 +596,13 @@ model AgentRun {
|
||||
summary String?
|
||||
inputTokens Int?
|
||||
outputTokens Int?
|
||||
/// Provider/gateway-reported cost in USD. Null means this run has no trusted cost fact.
|
||||
/// ADR-0026: derived rollup cache of UsageFact rows for this run. The truth
|
||||
/// is in UsageFact; this column is convenience for existing readers. Null
|
||||
/// means this run has no trusted cost fact (ADR-0022: missing cost ≠ zero).
|
||||
costUsd Decimal? @db.Decimal(18, 8)
|
||||
/// Semantic source of costUsd, e.g. provider_reported. Not a pricing-estimate fallback.
|
||||
/// ADR-0026: semantic source of the rolled-up costUsd (provider_reported |
|
||||
/// pricebook_derived | unknown). Mirrors the dominant CostSource of the
|
||||
/// run's facts; not a pricing-estimate fallback.
|
||||
costSource String?
|
||||
metadata Json
|
||||
error String?
|
||||
@@ -612,6 +616,7 @@ model AgentRun {
|
||||
projectLock ProjectAgentLock?
|
||||
messages AgentMessage[] @relation("runMessages")
|
||||
fileChanges AgentFileChange[] @relation("runFileChanges")
|
||||
usageFacts UsageFact[] @relation("runUsageFacts")
|
||||
|
||||
@@index([projectId, status])
|
||||
@@index([projectId, finishedAt])
|
||||
@@ -817,3 +822,50 @@ model AgentFileChange {
|
||||
@@index([projectId])
|
||||
@@index([path])
|
||||
}
|
||||
|
||||
// --- Usage fact ledger (ADR-0026) ----------------------------------------
|
||||
|
||||
/// ADR-0026: one billable consumption event inside an AgentRun. Append-only;
|
||||
/// the run's AgentRun.costUsd/inputTokens/outputTokens are a derived rollup
|
||||
/// of these rows, not the truth. kind=external_capability carries capabilityId
|
||||
/// for media transforms (PDF→MD, audio/video→text, …). costUsd=null means
|
||||
/// unknown, not zero (ADR-0022). The kind set is OPEN: new kinds must be
|
||||
/// surfaced, not silently folded in.
|
||||
model UsageFact {
|
||||
id String @id @default(cuid())
|
||||
runId String
|
||||
/// When the consumption happened. Pricebook derivation uses this, not
|
||||
/// AgentRun.finishedAt, because an external capability may finish before
|
||||
/// the run ends.
|
||||
occurredAt DateTime
|
||||
/// model_completion | external_capability | tool_proxy. OPEN set.
|
||||
kind String
|
||||
/// e.g. openrouter, mineru, openai_whisper.
|
||||
provider String
|
||||
model String?
|
||||
inputTokens Int?
|
||||
outputTokens Int?
|
||||
/// Non-token meter (pages, audio_seconds, invocations). Coexists with tokens.
|
||||
quantity Decimal? @db.Decimal(18, 6)
|
||||
unit String?
|
||||
/// USD. Null = unknown, NOT zero (ADR-0022). When null, costSource=unknown.
|
||||
costUsd Decimal? @db.Decimal(18, 8)
|
||||
/// provider_reported | pricebook_derived | unknown. Required even when
|
||||
/// costUsd is null, so a reader can distinguish "reported zero" from
|
||||
/// "no report at all".
|
||||
costSource String
|
||||
/// For kind=external_capability, the registered capability id
|
||||
/// (e.g. pdf_to_md_bundle, audio_video_to_text). Null for model_completion.
|
||||
capabilityId String?
|
||||
/// External request id for reconciliation / idempotency. Not an aggregation key.
|
||||
correlationId String?
|
||||
metadata Json
|
||||
createdAt DateTime @default(now())
|
||||
|
||||
run AgentRun @relation("runUsageFacts", fields: [runId], references: [id], onDelete: Cascade)
|
||||
|
||||
@@index([runId, occurredAt])
|
||||
@@index([runId, kind])
|
||||
@@index([provider, model, occurredAt])
|
||||
@@index([capabilityId, occurredAt])
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user