diff --git a/docs/adr/0030-filelib-version-store-is-real-git.md b/docs/adr/0030-filelib-version-store-is-real-git.md new file mode 100644 index 0000000..7df82b2 --- /dev/null +++ b/docs/adr/0030-filelib-version-store-is-real-git.md @@ -0,0 +1,157 @@ +# ADR 0030: The File Library VersionStore Is a Real Git Repository per Project + +## Status + +Accepted. + +## Context + +The file library (`hub/src/database/filelib/`, an independent subsystem that does +not reuse the Hub's own `Folder`/`Project` tree from ADR-0021) stores each project +as a versioned file tree behind the `VersionStore` port (contract C1). Until now +the only implementation was `createInMemoryVersionStore`: a `Map` of per-file +version chains, with `VersionId` as a per-repository monotonic counter +(`v1`, `v2`, …), a hand-written line differ, and an optional JSON snapshot of the +entire storage root written to `/.version-store.json` so that a +process restart did not lose the demo data. + +Two things about the surrounding design were already settled in code and are +confirmed here rather than changed: + +- **A `FOLDER` node has no on-disk existence.** `FileLibNode.storageDir` is + `NULL` for folders. The tree is `parentId` plus the `pathIds` materialized path; + nothing in the filesystem mirrors it. +- **Projects are flat under one root, keyed by id.** `storageDir` is + `/` where `nodeId` is a `randomUUID()`. Names never enter + the path, which is why `renameNode` touches no disk state and does not rewrite + descendant paths. + +What was never true is the part the names implied. `HUB_FILELIB_STORAGE_ROOT` was +documented as "the project git repository root" and `fileService` was documented +as observing a "git first, then audit" ordering, but no code in the repository +ever invoked git. `versionStore.init(storageDir)` inserted a `Map` entry; the +directory was never created. Every project's entire content and history lived in +one process-global JSON file. The header comment and `README.md` both marked this +as a placeholder awaiting an npm package from the versioning team. + +That package has not arrived, and the in-memory store's properties are not +acceptable for real teacher data: a corrupt or lost `.version-store.json` loses +every project at once, the whole storage root is rewritten on every commit, and +`VersionId` values are meaningless outside the process that minted them. + +## Decision + +**Each file library project is a real Git repository at +`/`.** `VersionStore.init` creates the directory and runs +`git init` there. This is the production implementation; +`createInMemoryVersionStore` is retained for tests only. + +**`VersionId` is a Git commit hash.** The full 40-hex object name, as printed by +`git rev-parse`. It is no longer a per-repository counter. + +**File-level versioning (D16) maps onto commit history as follows.** A write +touches exactly one path and produces exactly one commit. The version of a file is +the hash of the most recent commit that modified that path — `git log -1 -- +`. Consequently: + +- Two files in one project have independent versions, because a commit that + touches `a.md` does not appear in `git log -- b.md`. This preserves the D16 + property that advancing one file does not invalidate another file's + `baseVersion`, even though commits are repository-global objects. +- `baseVersion` checking (S1/S2) compares the caller's id against the current + per-file version. `baseVersion: null` means create, and conflicts if the path + already exists at `HEAD`. +- Reading version `V` of a path means `git show V:`, which is the content as + of that commit, not the content the commit introduced to some other file. + +**Deletion is a commit, not a tombstone record.** `remove` runs `git rm` and +commits, so the path is absent from `HEAD` and `list` stops reporting it, while +`git show :` still resolves. The in-memory store expressed +this as a `deleted: true` chain entry; the observable API semantics are the same. + +**Git is invoked as a subprocess, not through a library.** `node:child_process` +`execFile` with an argument array, no new npm dependency. Every invocation is +hardened, and the hardening is load-bearing rather than incidental: + +- `-c core.hooksPath=` and `-c commit.gpgsign=false`, plus + `GIT_CONFIG_GLOBAL=/dev/null` and `GIT_CONFIG_SYSTEM=/dev/null`. A project + repository is *data*, uploaded by teachers. Without this, a committed + `.git/hooks/` entry or a developer's global `gitconfig` would execute or alter + server-side behavior. +- `GIT_LITERAL_PATHSPECS=1` and `--` before every path, so a filename is never + reinterpreted as an option or as pathspec magic (`:(glob)`). +- `GIT_TERMINAL_PROMPT=0`, so a repository never blocks a request waiting on + credentials. +- Author identity is passed per-commit via `GIT_AUTHOR_*`/`GIT_COMMITTER_*` + environment variables, never written into the repository's config. The git + author name is the acting user's `displayName` (falling back to `userId` when + absent), and the email is `@filelib.paradigm-edu.net`. The email + deliberately keys on `userId` rather than the display name, because nicknames + change and identity attribution must not drift with them. Characters that would + break git's ident line (`<`, `>`, newlines) are stripped from the name. +- `--git-dir=/.git` and `--work-tree=` are pinned on + every invocation, and `GIT_DIR`/`GIT_WORK_TREE`/`GIT_INDEX_FILE`/ + `GIT_OBJECT_DIRECTORY` are removed from the child environment. Git otherwise + searches *upward* for a `.git`, and the storage root is frequently nested inside + another repository — the local development default `hub/.filelib-repos` sits + inside this very repo. Without pinning, operations on a project directory that + has no repository of its own silently retarget the enclosing repository. + Existence is therefore tested on the filesystem (`/.git`), not with + `git rev-parse --git-dir`, which merely echoes a pinned value back. + +**Writes to one repository remain serialized in-process**, as under S4, because +concurrent git invocations contend on `index.lock`. This is a single-process +guarantee only; see Consequences. + +## Consequences + +- `.version-store.json` is not read or migrated by the new store. Existing + development data under `HUB_FILELIB_STORAGE_ROOT` does not appear in the git + store; those projects report `repo_not_found` until recreated. No production + data exists to migrate, since the in-memory store was never production-viable. +- `VersionId` changes shape in API responses (`GET .../files/*`, history, and the + 409 `currentVersion` detail). Clients must keep treating it as an opaque + string; `filelib-web` already does. +- `VersionInfo.author` now comes back as the git author name, which is the acting + user's display name at commit time (or the `userId` when no display name is + known). Commits written without an author carry a fixed `filelib` identity + rather than `undefined`. Display names are point-in-time: renaming a user does + not rewrite existing commits, and the stable identifier stays in the email. +- `VersionStore.commit`/`remove` take a structured `CommitAuthor` + (`{ userId, displayName? }`) rather than a bare author string, so the port can + express both the stable key and the display label. Deletion carries the same + identity as any other commit. +- Serialization is per-process. Two Hub processes sharing a storage root can race + on the same repository and surface a git lock error rather than a clean + conflict. The alpha Silo deployment (ADR-0025) is one process per organization, + so this is not currently reachable; a multi-process deployment needs either a + database advisory lock keyed by project id or a single writer. +- `git` must be present on the host. Absence is a startup-visible failure of + project creation (`provision_failed`), not a silent degradation. +- Repository content is now attacker-influenced data on disk. The path validation + in `fileService.validateFilePath` (rejecting `..`, `.git`, absolute paths, + control characters) moves from hygiene to a security boundary, and + `versionStore` re-checks it rather than trusting callers. + +## Alternatives considered + +- **`isomorphic-git` or `simple-git`.** Both add a dependency to carry work that + three `execFile` calls do. `isomorphic-git` additionally reimplements the object + layer, so its bugs would be ours to diagnose. +- **One commit per repository state, with the repository head as the version.** + Simpler mapping, but it breaks D16: any write would invalidate every other + file's `baseVersion`, turning independent edits into false conflicts. +- **Keeping the counter as `VersionId` alongside git.** Requires a durable + counter-to-hash mapping outside git, which is the state the decision removes. +- **Bare repositories with a git index-only write path.** Avoids a working tree, + but every read and write becomes plumbing (`hash-object`, `update-index`, + `commit-tree`), for no benefit at this scale. + +## Deferred + +- Cross-process write serialization (advisory lock keyed by project id). +- Garbage collection and pack maintenance policy for long-lived repositories. +- Whether export builds (`exportService`) should read a git tree directly instead + of going through the `listFiles`/`readFile` port. +- Recovering `provisionStatus=FAILED` projects by re-running `init`; the status + machine records the failure but nothing retries it yet. diff --git a/hub/filelib-web/src/lib/FilesPanel.svelte b/hub/filelib-web/src/lib/FilesPanel.svelte index 0806afd..a98aaed 100644 --- a/hub/filelib-web/src/lib/FilesPanel.svelte +++ b/hub/filelib-web/src/lib/FilesPanel.svelte @@ -2,6 +2,7 @@ import { api } from "./api.js"; import { toastOk, toastErr } from "./stores.js"; import { selectedFilePath, filesVersion } from "./browser.js"; + import { loadConfig } from "./config.js"; import type { FileEntry, NodeDetail } from "./types.js"; import Modal from "./Modal.svelte"; import Icon from "./Icon.svelte"; @@ -13,8 +14,20 @@ let showNewFile = $state(false); let newPath = $state(""); let newContent = $state(""); + let newMessage = $state(""); + // 上传弹窗:选完文件先暂存,等用户确认路径与 commit 信息再传。 + let pendingFile = $state(null); + let uploadPath = $state(""); + let uploadMessage = $state(""); + let uploading = $state(false); // bind:this 的目标要用 $state,否则 svelte 5 warn 不会正确更新。 let uploadInput = $state(null); + /** 上传上限由 /database/config 下发(后端 HUB_FILELIB_MAX_FILE_BYTES)。 */ + let maxFileBytes = $state(null); + + const maxLabel = $derived( + maxFileBytes === null ? "" : `${(maxFileBytes / 1024 / 1024).toFixed(maxFileBytes % (1024 * 1024) === 0 ? 0 : 1)}MB`, + ); const canEdit = $derived(node.role !== "VIEW"); @@ -34,17 +47,25 @@ void loadFiles(); }); + $effect(() => { + void loadConfig() + .then((c) => (maxFileBytes = c.maxFileBytes)) + .catch(() => (maxFileBytes = null)); + }); + async function submitNewFile(): Promise { const path = newPath.trim(); if (path === "") return; + const message = newMessage.trim(); try { await api(`/database/api/projects/${node.id}/file`, { method: "PUT", - body: { path, content: newContent }, + // message 缺失时不传 —— 后端回退到【用户名】修改了【路径】。 + body: message === "" ? { path, content: newContent } : { path, content: newContent, message }, }); toastOk("已创建"); showNewFile = false; - newPath = ""; newContent = ""; + newPath = ""; newContent = ""; newMessage = ""; await loadFiles(); } catch (e) { toastErr(e instanceof Error ? e.message : String(e)); @@ -60,28 +81,51 @@ return btoa(bin); } - async function doUpload(e: Event): Promise { + /** 选文件只负责暂存与预填;真正上传在弹窗确认后。 */ + function pickFile(e: Event): void { const input = e.target as HTMLInputElement; const file = input.files?.[0]; input.value = ""; if (!file) return; - if (file.size > 10 * 1024 * 1024) { - toastErr("文件超过 10MB 上限"); + // 上限取后端值;拉不到就不在前端拦 —— 后端横竖会以 413 file_too_large 兼底, + // 前端这道只是省一次往返。 + if (maxFileBytes !== null && file.size > maxFileBytes) { + toastErr(`文件超过 ${maxLabel} 上限`); return; } - const targetPath = prompt("保存到路径(可含目录):", "材料/" + file.name); - if (!targetPath) return; - const bytes = new Uint8Array(await file.arrayBuffer()); - const isBinary = bytes.includes(0); - const body = isBinary - ? { path: targetPath, content: u8ToBase64(bytes), encoding: "base64" } - : { path: targetPath, content: new TextDecoder("utf-8").decode(bytes), encoding: "utf8" }; + pendingFile = file; + uploadPath = `材料/${file.name}`; + uploadMessage = ""; + } + + function cancelUpload(): void { + pendingFile = null; + uploadPath = ""; + uploadMessage = ""; + } + + async function submitUpload(): Promise { + const file = pendingFile; + const targetPath = uploadPath.trim(); + if (file === null || targetPath === "") return; + uploading = true; try { + const bytes = new Uint8Array(await file.arrayBuffer()); + const isBinary = bytes.includes(0); + const message = uploadMessage.trim(); + const body: Record = isBinary + ? { path: targetPath, content: u8ToBase64(bytes), encoding: "base64" } + : { path: targetPath, content: new TextDecoder("utf-8").decode(bytes), encoding: "utf8" }; + // message 缺失时不传 —— 后端回退到【用户名】修改了【路径】。 + if (message !== "") body["message"] = message; await api(`/database/api/projects/${node.id}/file`, { method: "PUT", body }); - toastOk("已上传 " + file.name); + toastOk(`已上传 ${file.name}`); + cancelUpload(); await loadFiles(); } catch (err) { toastErr(err instanceof Error ? err.message : String(err)); + } finally { + uploading = false; } } @@ -93,7 +137,7 @@
- +
{/if} @@ -131,9 +175,36 @@ +
+ + +
{/if} + +{#if pendingFile} + +
+ 已选文件 +

{pendingFile.name} · {pendingFile.size} B

+
+
+ + +
+
+ + +
+
+ + +
+
+{/if} diff --git a/hub/filelib-web/src/lib/config.ts b/hub/filelib-web/src/lib/config.ts index 95f8a6d..02a939f 100644 --- a/hub/filelib-web/src/lib/config.ts +++ b/hub/filelib-web/src/lib/config.ts @@ -12,6 +12,8 @@ import { api } from "./api.js"; export interface AppConfig { readonly orgSlug: string; readonly devLoginEnabled: boolean; + /** 单文件上传上限(字节)。后端 `HUB_FILELIB_MAX_FILE_BYTES` 的生效值。 */ + readonly maxFileBytes: number; } let cached: AppConfig | null = null; diff --git a/hub/src/database/README.md b/hub/src/database/README.md index af2bde1..3543cec 100644 --- a/hub/src/database/README.md +++ b/hub/src/database/README.md @@ -110,7 +110,8 @@ allowDevLoginBypass = (NODE_ENV !== "production") && HUB_DEV_LOGIN_BYPASS 为真 | `filelib/grantService.ts` | 授权管理 + 契约 8.1 矩阵强制 + force_adjust | | `filelib/fileService.ts` | 文件路径安全 + 版本化读写(先 git 后审计的顺序铁律) | | `filelib/exportService.ts` | 导出 job 状态机(D10 异步)+ ExportAdapter port | -| `filelib/versionStore.ts` | 契约 C1 port + 内存实现(版本团队 npm 包到位后替换) | +| `filelib/versionStore.ts` | 契约 C1 port + 内存实现(**仅测试用**,ADR-0030) | +| `filelib/gitVersionStore.ts` | **生产** C1 实现:一项目一 git 仓库,VersionId = commit hash(ADR-0030) | | `filelib/groupResolver.ts` | 契约 C2 port(+ 已弃用的 Team 过渡实现,ADR-0028) | | `filelib/memberGroupResolver.ts` | **默认** C2 实现:读 in-hub MemberGroup 闭包(ADR-0028) | | `filelib/memberGroupService.ts` | 成员组 CRUD(含改名)+ 成员增删 + 闭包维护 + 搜索(ADR-0028) | @@ -125,9 +126,20 @@ allowDevLoginBypass = (NODE_ENV !== "production") && HUB_DEV_LOGIN_BYPASS 为真 - `HUB_GROUP_SERVICE_URL` — 外部 Group 服务地址(C2);**未配置时读 in-hub MemberGroup 闭包**(ADR-0028 起的默认;此前是扁平 hub Team) -> ⚠️ 开发期注意:当前 VersionStore 是**进程内存**实现,**服务重启后仓库全失**, -> 此前创建的项目再访问文件会报 `repo_not_found`(需重建项目)。版本团队的 -> 持久化 git 包到位后此问题消失。 +## 存储布局(ADR-0030) + +- **文件夹不落盘。** `FOLDER` 节点的 `storageDir` 永为 `NULL`,磁盘上不存在任何 + 对应目录;树形只由 `parentId` + `pathIds`(id 编码的物化路径)表达。 +- **项目扁平、以 id 命名。** 仓库就是 `/`(nodeId 是 uuid)。 + 名字不进路径 —— 这是 rename 不动磁盘、也不重写后代路径的原因。 +- **一项目一真 git 仓库。** `init` 建目录 + `git init`;`VersionId` 是 40 位 commit + hash;某文件的版本 = `git log -1 -- `,所以一个文件的提交不会使另一个 + 文件的 `baseVersion` 失效(D16)。删除也是一个 commit,旧版本仍可读。 +- git 用 `execFile` 调系统二进制,不引依赖。**每次调用都禁 hooks、隔离全局/系统 + gitconfig、`GIT_LITERAL_PATHSPECS=1`** —— 项目仓库是老师上传的**数据**而非可信代码。 +- 宿主必须有 `git`;缺失时建项目报 `provision_failed`。 +- 写入仅**进程内**串行化。共享 storage root 的多进程会在同一仓库上竞争 git 锁; + alpha Silo 是一 org 一进程(ADR-0025),目前不可达。 关键语义速查: diff --git a/hub/src/database/filelib/fileService.ts b/hub/src/database/filelib/fileService.ts index f6cac66..02f7e4a 100644 --- a/hub/src/database/filelib/fileService.ts +++ b/hub/src/database/filelib/fileService.ts @@ -17,7 +17,20 @@ import type { FileLibNode, PrismaClient } from "@prisma/client"; export const FILE_PATH_MAX_LENGTH = 512; export const FILE_PATH_MAX_DEPTH = 32; -export const FILE_CONTENT_MAX_BYTES = 10 * 1024 * 1024; // OPEN-5 初值 + +/** 单文件上限的出厂默认值(OPEN-5 初值)。实际生效值见 `resolveMaxFileBytes`。 */ +export const FILE_CONTENT_MAX_BYTES_DEFAULT = 10 * 1024 * 1024; + +/** + * 单文件字节上限:`HUB_FILELIB_MAX_FILE_BYTES` 覆盖,缺省 10MiB。 + * 非法值(非正整数/NaN)按缺省处理 —— 配置写错不该让上传静默变成 0 上限。 + */ +export function resolveMaxFileBytes(raw: string | undefined = process.env["HUB_FILELIB_MAX_FILE_BYTES"]): number { + if (raw === undefined || raw.trim() === "") return FILE_CONTENT_MAX_BYTES_DEFAULT; + const parsed = Number(raw.trim()); + if (!Number.isSafeInteger(parsed) || parsed <= 0) return FILE_CONTENT_MAX_BYTES_DEFAULT; + return parsed; +} const CONTROL_CHARS = /[\p{C}]/u; const FORBIDDEN_SEGMENTS = new Set(["", ".", "..", ".git"]); @@ -52,6 +65,8 @@ export function validateFilePath(raw: string): string { export interface FileDeps extends AccessDeps { readonly prisma: PrismaClient; readonly versionStore: VersionStore; + /** 单文件字节上限。装配处用 `resolveMaxFileBytes()` 求值,缺省即出厂值。 */ + readonly maxFileBytes?: number | undefined; } type ProjectChain = { readonly node: FileLibNode; readonly storageDir: string }; @@ -101,13 +116,25 @@ function encodeContent(buffer: Buffer): { readonly encoding: FileContentEncoding : { encoding: "utf8", content: buffer.toString("utf8") }; } -function checkSize(content: string | Buffer): void { +function checkSize(content: string | Buffer, maxBytes: number): void { const bytes = typeof content === "string" ? Buffer.byteLength(content, "utf8") : content.byteLength; - if (bytes > FILE_CONTENT_MAX_BYTES) { - throw new FileLibError(413, "file_too_large", `file exceeds ${FILE_CONTENT_MAX_BYTES} bytes`); + if (bytes > maxBytes) { + throw new FileLibError(413, "file_too_large", `file exceeds ${maxBytes} bytes`); } } +/** + * commit message 的默认文案:`【用户名】修改了【路径】`。 + * 用户名取 displayName,缺失回退 userId(权限判定从不看它)。 + * 显式传 message 的调用方优先 —— 这里只填空缺。 + */ +export function defaultCommitMessage(actor: FileLibActor, filePath: string): string { + const who = actor.displayName === undefined || actor.displayName.trim() === "" + ? actor.userId + : actor.displayName.trim(); + return `【${who}】修改了【${filePath}】`; +} + async function auditFile( deps: FileDeps, actor: FileLibActor, @@ -221,14 +248,21 @@ export async function commitFile( ): Promise<{ readonly version: string }> { const filePath = validateFilePath(input.path); const content = decodeContent(input.content, input.encoding ?? "utf8"); - checkSize(content); + checkSize(content, deps.maxFileBytes ?? resolveMaxFileBytes()); const project = await requireProject(deps, actor, projectId, "EDIT"); + // 调用方传了非空 message 则用它,否则回退默认文案。 + // 空串必须当作没传:`git commit -m ""` 会以 empty commit message 失败。 + const trimmedMessage = input.message?.trim(); + const message = trimmedMessage === undefined || trimmedMessage === "" + ? defaultCommitMessage(actor, filePath) + : trimmedMessage; const result: CommitResult = await deps.versionStore.commit(project.storageDir, filePath, { baseVersion: input.baseVersion, content, - message: input.message, - author: actor.userId, + message, + // 身份:name=displayName(回退 userId),email=@域名(git 实现里拼)。 + author: { userId: actor.userId, displayName: actor.displayName }, }); if (result.status === "conflict") { @@ -247,7 +281,7 @@ export async function commitFile( input.baseVersion === null ? FILE_LIB_AUDIT_ACTIONS.fileUpload : FILE_LIB_AUDIT_ACTIONS.fileCommit, project, filePath, - { version: result.version, message: input.message ?? null }, + { version: result.version, message }, ); return { version: result.version }; } @@ -261,7 +295,10 @@ export async function deleteFile( ): Promise { const filePath = validateFilePath(rawPath); const project = await requireProject(deps, actor, projectId, "EDIT"); - const result = await deps.versionStore.remove(project.storageDir, filePath, baseVersion); + const result = await deps.versionStore.remove(project.storageDir, filePath, baseVersion, { + userId: actor.userId, + displayName: actor.displayName, + }); if (result.status === "conflict") { await auditFile(deps, actor, FILE_LIB_AUDIT_ACTIONS.fileConflictDetected, project, filePath, { baseVersion, diff --git a/hub/src/database/filelib/gitVersionStore.ts b/hub/src/database/filelib/gitVersionStore.ts new file mode 100644 index 0000000..443394e --- /dev/null +++ b/hub/src/database/filelib/gitVersionStore.ts @@ -0,0 +1,408 @@ +/** + * VersionStore(契约 C1)的真 git 实现 —— ADR-0030。 + * + * 一个项目一个 git 仓库,位于 /(nodeId 是 uuid; + * 文件夹不落盘,见 ADR-0030 Context)。VersionId = commit hash。 + * + * D16 文件级版本的映射(ADR-0030 Decision):一次写只碰一个路径、只产生一个 + * commit;某文件的版本 = `git log -1 -- ` 的 hash。因此 a.md 的提交不出现在 + * b.md 的 log 里,两者 baseVersion 互不失效 —— 尽管 commit 本身是仓库级对象。 + * + * 不引 npm 依赖:三条 execFile 就够,见 ADR-0030 Alternatives。 + */ + +import { execFile } from "node:child_process"; +import { access, mkdir, writeFile } from "node:fs/promises"; +import path from "node:path"; +import { FileLibError } from "./model.js"; +import type { + CommitAuthor, + CommitRequest, + CommitResult, + FileEntry, + VersionId, + VersionInfo, + VersionStore, +} from "./versionStore.js"; + +/** 无 author 时的固定身份(ADR-0030 Consequences:不再是 undefined)。 */ +const FALLBACK_AUTHOR = "filelib"; +/** 提交者 email 域名:`@filelib.paradigm-edu.net`。 */ +const AUTHOR_EMAIL_DOMAIN = "filelib.paradigm-edu.net"; + +/** + * 每次调用都带的加固(ADR-0030 Decision)。项目仓库是老师上传的**数据**, + * 不是可信代码:hooks 必须禁用,全局/系统 gitconfig 必须隔离,否则仓库内容 + * 或开发者机器上的配置就能改变服务端行为。 + */ +const HARDENING_ARGS = ["-c", "core.hooksPath=", "-c", "commit.gpgsign=false"] as const; + +/** + * 仓库定位参数。**这两个不能省**:git 默认会沿目录树**向上**找 `.git`, + * 而 storage root 很可能就在另一个 git 仓库里(本地开发的默认值 + * `hub/.filelib-repos` 就在本 repo 内)。不钉死的后果是项目目录没自己的 `.git` + * 时,所有命令默默落到**外层仓库**上 —— 轻则 `git add` 报 ignored, + * 重则把老师的文件提交进源码仓。 + */ +function repoArgs(projectDir: string): readonly string[] { + return [`--git-dir=${path.join(projectDir, ".git")}`, `--work-tree=${projectDir}`]; +} + +const HARDENING_ENV = { + GIT_CONFIG_GLOBAL: "/dev/null", + GIT_CONFIG_SYSTEM: "/dev/null", + // 文件名永远不被重解释为 pathspec magic(`:(glob)` 等)。 + GIT_LITERAL_PATHSPECS: "1", + // 仓库不得因为凭据提示卡住一个 HTTP 请求。 + GIT_TERMINAL_PROMPT: "0", +} as const; + +/** + * 继承来的这几个会劫持全部命令(比如 hub 自身被一个 git hook 启动时), + * 必须从子进程 env 里**删掉**而不是置空 —— 置空在 git 里的含义并不统一。 + * 我们只认 repoArgs 里显式传的那一份。 + */ +const STRIPPED_ENV = ["GIT_DIR", "GIT_WORK_TREE", "GIT_INDEX_FILE", "GIT_OBJECT_DIRECTORY"] as const; + +function childEnv(extra: Readonly>): NodeJS.ProcessEnv { + const env: NodeJS.ProcessEnv = { ...process.env, ...HARDENING_ENV, ...extra }; + for (const key of STRIPPED_ENV) delete env[key]; + return env; +} + +interface GitResult { + readonly stdout: Buffer; + readonly code: number; + readonly stderr: string; +} + +/** execFile 的 args 数组形式:不拼 shell,文件名不参与命令解析。 */ +function runGit( + cwd: string, + args: readonly string[], + env: Readonly> = {}, +): Promise { + return execGit(cwd, [...repoArgs(cwd), ...args], env); +} + +/** + * 不钉 --git-dir 的调用。**只给 `git init` 用** —— 那一刻 `.git` 尚不存在, + * 钉上去 git 会直接报错。init 自己总是在 cwd 建仓,不会向上找。 + */ +function runGitBare( + cwd: string, + args: readonly string[], + env: Readonly> = {}, +): Promise { + return execGit(cwd, args, env); +} + +function execGit( + cwd: string, + args: readonly string[], + env: Readonly>, +): Promise { + return new Promise((resolve, reject) => { + execFile( + "git", + [...HARDENING_ARGS, ...args], + { + cwd, + encoding: "buffer", + env: childEnv(env), + maxBuffer: 64 * 1024 * 1024, + windowsHide: true, + }, + (error, stdout, stderr) => { + const out = Buffer.isBuffer(stdout) ? stdout : Buffer.from(String(stdout)); + const err = Buffer.isBuffer(stderr) ? stderr.toString("utf8") : String(stderr); + if (error === null) { + resolve({ stdout: out, code: 0, stderr: err }); + return; + } + const code = (error as NodeJS.ErrnoException & { code?: number | string }).code; + if (code === "ENOENT") { + // 坑:spawn 的 ENOENT 有两种来源且**报错完全一样**(都是 path:"git"、 + // syscall:"spawn git") —— git 真的不在 PATH 上,或者 cwd 目录不存在。 + // 后者在这里是常态(DB 里有 storageDir、磁盘上却没建过,比如内存 store + // 时代留下的旧项目),必须报 repo_not_found 而不是冤枉 git 没装。 + void access(cwd).then( + () => reject(new FileLibError(500, "git_missing", "git executable not found on PATH")), + () => reject(new FileLibError(404, "repo_not_found", `repository directory missing: ${cwd}`)), + ); + return; + } + // 非零退出是常规控制流(文件不存在、空仓库等),交给调用点判断。 + resolve({ stdout: out, code: typeof code === "number" ? code : 1, stderr: err }); + }, + ); + }); +} + +async function requireGit(cwd: string, args: readonly string[], env?: Record): Promise { + const res = await runGit(cwd, args, env); + if (res.code !== 0) { + throw new FileLibError(500, "git_failed", `git ${args[0] ?? ""} failed: ${res.stderr.trim()}`); + } + return res.stdout; +} + +async function requireGitBare(cwd: string, args: readonly string[]): Promise { + const res = await runGitBare(cwd, args); + if (res.code !== 0) { + throw new FileLibError(500, "git_failed", `git ${args[0] ?? ""} failed: ${res.stderr.trim()}`); + } + return res.stdout; +} + +/** + * S4:同仓库写操作串行化。git 的并发写会在 index.lock 上打架, + * 串行化把它变成干净的 conflict 返回值而不是锁错误。仅进程内有效(ADR-0030)。 + */ +function createKeySerializer(): (key: string, fn: () => Promise) => Promise { + const tails = new Map>(); + return (key: string, fn: () => Promise): Promise => { + const prev = tails.get(key) ?? Promise.resolve(); + const next = prev.then(fn, fn); + tails.set(key, next.catch(() => undefined)); + return next; + }; +} + +/** + * 仓库内相对路径的再校验。fileService.validateFilePath 已经把过一遍, + * 但 ADR-0030 把这条从卫生升级为安全边界 —— 本层不信调用方。 + */ +function safeRelPath(filePath: string): string { + const normalized = filePath.normalize("NFC"); + if (normalized === "" || path.isAbsolute(normalized) || normalized.includes("\\")) { + throw new FileLibError(400, "invalid_path", `unsafe path: ${filePath}`); + } + const segments = normalized.split("/"); + for (const segment of segments) { + if (segment === "" || segment === "." || segment === ".." || segment === ".git") { + throw new FileLibError(400, "invalid_path", `unsafe path segment in: ${filePath}`); + } + } + // 解析后必须仍在仓库内(符号链接由 git 自身不跟随 + 此处前缀检查共同兜住)。 + const resolved = path.posix.normalize(normalized); + if (resolved.startsWith("..") || path.isAbsolute(resolved)) { + throw new FileLibError(400, "invalid_path", `unsafe path: ${filePath}`); + } + return resolved; +} + +/** + * 提交者身份 → git author/committer。 + * name = displayName(缺失回退 userId);email = `@filelib.paradigm-edu.net`。 + * email 用 userId 而不用 displayName:昵称会改,身份追溯不能跟着漂。 + * name 里的换行/`<`/`>` 必须清掉 —— 它们会破坏 git 的 ident 行格式。 + */ +function authorEnv(author: CommitAuthor | undefined): Record { + const rawName = author?.displayName; + const fallback = author?.userId ?? FALLBACK_AUTHOR; + const name = (rawName === undefined || rawName.trim() === "" ? fallback : rawName.trim()) + .replace(/[<>\n\r]/g, " ") + .trim(); + const localPart = (author?.userId ?? FALLBACK_AUTHOR).replace(/[^\w.-]/g, "_"); + const email = `${localPart}@${AUTHOR_EMAIL_DOMAIN}`; + return { + GIT_AUTHOR_NAME: name === "" ? FALLBACK_AUTHOR : name, + GIT_AUTHOR_EMAIL: email, + GIT_COMMITTER_NAME: name === "" ? FALLBACK_AUTHOR : name, + GIT_COMMITTER_EMAIL: email, + }; +} + +export function createGitVersionStore(): VersionStore { + const serialize = createKeySerializer(); + + /** + * 未 init → repo_not_found。 + * 不能用 `git rev-parse --git-dir`:repoArgs 已把 --git-dir 钉死,rev-parse 会 + * 原样回显它而不验证存在;而不钉死时它又会向上找到外层仓库。所以直接 + * 测文件系统:项目目录里必须有属于它自己的 `.git`。 + */ + async function requireRepo(projectDir: string): Promise { + try { + await access(path.join(projectDir, ".git")); + } catch { + throw new FileLibError(404, "repo_not_found", `repository not initialized: ${projectDir}`); + } + } + + /** 该路径在 HEAD 上的当前版本;不存在(或从未提交)→ null。 */ + async function currentVersion(projectDir: string, filePath: string): Promise { + const exists = await runGit(projectDir, ["cat-file", "-e", `HEAD:${filePath}`]); + if (exists.code !== 0) return null; // 空仓库、已删除、或从无此文件 + const log = await runGit(projectDir, ["log", "-1", "--format=%H", "--", filePath]); + if (log.code !== 0) return null; + const hash = log.stdout.toString("utf8").trim(); + return hash === "" ? null : hash; + } + + async function headVersion(projectDir: string, filePath: string): Promise { + const version = await currentVersion(projectDir, filePath); + if (version === null) { + throw new FileLibError(404, "file_not_found", `file not found: ${filePath}`); + } + return version; + } + + /** commit id 必须存在,否则 version_not_found(而非把 git 错误透出去)。 */ + async function requireCommit(projectDir: string, version: VersionId): Promise { + const res = await runGit(projectDir, ["rev-parse", "--verify", "--quiet", `${version}^{commit}`]); + if (res.code !== 0) { + throw new FileLibError(404, "version_not_found", `version not found: ${version}`); + } + } + + /** 提交暂存区里已备好的单个路径。返回新 commit hash。 */ + async function commitPath( + projectDir: string, + filePath: string, + message: string, + author: CommitAuthor | undefined, + ): Promise { + const env = authorEnv(author); + await requireGit(projectDir, ["commit", "--quiet", "--allow-empty", "-m", message, "--", filePath], env); + const hash = await requireGit(projectDir, ["rev-parse", "HEAD"]); + return hash.toString("utf8").trim(); + } + + return { + /** S7 幂等:已有仓库就不重建。 */ + async init(projectDir) { + await serialize(projectDir, async () => { + await mkdir(projectDir, { recursive: true }); + try { + await access(path.join(projectDir, ".git")); + return; // 已是仓库,不清空 + } catch { /* 继续 init */ } + // init 用 runGitBare:此时 .git 尚不存在,钉 --git-dir 反而会让 git 报错。 + await requireGitBare(projectDir, ["init", "--quiet"]); + // 默认分支名不依赖宿主 git 版本/配置(全局配置已被隔离)。 + await requireGit(projectDir, ["symbolic-ref", "HEAD", "refs/heads/main"]); + }); + }, + + async list(projectDir, prefix) { + await requireRepo(projectDir); + const res = await runGit(projectDir, ["ls-tree", "-r", "-l", "-z", "HEAD"]); + if (res.code !== 0) return []; // 空仓库(无 HEAD) + const out: FileEntry[] = []; + for (const record of res.stdout.toString("utf8").split("\0")) { + if (record === "") continue; + // 形如:" \t" + const tab = record.indexOf("\t"); + if (tab === -1) continue; + const meta = record.slice(0, tab).split(/\s+/); + const entryPath = record.slice(tab + 1); + if (meta[1] !== "blob") continue; + if (prefix !== undefined && !entryPath.startsWith(prefix)) continue; + out.push({ path: entryPath, size: Number.parseInt(meta[3] ?? "0", 10) || 0 }); + } + return out.sort((a, b) => a.path.localeCompare(b.path)); + }, + + async head(projectDir, filePath) { + await requireRepo(projectDir); + return headVersion(projectDir, safeRelPath(filePath)); + }, + + async read(projectDir, filePath, at) { + await requireRepo(projectDir); + const rel = safeRelPath(filePath); + if (at === undefined) { + await headVersion(projectDir, rel); // 存在性 → 404 file_not_found + const res = await runGit(projectDir, ["show", `HEAD:${rel}`]); + if (res.code !== 0) { + throw new FileLibError(404, "file_not_found", `file not found: ${rel}`); + } + return res.stdout; + } + await requireCommit(projectDir, at); + const res = await runGit(projectDir, ["show", `${at}:${rel}`]); + if (res.code !== 0) { + // commit 存在但该版本里没有这个路径。 + throw new FileLibError(404, "version_not_found", `version not found: ${rel}@${at}`); + } + return res.stdout; + }, + + async commit(projectDir, filePath, req: CommitRequest): Promise { + const rel = safeRelPath(filePath); + return serialize(projectDir, async (): Promise => { + await requireRepo(projectDir); + const current = await currentVersion(projectDir, rel); + + // S2:baseVersion=null 表新建,已存在即 conflict; + // S1:否则要求 baseVersion 精确等于当前版本。 + if (req.baseVersion === null) { + if (current !== null) return { status: "conflict", currentVersion: current }; + } else if (req.baseVersion !== current) { + return { status: "conflict", currentVersion: current ?? req.baseVersion }; + } + + const abs = path.join(projectDir, rel); + await mkdir(path.dirname(abs), { recursive: true }); + await writeFile(abs, req.content); + await requireGit(projectDir, ["add", "--", rel]); + const message = req.message ?? (req.baseVersion === null ? `create ${rel}` : `update ${rel}`); + const version = await commitPath(projectDir, rel, message, req.author); + return { status: "ok", version }; + }); + }, + + async remove(projectDir, filePath, baseVersion, author): Promise { + const rel = safeRelPath(filePath); + return serialize(projectDir, async (): Promise => { + await requireRepo(projectDir); + const current = await currentVersion(projectDir, rel); + if (current === null) { + throw new FileLibError(404, "file_not_found", `file not found: ${rel}`); + } + if (baseVersion !== current) return { status: "conflict", currentVersion: current }; + await requireGit(projectDir, ["rm", "--quiet", "--", rel]); + const version = await commitPath(projectDir, rel, `remove ${rel}`, author); + return { status: "ok", version }; + }); + }, + + async diff(projectDir, filePath, from, to) { + await requireRepo(projectDir); + const rel = safeRelPath(filePath); + await requireCommit(projectDir, from); + await requireCommit(projectDir, to); + const res = await runGit(projectDir, ["diff", from, to, "--", rel]); + if (res.code !== 0) { + throw new FileLibError(500, "git_failed", `git diff failed: ${res.stderr.trim()}`); + } + return res.stdout.toString("utf8"); + }, + + async history(projectDir, filePath, limit) { + await requireRepo(projectDir); + const rel = safeRelPath(filePath); + const args = ["log", "--format=%H%x1f%an%x1f%aI%x1f%s%x1e"]; + if (limit !== undefined) args.push(`-${limit}`); + args.push("--", rel); + const res = await runGit(projectDir, args); + if (res.code !== 0) return []; // 空仓库 + const out: VersionInfo[] = []; + for (const record of res.stdout.toString("utf8").split("\x1e")) { + const line = record.trim(); + if (line === "") continue; + const [version, author, committedAt, message] = line.split("\x1f"); + if (version === undefined) continue; + out.push({ + version, + message: message ?? "", + author: author === undefined || author === "" ? undefined : author, + committedAt: committedAt ?? "", + }); + } + return out; // git log 已是新→旧 + }, + }; +} diff --git a/hub/src/database/filelib/guards.ts b/hub/src/database/filelib/guards.ts index e934302..5d84b32 100644 --- a/hub/src/database/filelib/guards.ts +++ b/hub/src/database/filelib/guards.ts @@ -43,5 +43,7 @@ export async function requireFileLibActor( return { userId: auth.user.id, isWebsiteAdmin: WEBSITE_ADMIN_ROLES.includes(membership.role), + // 仅用于 commit message 的【用户名】;权限判定一律走 userId。 + displayName: auth.user.displayName, }; } diff --git a/hub/src/database/filelib/routeShared.ts b/hub/src/database/filelib/routeShared.ts index a4d23fa..8e622dd 100644 --- a/hub/src/database/filelib/routeShared.ts +++ b/hub/src/database/filelib/routeShared.ts @@ -21,6 +21,8 @@ export interface FileLibRouteDeps { readonly groupResolver: GroupResolver; readonly versionStore: VersionStore; readonly exportAdapters: readonly ExportAdapter[]; + /** 单文件字节上限(`HUB_FILELIB_MAX_FILE_BYTES`)。 */ + readonly maxFileBytes: number; } /** 组装 treeService 依赖(路由处理内直接使用)。 */ diff --git a/hub/src/database/filelib/treeService.ts b/hub/src/database/filelib/treeService.ts index 75a1825..2658dcc 100644 --- a/hub/src/database/filelib/treeService.ts +++ b/hub/src/database/filelib/treeService.ts @@ -35,6 +35,11 @@ export interface FileLibActor { readonly userId: string; /** silo org OWNER/ADMIN(契约 C4 适配)。仅 root 创建/force_adjust 用,不给读旁路。 */ readonly isWebsiteAdmin: boolean; + /** + * 展示名(飞书昵称)。只用于生成 commit message 的【用户名】部分; + * 权限判定一律用 userId。缺失时回退到 userId。 + */ + readonly displayName?: string | undefined; } export interface TreeServiceDeps { @@ -229,6 +234,8 @@ export async function createNode( } if (input.kind === "PROJECT") { + // ADR-0030:项目扁平居于同一根下、以 uuid 命名;名字不进路径(所以 rename + // 不动磁盘)。FOLDER 永不赋值 —— 文件夹不落盘,只存在于 DB 的 parentId/pathIds。 storageDir = path.join(deps.storageRoot, id); } @@ -308,6 +315,7 @@ export async function createNode( }); // provisioning 状态机(Metis 风险#1):DB 行已持久,init 失败 → FAILED 可重试/对账。 + // ADR-0030:这一步真的建目录并 `git init`;宿主无 git 则此处 provision_failed。 if (input.kind === "PROJECT" && storageDir !== null) { try { await deps.versionStore.init(storageDir); diff --git a/hub/src/database/filelib/versionStore.ts b/hub/src/database/filelib/versionStore.ts index 08cd844..9f5a1ac 100644 --- a/hub/src/database/filelib/versionStore.ts +++ b/hub/src/database/filelib/versionStore.ts @@ -1,12 +1,14 @@ /** - * VersionStore port(契约 C1)+ 开发用内存实现。 + * VersionStore port(契约 C1)+ **仅测试用**的内存实现。 * - * 版本团队交付 npm 工具包后,用同一接口替换 createInMemoryVersionStore。 + * 生产实现是 `gitVersionStore.ts`(一项目一 git 仓库,VersionId = commit hash), + * 见 ADR-0030。本文件留下来只为让不关心版本落盘的测试快速起个 store;它的 + * VersionId 是每仓库计数器(`v1`/`v2`),与生产**不同形**,不要据此写断言。 * 语义红线(计划"Mock 保真红线"):冲突走返回值(S1)、baseVersion=null 表新建(S2)、 * init 幂等(S7)、同 projectDir 写操作串行化(S4)、文件级版本(D16)。 * - * 持久化:传 persistPath 时把仓库快照落盘(JSON),重启后恢复 —— 纯粹为开发期 - * demo 稳定,不改变任何语义;生产由真包替换,此文件不参与。 + * 持久化:传 persistPath 时把仓库快照落盘(JSON),重启后恢复。ADR-0030 之后 + * 已无生产调用点 —— 生产走 git,不再有这份进程级 JSON 快照。 */ import { mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs"; @@ -15,12 +17,21 @@ import { FileLibError } from "./model.js"; export type VersionId = string; +/** + * 提交者身份。`userId` 是稳定主键(追溯用),`displayName` 只影响展示。 + * git 实现把它们映射成 `name `(ADR-0030)。 + */ +export interface CommitAuthor { + readonly userId: string; + readonly displayName?: string | undefined; +} + export interface CommitRequest { /** 编辑起始版本;null 表示新建文件(已存在则 conflict,S2)。 */ readonly baseVersion: VersionId | null; readonly content: string | Buffer; readonly message?: string | undefined; - readonly author?: string | undefined; + readonly author?: CommitAuthor | undefined; } export type CommitResult = @@ -45,7 +56,13 @@ export interface VersionStore { head(projectDir: string, filePath: string): Promise; read(projectDir: string, filePath: string, at?: VersionId): Promise; commit(projectDir: string, filePath: string, req: CommitRequest): Promise; - remove(projectDir: string, filePath: string, baseVersion: VersionId): Promise; + /** 删除也是一次提交,所以同样带提交者身份。 */ + remove( + projectDir: string, + filePath: string, + baseVersion: VersionId, + author?: CommitAuthor, + ): Promise; diff(projectDir: string, filePath: string, from: VersionId, to: VersionId): Promise; history(projectDir: string, filePath: string, limit?: number): Promise; } @@ -80,6 +97,13 @@ function toBuffer(content: string | Buffer): Buffer { return typeof content === "string" ? Buffer.from(content, "utf8") : content; } +/** VersionInfo.author 是展示字符串;语义与 git 实现对齐(取 displayName,回退 userId)。 */ +function authorLabel(author: CommitAuthor | undefined): string | undefined { + if (author === undefined) return undefined; + const name = author.displayName; + return name === undefined || name.trim() === "" ? author.userId : name.trim(); +} + /** 极简 unified-diff(mock 保真够用;真包的 diff 以版本团队为准)。 */ function naiveDiff(fromText: string, toText: string): string { const a = fromText.split("\n"); @@ -221,7 +245,7 @@ export function createInMemoryVersionStore(persistPath?: string): VersionStore { version, content: toBuffer(req.content), message: req.message ?? `commit ${version}`, - author: req.author, + author: authorLabel(req.author), committedAt: new Date().toISOString(), deleted: false, }); @@ -231,7 +255,7 @@ export function createInMemoryVersionStore(persistPath?: string): VersionStore { }); }, - async remove(projectDir, filePath, baseVersion) { + async remove(projectDir, filePath, baseVersion, author) { return serialize(projectDir, async (): Promise => { const repo = requireRepo(projectDir); const chain = repo.files.get(filePath) ?? []; @@ -247,7 +271,7 @@ export function createInMemoryVersionStore(persistPath?: string): VersionStore { version, content: Buffer.alloc(0), message: `remove ${filePath}`, - author: undefined, + author: authorLabel(author), committedAt: new Date().toISOString(), deleted: true, }); diff --git a/hub/src/database/routes/databaseRoutes.ts b/hub/src/database/routes/databaseRoutes.ts index 9cfaf35..cdfe9b7 100644 --- a/hub/src/database/routes/databaseRoutes.ts +++ b/hub/src/database/routes/databaseRoutes.ts @@ -29,7 +29,8 @@ import { registerFileLibRoutes } from "./filelibRoutes.js"; import { registerFileRoutes } from "./fileRoutes.js"; import { registerMemberGroupRoutes } from "./memberGroupRoutes.js"; import { registerTeacherApp } from "./teacherApp.js"; -import { createInMemoryVersionStore } from "../filelib/versionStore.js"; +import { createGitVersionStore } from "../filelib/gitVersionStore.js"; +import { resolveMaxFileBytes } from "../filelib/fileService.js"; import { createMemberGroupResolver } from "../filelib/memberGroupResolver.js"; import { createHttpGroupResolver } from "../filelib/groupResolverHttp.js"; import { createManifestStubAdapter } from "../filelib/exportService.js"; @@ -60,6 +61,8 @@ export async function registerDatabaseRoutes( app.get("/database/config", async () => ({ orgSlug: config.siloOrganizationSlug, devLoginEnabled: config.allowDevLoginBypass, + // 上传上限由后端下发,前端不再写死 —— 两处各自硬编码会随配置漂。 + maxFileBytes: resolveMaxFileBytes(), })); // 概览页统计。登录 + silo org OWNER/ADMIN 才给 —— 它聚合的是全 org 口径的 @@ -132,7 +135,8 @@ export async function registerDatabaseRoutes( } // 文件库(独立模块,《文件库-接口契约.md》):API + 老师端 /app 静态托管。 - // 依赖装配:VersionStore 当前为内存+快照实现(版本团队 npm 包到位后替换); + // 依赖装配:VersionStore 是真 git —— 一项目一仓库 /, + // VersionId = commit hash(ADR-0030); // GroupResolver 默认读 in-hub MemberGroup 闭包(ADR-0028), // HUB_GROUP_SERVICE_URL 配置后切 HTTP(C2); // 导出适配器当前为 manifest stub(OPEN-6,真导出工具到位后替换)。 @@ -145,7 +149,7 @@ export async function registerDatabaseRoutes( return; } const storageRoot = process.env["HUB_FILELIB_STORAGE_ROOT"] ?? path.resolve(".filelib-repos"); - const versionStore = createInMemoryVersionStore(path.join(storageRoot, ".version-store.json")); + const versionStore = createGitVersionStore(); const groupServiceUrl = process.env["HUB_GROUP_SERVICE_URL"]; const filelibDeps: FileLibRouteDeps = { prisma: config.prisma, @@ -157,6 +161,7 @@ export async function registerDatabaseRoutes( : createHttpGroupResolver({ baseUrl: groupServiceUrl }), versionStore, exportAdapters: [createManifestStubAdapter(versionStore)], + maxFileBytes: resolveMaxFileBytes(), }; await registerFileLibRoutes(app, filelibDeps); await registerFileRoutes(app, filelibDeps); diff --git a/hub/src/database/routes/fileRoutes.ts b/hub/src/database/routes/fileRoutes.ts index 1a049a8..8660594 100644 --- a/hub/src/database/routes/fileRoutes.ts +++ b/hub/src/database/routes/fileRoutes.ts @@ -39,6 +39,7 @@ export async function registerFileRoutes( organizationId: deps.organizationId, groupResolver: deps.groupResolver, versionStore: deps.versionStore, + maxFileBytes: deps.maxFileBytes, }; const exportDeps = { ...fileDeps, adapters: deps.exportAdapters }; diff --git a/hub/test/integration/filelib-routes.test.ts b/hub/test/integration/filelib-routes.test.ts index 39ca9a1..b813cf9 100644 --- a/hub/test/integration/filelib-routes.test.ts +++ b/hub/test/integration/filelib-routes.test.ts @@ -54,6 +54,7 @@ beforeEach(async () => { groupResolver: createStaticGroupResolver({}), versionStore, exportAdapters: [createManifestStubAdapter(versionStore)], + maxFileBytes: 10 * 1024 * 1024, }; app = Fastify({ logger: false }); await app.register(fastifyCookie); diff --git a/hub/test/unit/filelib-file-limits.test.ts b/hub/test/unit/filelib-file-limits.test.ts new file mode 100644 index 0000000..c45490f --- /dev/null +++ b/hub/test/unit/filelib-file-limits.test.ts @@ -0,0 +1,73 @@ +/** + * 单文件上限的环境变量解析 + commit message 默认文案。 + * + * 上限:`HUB_FILELIB_MAX_FILE_BYTES` 覆盖,缺省 10MiB;非法值按缺省 —— + * 配置写错不该让上传静默变成 0 上限(那会把每次上传都拒掉)。 + * 文案:`【用户名】修改了【路径】`,用户名取 displayName、缺失回退 userId。 + */ +import { describe, expect, it } from "vitest"; +import { + FILE_CONTENT_MAX_BYTES_DEFAULT, + defaultCommitMessage, + resolveMaxFileBytes, +} from "../../src/database/filelib/fileService.js"; + +describe("resolveMaxFileBytes", () => { + it("缺省 10MiB", () => { + expect(FILE_CONTENT_MAX_BYTES_DEFAULT).toBe(10 * 1024 * 1024); + expect(resolveMaxFileBytes(undefined)).toBe(FILE_CONTENT_MAX_BYTES_DEFAULT); + expect(resolveMaxFileBytes("")).toBe(FILE_CONTENT_MAX_BYTES_DEFAULT); + expect(resolveMaxFileBytes(" ")).toBe(FILE_CONTENT_MAX_BYTES_DEFAULT); + }); + + it("合法正整数生效(可上调也可下调)", () => { + expect(resolveMaxFileBytes("1048576")).toBe(1024 * 1024); + expect(resolveMaxFileBytes(" 52428800 ")).toBe(50 * 1024 * 1024); + expect(resolveMaxFileBytes("1")).toBe(1); + }); + + it("非法值一律回退缺省,不产生 0 或负数上限", () => { + for (const bad of ["0", "-1", "abc", "1.5", "NaN", "Infinity", "1e999", "10MB"]) { + expect(resolveMaxFileBytes(bad)).toBe(FILE_CONTENT_MAX_BYTES_DEFAULT); + } + }); +}); + +describe("defaultCommitMessage", () => { + it("【用户名】修改了【路径】,用户名取 displayName", () => { + expect(defaultCommitMessage({ userId: "u1", isWebsiteAdmin: false, displayName: "张老师" }, "讲义/第一课.md")) + .toBe("【张老师】修改了【讲义/第一课.md】"); + }); + + it("displayName 缺失/空白时回退 userId", () => { + expect(defaultCommitMessage({ userId: "u_alice", isWebsiteAdmin: false }, "a.md")) + .toBe("【u_alice】修改了【a.md】"); + expect(defaultCommitMessage({ userId: "u_alice", isWebsiteAdmin: false, displayName: " " }, "a.md")) + .toBe("【u_alice】修改了【a.md】"); + }); +}); + +/** + * message 优先级的纯函数复刻(commitFile 里的那三行): + * 非空则用调用方的,否则回退默认文案。空串必须算“没传” —— + * `git commit -m ""` 会以 empty commit message 失败。 + */ +function resolveMessage(input: string | undefined, fallback: string): string { + const trimmed = input?.trim(); + return trimmed === undefined || trimmed === "" ? fallback : trimmed; +} + +describe("commit message 优先级", () => { + const fallback = "【张老师】修改了【a.md】"; + + it("手填了就用手填的", () => { + expect(resolveMessage("补上第三课的课件", fallback)).toBe("补上第三课的课件"); + expect(resolveMessage(" 前后有空格 ", fallback)).toBe("前后有空格"); + }); + + it("未传/空/约空白一律回退默认文案", () => { + for (const raw of [undefined, "", " ", "\n\t"]) { + expect(resolveMessage(raw, fallback)).toBe(fallback); + } + }); +}); diff --git a/hub/test/unit/filelib-git-version-store.test.ts b/hub/test/unit/filelib-git-version-store.test.ts new file mode 100644 index 0000000..9ca8e3f --- /dev/null +++ b/hub/test/unit/filelib-git-version-store.test.ts @@ -0,0 +1,303 @@ +/** + * 真 git VersionStore 语义单测(ADR-0030)。 + * + * 与 filelib-version-store.test.ts(内存实现)逐条对齐同一份 C1 语义: + * S1/S2 冲突走返回值、S7 init 幂等、D16 文件级版本互不影响、S4 同仓库写串行。 + * 差别只在 VersionId 是 commit hash 而非计数器 —— 断言因此不写死字面量。 + * + * 每个用例一个真临时仓库,跑真 git;没有 mock。 + */ +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { mkdtemp, rm, readFile } from "node:fs/promises"; +import { existsSync } from "node:fs"; +import { execFileSync } from "node:child_process"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { createGitVersionStore } from "../../src/database/filelib/gitVersionStore.js"; +import { FileLibError } from "../../src/database/filelib/model.js"; + +const HEX40 = /^[0-9a-f]{40}$/; + +let root: string; +let dir: string; +const store = createGitVersionStore(); + +beforeEach(async () => { + root = await mkdtemp(path.join(tmpdir(), "filelib-git-")); + dir = path.join(root, "11111111-2222-3333-4444-555555555555"); +}); + +afterEach(async () => { + await rm(root, { recursive: true, force: true }); +}); + +/** 取 ok 版本号;conflict 直接让用例失败(比 as 断言更早暴露问题)。 */ +function okVersion(result: { status: string; version?: string }): string { + expect(result.status).toBe("ok"); + return result.version as string; +} + +describe("GitVersionStore · C1 语义(ADR-0030)", () => { + it("init 建目录 + git init;幂等(S7);未 init → repo_not_found", async () => { + await expect(store.head(dir, "a.md")).rejects.toThrowError(FileLibError); + await store.init(dir); + expect(existsSync(path.join(dir, ".git"))).toBe(true); + await store.init(dir); // 幂等 + const v1 = okVersion(await store.commit(dir, "a.md", { baseVersion: null, content: "hello" })); + await store.init(dir); // 已有内容后再 init 不清空 + expect(await store.head(dir, "a.md")).toBe(v1); + expect((await store.read(dir, "a.md")).toString()).toBe("hello"); + }); + + it("VersionId 是 40 位 commit hash,且真的落到磁盘工作区", async () => { + await store.init(dir); + const v = okVersion(await store.commit(dir, "docs/a.md", { baseVersion: null, content: "内容" })); + expect(v).toMatch(HEX40); + // 工作区里是一个真文件(不只是 git 对象)。 + expect(await readFile(path.join(dir, "docs/a.md"), "utf8")).toBe("内容"); + }); + + it("对已存在文件再次『新建』 → conflict(S2)", async () => { + await store.init(dir); + const v1 = okVersion(await store.commit(dir, "a.md", { baseVersion: null, content: "1" })); + expect(await store.commit(dir, "a.md", { baseVersion: null, content: "2" })).toEqual({ + status: "conflict", + currentVersion: v1, + }); + }); + + it("baseVersion 落后于当前 → conflict 并带 currentVersion(S1)", async () => { + await store.init(dir); + const v1 = okVersion(await store.commit(dir, "a.md", { baseVersion: null, content: "1" })); + const v2 = okVersion(await store.commit(dir, "a.md", { baseVersion: v1, content: "2" })); + expect(await store.commit(dir, "a.md", { baseVersion: v1, content: "3" })).toEqual({ + status: "conflict", + currentVersion: v2, + }); + }); + + it("旧版本仍可按 hash 读回", async () => { + await store.init(dir); + const v1 = okVersion(await store.commit(dir, "a.md", { baseVersion: null, content: "一" })); + await store.commit(dir, "a.md", { baseVersion: v1, content: "二" }); + expect((await store.read(dir, "a.md", v1)).toString()).toBe("一"); + expect((await store.read(dir, "a.md")).toString()).toBe("二"); + }); + + it("D16 文件级版本:a.md 的提交不推进 b.md 的 base", async () => { + await store.init(dir); + const a1 = okVersion(await store.commit(dir, "a.md", { baseVersion: null, content: "A1" })); + const b1 = okVersion(await store.commit(dir, "b.md", { baseVersion: null, content: "B1" })); + await store.commit(dir, "a.md", { baseVersion: a1, content: "A2" }); // 仓库 HEAD 前进 + // b.md 仍以自己的版本为 base —— 这正是 commit 级 hash 做文件级版本的关键。 + expect((await store.commit(dir, "b.md", { baseVersion: b1, content: "B2" })).status).toBe("ok"); + expect(await store.head(dir, "b.md")).not.toBe(b1); + }); + + it("二进制内容原样往返(NUL 字节不被破坏)", async () => { + await store.init(dir); + const bytes = Buffer.from([0x00, 0x01, 0xff, 0x00, 0x7f]); + await store.commit(dir, "blob.bin", { baseVersion: null, content: bytes }); + expect(Buffer.compare(await store.read(dir, "blob.bin"), bytes)).toBe(0); + }); + + it("remove:正确 base → ok;head 随后 404;stale/缺失 base → conflict/404", async () => { + await store.init(dir); + const v1 = okVersion(await store.commit(dir, "a.md", { baseVersion: null, content: "1" })); + const v2 = okVersion(await store.commit(dir, "a.md", { baseVersion: v1, content: "2" })); + expect(await store.remove(dir, "a.md", v1)).toEqual({ status: "conflict", currentVersion: v2 }); + const v3 = okVersion(await store.remove(dir, "a.md", v2)); + await expect(store.head(dir, "a.md")).rejects.toThrowError(FileLibError); + expect(existsSync(path.join(dir, "a.md"))).toBe(false); + // 删除是一个 commit:旧版本内容仍可取回(ADR-0030)。 + expect((await store.read(dir, "a.md", v1)).toString()).toBe("1"); + expect(v3).toMatch(HEX40); + await expect(store.remove(dir, "a.md", v3)).rejects.toThrowError(FileLibError); + }); + + it("history 新→旧,支持 limit,author 取 displayName", async () => { + await store.init(dir); + const v1 = okVersion( + await store.commit(dir, "docs/a.md", { + baseVersion: null, + content: "1", + message: "初版", + author: { userId: "u1", displayName: "张老师" }, + }), + ); + await store.commit(dir, "docs/a.md", { + baseVersion: v1, + content: "2", + message: "二版", + author: { userId: "u2", displayName: "李老师" }, + }); + await store.commit(dir, "other/b.md", { baseVersion: null, content: "x" }); + const h = await store.history(dir, "docs/a.md"); + expect(h.map((v) => v.message)).toEqual(["二版", "初版"]); + expect(h.map((v) => v.author)).toEqual(["李老师", "张老师"]); + expect(h[0]?.version).toMatch(HEX40); + expect((await store.history(dir, "docs/a.md", 1)).map((v) => v.message)).toEqual(["二版"]); + // other/b.md 的提交不出现在 docs/a.md 的历史里。 + expect(h).toHaveLength(2); + }); + + it("list 排除已删、按前缀过滤、空仓库返回空", async () => { + await store.init(dir); + expect(await store.list(dir)).toEqual([]); + await store.commit(dir, "docs/a.md", { baseVersion: null, content: "1" }); + const bv = okVersion(await store.commit(dir, "other/b.md", { baseVersion: null, content: "xy" })); + expect((await store.list(dir)).map((f) => f.path)).toEqual(["docs/a.md", "other/b.md"]); + expect((await store.list(dir, "docs/")).map((f) => f.path)).toEqual(["docs/a.md"]); + expect((await store.list(dir)).find((f) => f.path === "other/b.md")?.size).toBe(2); + await store.remove(dir, "other/b.md", bv); + expect((await store.list(dir)).map((f) => f.path)).toEqual(["docs/a.md"]); + }); + + it("S4 同仓库写串行:并发同 base 提交,恰好一成一冲突", async () => { + await store.init(dir); + const v1 = okVersion(await store.commit(dir, "a.md", { baseVersion: null, content: "1" })); + const [r1, r2] = await Promise.all([ + store.commit(dir, "a.md", { baseVersion: v1, content: "x" }), + store.commit(dir, "a.md", { baseVersion: v1, content: "y" }), + ]); + expect([r1.status, r2.status].sort()).toEqual(["conflict", "ok"]); + }); + + it("目录不存在 → repo_not_found,不该报 git_missing", async () => { + // 回归:spawn 的 ENOENT 有两种来源且报错一模一样 —— git 不在 PATH 上,或 cwd 不存在。 + // 早先把两者一律归为 git_missing,于是“项目目录没建过”会被误报成“git 没装”。 + // 内存 store 时代建的旧项目就是这个形态:DB 有 storageDir,磁盘上什么都没有。 + const missing = path.join(root, "never-created"); + for (const op of [ + () => store.head(missing, "a.md"), + () => store.list(missing), + () => store.read(missing, "a.md"), + () => store.commit(missing, "a.md", { baseVersion: null, content: "x" }), + () => store.remove(missing, "a.md", "0".repeat(40)), + () => store.history(missing, "a.md"), + ]) { + await expect(op()).rejects.toMatchObject({ code: "repo_not_found", statusCode: 404 }); + } + }); + + it("目录存在但不是 git 仓库 → repo_not_found", async () => { + const { mkdir } = await import("node:fs/promises"); + const plain = path.join(root, "plain-dir"); + await mkdir(plain, { recursive: true }); + await expect(store.head(plain, "a.md")).rejects.toMatchObject({ code: "repo_not_found" }); + }); + + it("仍然能从零开始:init 会把缺失的目录建出来", async () => { + const deep = path.join(root, "a", "b", "c-uuid"); + await store.init(deep); + expect((await store.commit(deep, "a.md", { baseVersion: null, content: "1" })).status).toBe("ok"); + }); + + it("提交身份:name=displayName,email=@filelib.paradigm-edu.net", async () => { + await store.init(dir); + await store.commit(dir, "a.md", { + baseVersion: null, + content: "1", + author: { userId: "u_alice", displayName: "张老师" }, + }); + const ident = execFileSync("git", ["-C", dir, "log", "-1", "--format=%an|%ae|%cn|%ce"]).toString().trim(); + expect(ident).toBe("张老师|u_alice@filelib.paradigm-edu.net|张老师|u_alice@filelib.paradigm-edu.net"); + }); + + it("displayName 缺失时 name 回退 userId,email 仍用 userId", async () => { + await store.init(dir); + await store.commit(dir, "a.md", { baseVersion: null, content: "1", author: { userId: "u_bob" } }); + expect(execFileSync("git", ["-C", dir, "log", "-1", "--format=%an|%ae"]).toString().trim()) + .toBe("u_bob|u_bob@filelib.paradigm-edu.net"); + }); + + it("displayName 里的 <>/换行不能破坏 git ident 行", async () => { + await store.init(dir); + await store.commit(dir, "a.md", { + baseVersion: null, + content: "1", + author: { userId: "u_x", displayName: "a \n换行" }, + }); + const out = execFileSync("git", ["-C", dir, "log", "-1", "--format=%an|%ae"]).toString().trim(); + const [name, email] = out.split("|"); + // 断言性质而不是具体空白(git 自己还会压缩 ident 里的空白)。 + expect(name).not.toContain("<"); + expect(name).not.toContain(">"); + expect(name).not.toContain("\n"); + expect(name).toContain("换行"); + expect(email).toBe("u_x@filelib.paradigm-edu.net"); + }); + + it("删除提交也带提交者身份", async () => { + await store.init(dir); + const v = okVersion(await store.commit(dir, "a.md", { baseVersion: null, content: "1" })); + okVersion(await store.remove(dir, "a.md", v, { userId: "u_del", displayName: "删除者" })); + expect(execFileSync("git", ["-C", dir, "log", "-1", "--format=%an|%ae"]).toString().trim()) + .toBe("删除者|u_del@filelib.paradigm-edu.net"); + }); + + it("storage root 在另一个 git 仓库内时,不得落到外层仓库上", async () => { + // 回归:早先用 `git rev-parse --git-dir` 判定“是否已 init”,而 git 会沿目录树 + // **向上**找 `.git`。本地开发的默认 storage root 就是 `hub/.filelib-repos`, + // 在本 repo 内 —— 于是未 init 的项目目录被误判为“已是仓库”,后续命令全部 + // 落到外层源码仓上(表现为 `git add` 报 “paths are ignored by .gitignore”)。 + const outer = path.join(root, "outer"); + const { mkdir, writeFile } = await import("node:fs/promises"); + await mkdir(outer, { recursive: true }); + execFileSync("git", ["-C", outer, "init", "--quiet"]); + await writeFile(path.join(outer, ".gitignore"), "repos/\n"); + const inner = path.join(outer, "repos", "proj-uuid"); + await mkdir(inner, { recursive: true }); + + // 未 init 的子目录:必须报 repo_not_found,而不是静默用外层仓库。 + await expect(store.head(inner, "a.md")).rejects.toMatchObject({ code: "repo_not_found" }); + + // init 后的写入必须进自己的仓库,外层仓库保持干净。 + await store.init(inner); + okVersion(await store.commit(inner, "a.md", { baseVersion: null, content: "1" })); + expect(existsSync(path.join(inner, ".git"))).toBe(true); + expect(execFileSync("git", ["-C", outer, "status", "--porcelain"]).toString().trim()).toBe("?? .gitignore"); + expect(execFileSync("git", ["-C", inner, "log", "--format=%s"]).toString().trim()).toBe("create a.md"); + }); + + it("显式 message 原样落到 commit;不传则用调用方给的默认值", async () => { + await store.init(dir); + okVersion(await store.commit(dir, "a.md", { baseVersion: null, content: "1", message: "手填的提交信息" })); + expect(execFileSync("git", ["-C", dir, "log", "-1", "--format=%s"]).toString().trim()).toBe("手填的提交信息"); + // 以 `-` 开头的 message 不得被当成选项(execFile 数组形式 + `-m` 传入)。 + okVersion(await store.commit(dir, "b.md", { baseVersion: null, content: "1", message: "--force 看着像选项" })); + expect(execFileSync("git", ["-C", dir, "log", "-1", "--format=%s"]).toString().trim()).toBe("--force 看着像选项"); + }); + + it("diff 是真 unified diff,含增删行", async () => { + await store.init(dir); + const v1 = okVersion(await store.commit(dir, "a.md", { baseVersion: null, content: "keep\nold\n" })); + const v2 = okVersion(await store.commit(dir, "a.md", { baseVersion: v1, content: "keep\nnew\n" })); + const d = await store.diff(dir, "a.md", v1, v2); + expect(d).toContain("-old"); + expect(d).toContain("+new"); + expect(d).toContain(" keep"); + }); + + it("不存在的 version → version_not_found;非法路径 → invalid_path", async () => { + await store.init(dir); + await store.commit(dir, "a.md", { baseVersion: null, content: "1" }); + const bogus = "0".repeat(40); + await expect(store.read(dir, "a.md", bogus)).rejects.toMatchObject({ code: "version_not_found" }); + for (const bad of ["../escape.md", "a/../../b.md", ".git/config", "/abs.md", "a\\b.md"]) { + await expect(store.read(dir, bad)).rejects.toMatchObject({ code: "invalid_path" }); + } + }); + + it("仓库内的 hooks 不被执行(ADR-0030 加固)", async () => { + await store.init(dir); + // 装一个会失败的 pre-commit hook:若 hooks 生效,下面的 commit 就会失败。 + const { mkdir, writeFile, chmod } = await import("node:fs/promises"); + const hookDir = path.join(dir, ".git", "hooks"); + await mkdir(hookDir, { recursive: true }); + const hook = path.join(hookDir, "pre-commit"); + await writeFile(hook, "#!/bin/sh\nexit 1\n"); + await chmod(hook, 0o755); + expect((await store.commit(dir, "a.md", { baseVersion: null, content: "1" })).status).toBe("ok"); + }); +}); diff --git a/hub/test/unit/filelib-version-store.test.ts b/hub/test/unit/filelib-version-store.test.ts index 977f4c1..272fc01 100644 --- a/hub/test/unit/filelib-version-store.test.ts +++ b/hub/test/unit/filelib-version-store.test.ts @@ -23,7 +23,11 @@ describe("InMemoryVersionStore · C1 语义", () => { it("新建(baseVersion null)→ ok;head/read 命中", async () => { const store = createInMemoryVersionStore(); await store.init(DIR); - const r = await store.commit(DIR, "a.md", { baseVersion: null, content: "v1 内容", author: "u1" }); + const r = await store.commit(DIR, "a.md", { + baseVersion: null, + content: "v1 内容", + author: { userId: "u1" }, + }); expect(r).toEqual({ status: "ok", version: "v1" }); expect((await store.read(DIR, "a.md")).toString()).toBe("v1 内容"); });