Commit Graph

94 Commits

Author SHA1 Message Date
e878b46701 Merge remote-tracking branch 'educraft/main' into merge/educraft-cph
# Conflicts:
#	.gitignore
#	hub/.env.example
#	hub/deploy/deploy_fleet_release.sh
#	hub/deploy/deploy_platform.sh
#	hub/test/integration/helpers.ts
2026-08-06 00:49:02 +08:00
dc271ff9f3 Merge branch 'maoyuanyang-main'
# Conflicts:
#	hub/filelib-web/src/lib/GrantsPanel.svelte
#	hub/filelib-web/src/lib/OverviewPanel.svelte
#	hub/filelib-web/src/lib/types.ts
#	hub/src/database/filelib/grantService.ts
2026-08-03 15:35:44 +08:00
ymy 39a2be6347 feat(filelib): 项目级授权恒生效,移除独立权限开关(ADR-0030)
- permission.ts: effectiveRole 删除 D11 冻结分支,输入不再含开关字段
- treeService: 停止读 FileLibProjectSettings;建项目不再写默认行
- grantService/routes: 删 setIndependentPermission 与 PUT 路由;节点详情 DTO 去掉 independentPermission
- filelib-web: 概览 tab 移除开关;NodeDetail 类型同步
- 测试: 单测/集成改为断言恒生效语义;ADR-0030 废除契约 D11/P5
- FileLibProjectSettings 表保留(存量行忽略,不再读写),审计词表保留历史读取
2026-07-30 22:27:45 +08:00
hongjr03 295f07d111 fix(hub): enable SDK auto-compact for resumed sessions
Sessions resume across runs (ADR-0017). Without auto-compact the SDK jsonl
grows unboundedly — a 7-day / 26-run session hit 31 MB / 1995 lines, making
every API call resend the full history and inflating a trivial "change a
title" task to 22 minutes. Enable autoCompactEnabled in the SDK settings so
the SDK compacts automatically when the context window fills.
2026-07-30 17:12:31 +08:00
hongjr03 e0e25ca4c5 fix(hub): expect always-on todo_write in pbank tool mapping 2026-07-30 12:51:48 +08:00
hongjr03 88386fb943 fix(hub): download Feishu resources via bot-owned lark-cli
Agent tool downloads and trigger attachment staging both used the SDK
messageResource path, which fails closed for multi-MB teacher files and
did not share the bot-identity transport contract. Route every download
through Hub-owned createFeishuBotCli (secret via stdin, disposable HOME,
HUB_FEISHU_CLI_BIN), keep workspace containment on write, and inject the
adapter in trigger tests.
2026-07-30 11:27:53 +08:00
hongjr03 97a99cd381 fix(hub): raise Docmind OSS upload timeout past httpx 3s default
SubmitDocParserJobAdvance uploads PDFs to Aliyun OSS via tea/httpx, which
defaults readTimeout to 3000ms when RuntimeOptions is empty. Multi-MB
teacher PDFs on para silo failed with ReadTimeout(3000) before the job
could start. Set connectTimeout=15s and readTimeout=5m.
2026-07-30 11:16:56 +08:00
ce5fbfb9a6 fix(filelib): 上传上限抬到 50MiB,并把它与 body limit 的串联写清
7.8MB 文件上传报 413:那是 Fastify 在 body 解析阶段拒的,不是
HUB_FILELIB_MAX_FILE_BYTES。上传把内容放在 JSON body 里、二进制过 base64
体积涨 4/3,所以有效上限是 min(该值, HUB_HTTP_BODY_LIMIT_BYTES × 3/4)。
原先 body limit 是 1MiB,10MiB 的文件上限根本不可达。

.env.example:body limit 1MiB → 70MiB,新增 HUB_FILELIB_MAX_FILE_BYTES=50MiB。
注意 body limit 同时是 ADR-0022 requestBodySize 维度的平台 ceiling,抬高它
对所有端点生效。

resolveMaxFileBytes 拆成 parseMaxFileBytes(纯解析)+ resolveMaxFileBytes(读
env):原先带默认参数,显式传 undefined 会回落到读 env,"没传值"与"读环境变量"
分不开,vitest 加载 .env 后测试会读到真实配置。
2026-07-27 16:14:50 +08:00
82241afb56 feat(filelib)!: VersionStore 改为真 git,一项目一仓库
内存 store 换成 gitVersionStore:init 建目录并 git init,VersionId 是
commit hash,某文件的版本取 `git log -1 -- <path>`(D16 文件级版本不因
别的文件提交而失效)。删除也是一个 commit,旧版本仍可读。决策见 ADR-0030。

git 用 execFile 调系统二进制,不引依赖。每次调用钉死 --git-dir/--work-tree
并禁 hooks、隔离全局 gitconfig:项目仓库是老师上传的数据,而 storage root
默认就在本 repo 内,不钉死会让命令落到外层仓库上。

同时:
- 单文件上限改为 HUB_FILELIB_MAX_FILE_BYTES(缺省 10MiB),前端从
  /database/config 读,不再两处硬编码
- commit 身份 name=displayName、email=<userId>@filelib.paradigm-edu.net;
  message 缺省为「【用户名】修改了【路径】」,调用方显式传则优先
- 上传改走弹窗,路径与 commit 信息可手填(原先 prompt 只能填路径)

BREAKING CHANGE: VersionId 由计数器(v1/v2)变为 commit hash;
CommitRequest.author 由字符串变为 { userId, displayName? }。
旧 .version-store.json 不迁移,此前建的项目报 repo_not_found。
2026-07-27 15:55:01 +08:00
hongjr03 34d5d5e88e fix(hub): do not crash Hub on missing DocMind input files
createReadStream emits async ENOENT without a listener, which became an
unhandled 'error' event and exited the silo process. Teachers then saw the
startup "process restart" notice. Wait for stream open and convert missing
files into DocmindClientError instead.
2026-07-27 12:25:04 +08:00
50ddf32cc2 fix(auth): POST /auth/logout 接受任意 Content-Type
该端点不读 body,但调用方(curl -d、Postman、部分 HTTP 客户端)常给空 POST
自动带上 Content-Type。Fastify 默认只有 JSON parser,遇到别的媒体类型在解析
阶段就以 415 FST_ERR_CTP_INVALID_MEDIA_TYPE 拒掉,进不到 handler。

修法是给它一个丢弃 body 的 catch-all parser,**封装在自己的 register 作用域
内**。不能加到外层实例上:admin plugin 没有 fastify-plugin 封装,那样会让全站
每个 POST/PUT/PATCH 都接受 form-urlencoded。而 form-urlencoded 是跨站 HTML
form 唯一能发出的媒体类型(application/json 会触发 CORS preflight),"只认
JSON"本身是一层 CSRF 纵深防御 —— 当前 sameSite=lax 还挡着,但不该为这个端点
全局放掉。

两处细节:
- "*" 只兜没有专属 parser 的媒体类型。内建 JSON parser 优先级更高,空 body 会
  被它判成 FST_ERR_CTP_EMPTY_JSON_BODY(400),故在本作用域内一并覆盖。
- 用 parseAs:"string" 让 Fastify 读完流(否则连接不释放),而非手写
  payload.resume()。

前端未改 —— 原本不带 Content-Type 的发法一直是 204,是正确的。

测试 5 个 case,最后一个是护栏:断言作用域外的 POST 路由发 form-encoded 仍为
415,防止以后有人把 parser 提到外层。
2026-07-26 22:39:46 +08:00
6990082247 build(deploy): 部署与限流配置切换到 filelib-web,并加共存回归测试
三处引用旧工程名/旧资源路径的地方一并更新,它们必须同时改 —— 少改一处
就是静默故障,而不是构建期报错:

1. 部署脚本(deploy_platform.sh / deploy_fleet_release.sh):npm ci 的
   prefix、rsync 排除项、构建产物存在性检查从 database-admin 换成
   filelib-web。最后一项是真门禁:static.ts 缺产物时只 warn 不注册路由,
   漏改会让 /app 与 /database 静默 404 —— 恰是 database-admin 长期处于
   禁用状态的原因。

2. silo 限流豁免:资源路径随 appDir 改名而变(/database/_app/* 已不存在,
   现为 /_filelib/*);/app/* 此前不在豁免列表,它现在也是 SPA 外壳,
   客户端路由无法预先枚举。
   注:/database/* 是整体豁免,filelib 的 JSON API 也绕过限流预算。这是
   迁移前就有的行为,原样保留,但覆盖面因多了 /app/* 而变宽。

3. 回归测试:把 registerStaticSpa 与 registerDatabaseSpa 挂到同一个
   Fastify 实例,断言 ready() 不因重复路由抛错 —— appDir 若用回默认的
   _app,这里会红(ADR-0029 的承重约束)。另断言 /app 与
   /database/dashboard/users 返回同一份字节(SPA 回退不读请求)、body 含
   /_filelib/。构建产物缺失时不 skip 而是直接失败:那说明该先跑
   filelib-web 的 build,不是测试不适用。
2026-07-26 20:23:15 +08:00
hongjr03 2699ff3679 fix(hub): include Edit in default single-agent tools
Removing the claude_code preset dropped Edit. Unrestricted roles need it for
in-place file edits; write_file also grants Edit for restricted roles.
2026-07-25 14:18:57 +08:00
hongjr03 755704e2ae fix(hub): stop unrestricted roles from loading multi-agent tools
The claude_code preset exposed Agent/SendMessage/Task. Background agents
abort with reason "background", which the SDK maps to Bash
toolDenialKind "cancelled" ("user doesn't want this action") and freezes
command execution mid-run.
2026-07-25 13:40:27 +08:00
ymy 4021e58d5d feat(database): init database folder frontend and permission 2026-07-23 23:41:11 +08:00
hongjr03 74f5c4a02e fix(hub): render Feishu checklist from TaskCreate/TaskUpdate
Headless Claud agents expose TaskCreate/TaskUpdate rather than TodoWrite.
Fold those tool events into the live card checklist, pass real tool names
and inputs through tool-result, and hide Task* noise once the panel is up.
2026-07-23 23:21:34 +08:00
hongjr03 e3b463d390 fix(hub): ship checklist via cph_hub todo_write MCP tool
Native Claude TodoWrite is not registered in headless agent mode even with
--tools default. Add mcp__cph_hub__todo_write (always enabled), mirror the
TodoWrite schema, instruct multi-step runs to use it, and keep the Feishu
progress panel parsing both native and hub tool names.
2026-07-23 23:08:42 +08:00
hongjr03 bb426dfaf5 fix(hub): expose TodoWrite via SDK default toolset
Unrestricted roles were still passed an explicit --tools name list. The
native Claude binary only reliably registers bundled tools like TodoWrite
on --tools default. Treat role tools JSON null as unrestricted, use the
claude_code preset (→ default) in that case, and keep TodoWrite on allowedTools.
2026-07-23 22:56:26 +08:00
hongjr03 2f79b7743f feat(hub): live TodoWrite checklist on Feishu agent cards
Always expose Claude Agent SDK TodoWrite (todoFeatureEnabled) so multi-step
runs can plan in the open. Parse TodoWrite payloads into a progress panel on
the streaming Feishu card (completed/in_progress/pending) and filter raw
TodoWrite noise out of the tool-use list.
2026-07-23 22:42:02 +08:00
hongjr03 8a81c60ea5 fix(hub): extract PBank zips in-process without host unzip
pbank materialize previously shelled out to `unzip` and soft-failed when
the binary was missing, so agents only saw titles. Read zip entries with
Node zlib (store/deflate) and write under workspace .pbank-sources.
2026-07-23 21:05:15 +08:00
hongjr03 54837717fd feat(hub): built-in PBank 题库 capability + role tools (v0.0.42)
Register pbank as an ADR-0027 external capability with org-scoped
username/password envelopes, readiness via /login, and in-process
cph_hub MCP tools (search/get/get_many) that materialize sources under
the run workspace. Extend the capability secret payload for docmind vs
pbank kinds, admin capabilities UI, role tool umbrella `pbank`, and the
pbank-problem-report skill. Credentials never reach the Agent process.
2026-07-23 20:13:00 +08:00
hongjr03 36660f72d6 fix(hub): enable tenant Typst package resolution 2026-07-22 17:47:56 +08:00
hongjr03 6f7497bce8 fix(hub): stamp CheckMark/CrossMark when agent run finishes (v0.0.41)
After removing the Typing reaction, add CheckMark on success or CrossMark
on failure so teachers can see completion on the source message without
opening the card.
2026-07-21 06:36:09 +00:00
hongjr03 db49a0d23d feat(hub): concurrent multi-PDF convert_pdf_to_md + readable skills (v0.0.40)
Teachers convert many PDFs in one tool call with bounded Docmind concurrency.
Each item keeps its own output_dir/document.md and UsageFact; failures are
per-file. Mirror role skills to .cph/runtime-skills and CPH_RUNTIME_SKILLS_DIR
so agents can Read SKILL.md instead of dead .claude/sandbox stubs.
2026-07-21 05:55:55 +00:00
hongjr03 3fbc4b81c2 fix(hub): forward host HTTP(S)_PROXY into agent sandbox (v0.0.39)
Host egress requires the local forward proxy; sandbox env previously
omitted PROXY vars so Bash/curl timed out on public image URLs. Pass
HTTP(S)/ALL/NO_PROXY (+ lowercase) and NODE_USE_ENV_PROXY from the
trusted service environment into the agent subprocess.
2026-07-20 13:46:07 +00:00
hongjr03 6cefb2a938 feat(hub): raise agent turns/time limits and notify teachers on failure
Defaults and silo env go to 150 turns / 1800s wall clock. Run completion
appends a clear Feishu notice for max-turns, timeout, and other failures
(partial answer kept). Startup process-restart kills notify the bound chat.
Release v0.0.38.
2026-07-20 12:07:45 +00:00
hongjr03 93f3f2424c fix(hub): strip card markdown images + prefer inline ![] over send_file
Feishu interactive markdown rejects ![](http...) without image_key
(error 230099 empty/missing imagekey). Always mask residual image md in
card builders; skip img tags with empty keys; skip inline-code examples;
fetch remote images with a browser UA and without env HTTP_PROXY.
Steer the agent: use ![alt](workspace-path) for 图文, send_file only for
downloadable attachments.

Release v0.0.37.
2026-07-20 11:38:55 +00:00
hongjr03 e21096c642 feat(hub): embed agent images via Feishu upload + release v0.0.36
Materialize markdown image refs on agent finish: fetch/read bytes, upload
im.v1.image, and render native card img elements so remote image URLs no
longer trip Feishu content-security. Stream masks image URLs mid-run;
card failure falls back to plain text plus standalone image messages.

Docs: clarify im:resource covers outbound Agent image send.
2026-07-20 10:40:03 +00:00
hongjr03 7f09fb1f13 feat(hub): drop redundant /admin/org/:slug path + release v0.0.35 (#11)
Silo hostname already carries tenancy. Admin SPA routes become /admin/..., legacy bookmarks redirect, login lands on /admin.

Co-authored-by: Hong Jiarong <me@jrhim.com>
Co-committed-by: Hong Jiarong <me@jrhim.com>
2026-07-19 01:36:10 +08:00
hongjr03 aaa098bb8b feat(hub): usage fact ledger for run-scoped cost attribution (ADR-0026) (#4)
Co-authored-by: Hong Jiarong <me@jrhim.com>
Co-committed-by: Hong Jiarong <me@jrhim.com>
2026-07-18 14:46:41 +08:00
hongjr03 97f7972cc5 chore(hub): remove markdown_to_pdf tool (#3)
Drop markdown_to_pdf MCP surface, implementation, tests, and md-to-pdf dependency.

Roles that still list markdown_to_pdf must be cleaned before startup.

Co-authored-by: Hong Jiarong <me@jrhim.com>
Co-committed-by: Hong Jiarong <me@jrhim.com>
2026-07-18 13:57:27 +08:00
hongjr03 4e01c18cac feat(hub): add markdown_to_pdf tool and default web tools
Teachers need ad-hoc Markdown → PDF. Ship an MCP tool powered by
md-to-pdf (Marked + headless Chrome) so remote images/CSS work, with
workspace-scoped basedir, front-matter stripped so untrusted markdown
cannot override dest/basedir/launch options, and MathJax for $/$ math.

Also include WebFetch and WebSearch in the unrestricted role tool
surface by default. Deploy skips Puppeteer's browser download and
expects a host Chrome/Chromium (PUPPETEER_EXECUTABLE_PATH / CHROME_PATH).
2026-07-18 13:38:07 +08:00
hongjr03 35251986af feat(hub): derive admin model picker from org provider connection via OpenRouter API
The admin role model picker was hardcoded to the env-default model registry
(createDefaultModelRegistry), which only ever returned a single Sonnet model.
Roles could not select any other model regardless of what the org's provider
connection supported.

Replace the env-only model list with a ProviderModelCatalog that:
- Resolves the org's ACTIVE provider connection credential (BYOK or
  platform-managed, encrypted via ADR-0024 envelope)
- Calls OpenRouter GET /v1/models?supported_parameters=tools to list
  tool-capable models available to that org
- Caches results in-memory with a 5-minute TTL per organization
- Falls back to the env-default registry when no ACTIVE provider exists

The runtime modelRegistry no longer validates role.defaultModel against the
env model list — the admin already validated by selection from the provider
catalog. The env list remains as the fallback for roles with null defaultModel.

The admin roles page loads models independently (non-blocking) so roles
remain editable even if the provider API is slow or unreachable.
2026-07-16 01:24:00 +08:00
hongjr03 79f72ecca8 feat(admin): web-based skill management with file editor
Add full skill lifecycle to the org-admin web surface: create, read,
edit, disable. Skills are directories (SKILL.md manifest + supporting
files), content-addressed by SHA-256 in an immutable store.

Backend:
- skillStore: extract commitSkillContent (shared populate→inspect→
  dedup→atomic rename); add importSkillFromFiles (in-memory file list
  ingestion) and readSkillFiles (read stored version back as UTF-8)
- configuration: add installSkillFromFiles, readSkillFiles, disableSkill
  (soft-delete + archive bound role sessions), updateSkillDescription
  (label-only, no archival); refactor installSkill to share
  commitInstalledSkill
- agentConfigRoutes: wire skillStoreRoot; add GET
  /agent-skills/:name/files, PUT /agent-skills/:name (create/replace),
  PATCH /agent-skills/:name (description/disable)
- orgRoutes: pass readSkillStoreRoot() to agent config routes

Frontend:
- api.ts: agentSkillFiles, installAgentSkill, patchAgentSkill methods
- SkillEditor.svelte: file tree + text editor + version/description form
- skills/+page.svelte: skill list, create form (generates SKILL.md
  template), per-skill editor
- layout: add 技能 nav item

ADR-0018: update Decision to reflect web surface joining host-console
CLI in the shared content-addressed ingestion pipeline.

Spec (AgentRole.lean): unchanged — storage mechanism is OPEN, web
installation is one implementation of it.
2026-07-16 01:11:49 +08:00
hongjr03 ae5f78f036 chore: release v0.0.28
Exempt SPA static assets and admin HTML shell from silo HTTP rate limit so
page loads no longer exhaust HUB_HTTP_REQUESTS_PER_MINUTE.
2026-07-15 22:33:07 +08:00
hongjr03 69837bd50c feat: redesign Feishu project console 2026-07-13 16:52:45 +08:00
hongjr03 82f57317df feat: archive bindings on Feishu lifecycle events 2026-07-13 15:53:28 +08:00
hongjr03 53d372e29b fix: report untracked legacy symlinks 2026-07-11 23:37:25 +08:00
hongjr03 530fcdd2b7 feat: migrate legacy projects through binding search 2026-07-11 23:33:23 +08:00
hongjr03 5b55cf18a8 Revert "fix: accept SDK provider capability headers"
This reverts commit e7ad5580ec.
2026-07-11 15:06:48 +08:00
hongjr03 b0d691d53f Revert "fix: pass provider capability as API key"
This reverts commit f065f9f978.
2026-07-11 15:06:48 +08:00
hongjr03 1f48c5b707 Revert "fix: provide capability for both SDK auth modes"
This reverts commit ebf870249f.
2026-07-11 15:06:48 +08:00
hongjr03 12a2f3117f Revert "fix: preserve run provider capability"
This reverts commit 63c86322de.
2026-07-11 15:06:48 +08:00
hongjr03 2ee84d9543 Revert "fix: carry provider capability in dedicated header"
This reverts commit 3087132083.
2026-07-11 15:06:47 +08:00
hongjr03 3087132083 fix: carry provider capability in dedicated header 2026-07-11 15:02:51 +08:00
hongjr03 63c86322de fix: preserve run provider capability 2026-07-11 15:00:34 +08:00
hongjr03 ebf870249f fix: provide capability for both SDK auth modes 2026-07-11 14:59:00 +08:00
hongjr03 f065f9f978 fix: pass provider capability as API key 2026-07-11 14:57:22 +08:00
hongjr03 e7ad5580ec fix: accept SDK provider capability headers 2026-07-11 14:55:29 +08:00
hongjr03 3bf643ff4d fix: guide Feishu users through onboarding 2026-07-11 13:52:19 +08:00