forked from bai/curriculum-project-hub
docs: resolve product surface audit
This commit is contained in:
@@ -0,0 +1,259 @@
|
||||
# Accepted SaaS product-surface completeness audit
|
||||
|
||||
Audit date: 2026-07-10
|
||||
|
||||
## Verdict
|
||||
|
||||
The current Hub is **not a usable or production-complete SaaS product surface**
|
||||
for the accepted destination. It has a worthwhile Organization-scoped backend
|
||||
foundation and a well-tested single-Feishu-application Agent Run slice, but the
|
||||
running product still has one process-global Feishu app/bot and one global model
|
||||
provider credential. It has no platform-administrator control plane, no
|
||||
Organization connection/secret plane, and no Organization-scoped Provider
|
||||
Credential Mode. Its only HTML product page is a login form; successful OAuth
|
||||
redirects an administrator to an unmounted `/admin/org/:orgSlug` route.
|
||||
|
||||
ADR-0022's required production safety surface is also almost wholly absent:
|
||||
there is no durable Organization-fair admission, layered platform/Organization
|
||||
limits, hard file/storage/run/process budgets, usage-alert control plane, or
|
||||
audited workload brake. These are release blockers, not optional polish.
|
||||
|
||||
The strongest code should be retained rather than rewritten: all current
|
||||
`/api/org/:orgSlug/*` routes reauthorize an active OWNER/ADMIN membership;
|
||||
Folder/Project, member, Team, Team-to-Project access, session, and usage services
|
||||
are Organization-scoped; and the shared project-onboarding services correctly
|
||||
create Project/chat/grant relationships for the single-app case. The shortest
|
||||
route is to add explicit Organization connection identity and the two private
|
||||
admin control planes around those seams.
|
||||
|
||||
The claim-by-claim source inventory, including route, schema and test locations,
|
||||
is in [accepted product-surface code inventory](product-surface-code-inventory.md).
|
||||
Official Feishu identifier, OAuth, event and long-connection semantics are in
|
||||
[Feishu product-surface primary sources](product-surface-feishu-primary-sources.md).
|
||||
|
||||
## Accepted boundary
|
||||
|
||||
The audit treats these as required because they are already accepted by the
|
||||
repository's `CONTEXT.md`, ADR-0020, ADR-0021, ADR-0022, and Lean contract:
|
||||
|
||||
- a Platform Administrator is separate from Organization membership and uses a
|
||||
private `/admin/platform` area to create and operate Organizations;
|
||||
- an Organization OWNER/ADMIN uses a private `/admin/org/:orgSlug` area to
|
||||
manage members, projects, Teams, connections, settings, sessions and usage;
|
||||
- every Organization authenticates and exchanges bot/directory traffic through
|
||||
its own customer-owned Feishu Application;
|
||||
- every Organization chooses BYOK or an Organization-distinct platform-managed
|
||||
Provider Connection, with different administrators owning the respective
|
||||
plaintext inputs;
|
||||
- ordinary members can create/bind a Project from an unbound Feishu group only
|
||||
under the Organization policy, while all Project actions retain Project-level
|
||||
authorization;
|
||||
- production has mandatory platform ceilings, optional lower Organization
|
||||
limits, durable fair admission, bounded work and audited emergency brakes.
|
||||
|
||||
The audit does not add deferred self-service signup, payment collection,
|
||||
folder ACL inheritance, commercial billing, or a live editor for platform
|
||||
capacity ceilings. Guided Feishu app setup and manually initiated Organization
|
||||
creation by an authenticated Platform Administrator remain valid for the pilot.
|
||||
|
||||
## Journey result
|
||||
|
||||
| Accepted journey | Current product entry | Result | Owning frontier |
|
||||
| --- | --- | --- | --- |
|
||||
| Platform Administrator signs in | No platform login/session/guard; only a stored `PlatformRoleAssignment` type | **Missing** | tickets 42, 45 |
|
||||
| Platform Administrator creates and operates Organizations | Direct Prisma calls in test fixtures only; no `/api/platform/*` or UI | **Test/database only** | ticket 45 |
|
||||
| Platform Administrator configures each customer Feishu app | One env `FEISHU_APP_ID/SECRET/BOT_OPEN_ID`; no connection or secret model | **Missing and contract-divergent** | tickets 43–45 |
|
||||
| Platform Administrator supplies a distinct platform-managed provider key/base URL | One process-global `ANTHROPIC_AUTH_TOKEN/BASE_URL`; resolver ignores Project scope | **Missing and contract-divergent** | tickets 43, 45 |
|
||||
| Platform Administrator applies and reverses `DRAIN`/`STOP_NOW` | No persistence, guard, API or UI | **Missing** | tickets 37, 45 |
|
||||
| Organization user logs in through that Organization's Feishu app | OAuth exists, but is built once from the global app and has no Organization/application issuer | **Single-app only** | tickets 43, 44 |
|
||||
| Logged-in Organization OWNER/ADMIN opens the management panel | OAuth redirects to `/admin/org/:slug`; no handler/static application serves it | **Stable 404** | ticket 46 |
|
||||
| Organization OWNER/ADMIN manages members, Teams, Projects/Folders and Team access | Guarded Organization JSON APIs and useful integration coverage exist | **Backend present; browser product absent** | ticket 46 |
|
||||
| Organization OWNER/ADMIN manages permission/model/role/run policy | Only `membersCanCreateProjects` is exposed; other stored/runtime settings have no control plane | **Partial** | ticket 46 |
|
||||
| Organization selects Provider Credential Mode and manages BYOK | No mode/connection model, API or UI | **Missing** | tickets 43, 46 |
|
||||
| Organization views sessions and usage | Guarded JSON APIs exist, but have no direct product-boundary tests or UI and cannot attribute Provider Connection | **Partial** | tickets 36, 46 |
|
||||
| Ordinary member creates or binds a Project in Feishu | Card/service path is present and well tested | **Works for the one-global-app case** | ticket 44 for SaaS routing |
|
||||
| Authorized member runs the agent through its Organization bot | Full single-app trigger path exists; provider/app resolution is global | **Works only as a single-app deployment** | tickets 43, 44 plus lifecycle tickets |
|
||||
| Accepted work queues fairly and durably across Organizations | Five-item process-memory Project queue | **Contract-divergent** | tickets 23, 32–35 |
|
||||
| Organization sets lower policy limits and receives soft alerts | No policy/alert persistence or product surface | **Missing** | tickets 32, 34–36, 46 |
|
||||
|
||||
## Executed defect evidence
|
||||
|
||||
The probes used the real Fastify plugin and test PostgreSQL database through a
|
||||
temporary Vitest integration file. The file was removed after capture and was
|
||||
run twice to separate deterministic behavior from a transient test failure.
|
||||
|
||||
### 1. The successful-login destination is not served
|
||||
|
||||
The probe seeded an active Organization ADMIN, minted the normal signed session
|
||||
cookie, and requested the exact destination selected by
|
||||
`resolvePostLoginRedirect`:
|
||||
|
||||
```text
|
||||
GET /admin/org/test-default
|
||||
expected_status=200
|
||||
actual_status=404
|
||||
```
|
||||
|
||||
Both runs returned 404. The root cause is structural: `registerAdminPlugin`
|
||||
mounts cookie, auth and Organization API routes only; its own comment says the
|
||||
static SPA is “later”. No other repository HTML/TSX/Vue/Svelte/CSS product
|
||||
application or deployment-level static handler exists. Existing OAuth tests
|
||||
assert only the redirect `Location`, so they stop immediately before the broken
|
||||
journey.
|
||||
|
||||
This is not an authorization defect in the JSON APIs. Those APIs correctly
|
||||
return 401 without a session and 403 to a MEMBER. It is a reachability defect:
|
||||
there is no management panel for either an authorized or unauthorized request
|
||||
to reach. Ticket 46 must guard both the page shell and its data, so an anonymous
|
||||
user may reach only the login entry and never management content.
|
||||
|
||||
### 2. Different Organizations start OAuth with the same application
|
||||
|
||||
The probe seeded two Organizations and requested the current OAuth entry with
|
||||
two different Organization return paths:
|
||||
|
||||
```text
|
||||
GET /auth/feishu?returnTo=/admin/org/test-default
|
||||
client_id=cli_process_global
|
||||
|
||||
GET /auth/feishu?returnTo=/admin/org/second
|
||||
client_id=cli_process_global
|
||||
```
|
||||
|
||||
Both runs produced the same result. The root cause is that
|
||||
`registerAuthRoutes` constructs one immutable OAuth configuration from plugin
|
||||
startup arguments. The `returnTo` path is signed only as a destination; it does
|
||||
not resolve or bind an Organization Feishu Application. `server.ts` obtains the
|
||||
same app credentials from one environment tuple and also uses them for the only
|
||||
WebSocket listener.
|
||||
|
||||
Changing only the redirect page cannot fix this. The composition root and every
|
||||
identifier-bearing record/action need an explicit Organization connection
|
||||
identity, backed by the encrypted resolver in ticket 43.
|
||||
|
||||
## Feishu identifier and transport consequence
|
||||
|
||||
Feishu's official identifier contract makes the current global identifier
|
||||
shape unsafe for customer-owned applications:
|
||||
|
||||
- `open_id` identifies a user inside one application and differs across apps;
|
||||
- `user_id` is stable inside one tenant but differs across tenants;
|
||||
- `union_id` can link apps from the same application service provider, but the
|
||||
service cannot assume independently customer-owned apps share that provider;
|
||||
- OAuth authorization identifies the app with `client_id`; the callback brings
|
||||
`code` and `state`, so state and token exchange must remain bound to the
|
||||
selected Organization/application;
|
||||
- v2 HTTP event/card headers expose application and tenant identity, while an
|
||||
encrypted outer request must be routed to the correct connection before its
|
||||
verification/decryption key is known; a long connection is itself created by
|
||||
one app credential pair, so the local client/listener identity is the issuer;
|
||||
- Feishu rejects cross-tenant chat/operator combinations and does not provide a
|
||||
contract that permits `chat_id` to be treated as a global SaaS key.
|
||||
|
||||
The current `User.feishuOpenId @unique`, session `{userId, feishuOpenId}`,
|
||||
`ProjectGroupBinding.chatId`, `FeishuEventReceipt.eventId`, principal IDs,
|
||||
sender caches, and batching keys omit this issuer namespace. An `open_id`
|
||||
collision across customer apps could merge unrelated external identities;
|
||||
different `open_id` values for one person across apps cannot be resolved as one
|
||||
global identity automatically. A globally looked-up `chatId` or event ID has
|
||||
the analogous ambiguity.
|
||||
|
||||
The implementation must therefore persist provider-local external identities
|
||||
under a Feishu Application Connection, for example conceptually
|
||||
`(connectionId, externalIdType, externalId)`, and carry that connection through
|
||||
OAuth state/session, ingress, deduplication, chat binding, authorization,
|
||||
onboarding/card actions, rate-limit keys, directory sync, outbound delivery and
|
||||
audit. Whether two external identities represent the same global `User` is a
|
||||
separate explicit linking decision; it must not be inferred from a provider ID
|
||||
outside its documented scope.
|
||||
|
||||
## Root causes rather than symptoms
|
||||
|
||||
### 1. Tenant data was added without tenant-scoping external connections
|
||||
|
||||
Organization ownership is correctly present on Projects, Teams, Folders and
|
||||
directory-connection metadata, but the process composition still comes from
|
||||
the former single-deployment shape. The app/bot/provider credentials and the
|
||||
identifiers they issue never crossed an Organization connection boundary.
|
||||
`RuntimeScope.projectId` exists but is discarded by the env resolver, making
|
||||
the missing seam especially explicit.
|
||||
|
||||
### 2. The control-plane work stopped at guarded Organization APIs
|
||||
|
||||
The Organization routes provide a credible backend slice, but there is no web
|
||||
application, platform route family, platform guard, Organization provisioning
|
||||
service, connection configuration, or browser journey test. The comment
|
||||
“until admin SPA lands” and redirect-only OAuth assertion document the precise
|
||||
unfinished handoff. A small catch-all page would only hide the gap; the fix must
|
||||
deliver the authenticated, authorized workflows and their safe states.
|
||||
|
||||
### 3. Production capacity semantics exist only in the contract
|
||||
|
||||
ADR-0022 intentionally declares the in-memory queue and global-only settings to
|
||||
be implementation divergences. There is no common admission identity that can
|
||||
join Organization, Feishu connection, Project, user and request, so fair
|
||||
scheduling, workload brakes, rate limits, alerts and control-plane operations
|
||||
cannot currently compose. The previously created capacity tickets own this
|
||||
implementation; the admin tickets must expose their safe operator/customer
|
||||
surfaces rather than invent parallel controls.
|
||||
|
||||
## What should remain
|
||||
|
||||
- Keep `requireOrgRole`'s per-request session, active Organization, active
|
||||
membership and OWNER/ADMIN checks; extend the same principle to page data and
|
||||
add a separate `requirePlatformAdmin` rather than an Organization override.
|
||||
- Keep the Organization-scoped explorer/member/Team/access/session/usage APIs
|
||||
as the initial backend for the org panel, while filling their missing tests,
|
||||
pagination/limits, audit and typed-error contracts through existing tickets.
|
||||
- Keep shared Project creation and chat-binding services so web and Feishu
|
||||
entries do not drift into two rule sets.
|
||||
- Keep the single-app Feishu handler's authorization, onboarding, run/session
|
||||
and dedup behavior, but call it from a connection-bound listener context and
|
||||
move accepted work/delivery into the durable lifecycle tickets.
|
||||
- Keep missing provider cost unknown rather than fabricating zero; add Provider
|
||||
Connection attribution before presenting SaaS usage claims.
|
||||
|
||||
## Newly clear execution frontier
|
||||
|
||||
The product audit creates four implementation/decision slices after the
|
||||
existing secret and browser-boundary design work:
|
||||
|
||||
1. [Decide the platform-administrator identity and audit boundary](../issues/42-decide-platform-admin-identity-audit.md).
|
||||
2. [Implement the Organization connection and secret plane](../issues/43-implement-org-connection-secret-plane.md).
|
||||
3. [Route Feishu identity and traffic by Organization connection](../issues/44-route-feishu-by-org-connection.md).
|
||||
4. [Build the private platform-administrator control plane](../issues/45-build-private-platform-admin-control-plane.md).
|
||||
5. [Build the private Organization-administrator control plane](../issues/46-build-private-org-admin-control-plane.md).
|
||||
|
||||
The Feishu routing slice precedes durable inbound/action/outbound and signed
|
||||
traffic limiting so those mechanisms use the right issuer namespace from their
|
||||
first durable schema. The platform panel precedes the emergency-brake product
|
||||
entry so ticket 37 can reuse one authenticated/audited operator boundary.
|
||||
Existing tickets 32–38 continue to own capacity enforcement, attribution,
|
||||
alerts, brakes and calibration; the new UI tickets do not duplicate them.
|
||||
|
||||
## Required production journey evidence
|
||||
|
||||
The final release gate must run against at least two configured customer-owned
|
||||
Feishu applications and two Organizations, using deliberately colliding fake
|
||||
provider-local IDs where the test harness permits. It must prove:
|
||||
|
||||
1. an unauthenticated request and an authenticated wrong-role user cannot load
|
||||
either management page data or perform mutations;
|
||||
2. a Platform Administrator can create an Organization, bootstrap its OWNER,
|
||||
configure/test its Feishu app and distinct platform-managed Provider
|
||||
Connection, with redacted responses and complete audit history;
|
||||
3. an Organization OWNER/ADMIN logs in through that Organization's app, reaches
|
||||
the real org panel, manages the accepted settings/resources, configures BYOK
|
||||
when selected, and cannot read another Organization or platform secret;
|
||||
4. each bot receives, binds, authorizes, queues and replies only inside its
|
||||
connection's Organization even when `open_id`, `chat_id`, event or callback
|
||||
values collide; SUSPENDED/ARCHIVED Organizations fail closed;
|
||||
5. provider resolution for every run returns that Organization's selected
|
||||
connection and never a process-global fallback;
|
||||
6. current member onboarding, Project authorization and Agent Run behavior
|
||||
remain intact after connection namespacing;
|
||||
7. layered limits, fair admission, usage alerts and `DRAIN`/`STOP_NOW` are
|
||||
visible only to the corresponding authorized control plane and behave as
|
||||
proven by their dedicated fault/capacity suites;
|
||||
8. the complete browser + Feishu journey survives the production-shaped deploy,
|
||||
restart, backup/restore and rollback evidence owned by ticket 08.
|
||||
Reference in New Issue
Block a user