7.5 KiB
ADR 0024: Local Secret Envelope And Connection Resolution
Status
Accepted.
Context
ADR-0021 requires every customer Feishu application and model provider credential to belong to one Organization. It forbids a process-global provider credential and requires business code to obtain plaintext only through a resolver. Before the pilot can store real customer credentials, the deployment also needs a concrete encryption, rotation, recovery, writer-authority, and connection-identity contract.
The initial production shape is one Hub service and one PostgreSQL database on a controlled host. An external KMS would add another remote availability and bootstrap dependency before the pilot has operational support for it. Keeping plaintext credentials in environment variables, configuration files readable by the service account, database columns, logs, or Agent environments is not an acceptable substitute.
Decision
Local master-key keyring
The pilot uses a versioned local master-key keyring. Each key is a random 256-bit key-encryption key (KEK) with a stable, non-secret key identifier. The keyring names exactly one active KEK and may retain previous KEKs while rotation is in progress. It is not stored in PostgreSQL, a release directory, an environment variable, or the ordinary database/workspace backup set.
In production, the source keyring is owned by root, mode 0600, and delivered
to the unprivileged Hub service by systemd LoadCredential. Hub reads the
read-only file under CREDENTIALS_DIRECTORY; production startup rejects a
direct arbitrary file path. Development and tests may use an explicit local
keyring file. Startup fails when the keyring is absent, malformed, contains an
invalid key, has no active key, or names an active key that is not present.
Envelope format and binding
Every immutable secret version receives a new random 256-bit data-encryption key (DEK). AES-256-GCM encrypts the secret payload with that DEK. AES-256-GCM also wraps the DEK with the keyring's active KEK. Nonces are random and never reused with the same key.
The versioned envelope persists only algorithm/version metadata, the KEK identifier, nonces, authentication tags, the wrapped DEK, and ciphertext. Authenticated additional data binds both layers to the secret purpose, Organization identifier, Connection identifier, immutable secret-version identifier, and envelope version. Copying ciphertext between rows, Organizations, connections, purposes, or versions therefore fails authentication. Unknown algorithms or versions, missing KEKs, malformed envelopes, and failed authentication are hard errors; there is no fallback to another credential source.
The encrypted payload is a versioned object specific to its connection type. For a Provider Connection, provider base URL and authentication material are inside the payload. For a Feishu Application Connection, application ID, application secret, verification/encryption material, and bot identity are inside the payload. HTTP responses, business records, logs, metrics, traces, audit facts, exceptions, and Agent process environments receive only redacted metadata, never those plaintext fields or a decrypted DEK.
Connection identity, lifecycle, and writer authority
A connection has a stable opaque identifier and belongs to exactly one
Organization. Secret versions are immutable; the connection points to one
active version. Rotation creates and validates a new version, atomically moves
the active pointer, and retains the previous encrypted version for rollback and
audit until retention policy removes it. A connection is DRAFT, ACTIVE, or
DISABLED; runtime resolution accepts only an active connection with a valid
active secret version.
There is one customer-owned Feishu Application Connection per Organization. Its provider-local user, chat, event, and callback identifiers are meaningful only together with that connection identity. Organization OWNER/ADMIN may create and rotate that customer-owned connection. Platform repair remains a separate stepped-up, reason-bearing, fail-closed-audited operation under ADR-0023, not an Organization API bypass.
A model Provider Connection is unique by Organization and provider identity.
In BYOK mode an Organization OWNER/ADMIN creates and rotates it. In
PLATFORM_MANAGED mode only a stepped-up Platform Administrator may create or
rotate it, with the platform mutation and redacted Platform Audit Entry in one
transaction. Switching modes creates or activates the correctly governed
connection version; it never re-labels plaintext or grants the other control
plane write authority.
Resolver boundary
Business code asks one connection resolver for an active connection using an explicit Organization, or using a Project that is first resolved to its active Organization. Provider resolution also requires the provider identity; Feishu resolution requires the intended Feishu Connection identity whenever external traffic enters the system. Resolution fails for missing, draft, disabled, cross-Organization, corrupt, or undecryptable records.
Decrypted values exist only in the resolver result for the duration of the outbound provider/Feishu operation. They are not cached across connections, returned through administrative reads, passed to Agent tools, or written back to business rows. Production has no process-global Feishu or model-provider credential fallback.
The Claude Agent child never receives an Organization's Provider credential. For each run, Hub opens a loopback-only proxy with a random, short-lived run capability. The child receives only that local endpoint and capability; Hub validates the capability, replaces it with the Organization credential at the upstream hop, refuses automatic redirects, and destroys the listener when the run ends. Agent sandbox credential rules hide even the run capability from tool subprocesses.
KEK rotation, backup, and recovery
KEK rotation is staged: install a new KEK alongside the old keys, mark it active, rewrap each stored DEK under the new KEK without decrypting its payload, verify every envelope, then retire the old KEK only after the separately protected recovery copy and restore drill are current. Secret rotation and KEK rotation are distinct operations.
A recoverable deployment requires both the ordinary database/workspace backup and a separately protected backup of the local keyring. Neither copy alone can recover plaintext credentials. Restore preflight verifies key identifiers and authenticates sample/all envelopes before traffic is enabled. A missing or incorrect keyring leaves the service unavailable and produces redacted diagnostics; operators must restore the correct keyring rather than reset or silently discard credentials.
Consequences
- The database may be restored or copied without exposing plaintext secrets, but losing the separately held keyring makes encrypted credentials unrecoverable.
- The service unit, installer, preflight, backup, restore, and rotation tooling must treat the keyring as an explicit production dependency.
- Administrative writes must accept secret material as write-only input and return only connection state, version, timestamps, and redacted fingerprints.
- Tests must prove cross-Organization/row substitution failure, wrong/missing key failure, version rotation, absence of global fallback, and absence of plaintext from persistence, responses, logs, and Agent environments.
- External KMS/HSM adoption is deferred. It may replace the local KEK provider without changing connection identity, immutable secret versions, AAD binding, writer authority, or resolver behavior.