This page is service-only. The embedded libraries’ KMS story is
covered separately under Managing Encryption Keys.
AWS access — what the service expects
The service uses the standard AWS credential provider chain to reachaws-kms and aws (Secrets Manager) slots. The exact resolution depends on where you run it:
The
CYBORGDB_S3_* storage credentials are deliberately separate from AWS_* KMS credentials. KMS uses the AWS default chain (or AssumeRole); S3 storage uses its own explicit keys when both CYBORGDB_S3_ACCESS_KEY and CYBORGDB_S3_SECRET_KEY are set. Set both for a non-AWS storage backend (MinIO, R2, etc.); if either is unset, storage falls back to the same AWS default chain that KMS uses.Required IAM permissions (per provider)
provider: aws-kms — on the wrap key:
provider: aws (Secrets Manager) — on the secret:
-* as shown.
Model
CyborgDB Service uses a two-key hierarchy per index:- KEK (Key-Encryption Key, 32 bytes) — per index. Resolved
from the named KMS registry slot at index creation, then re-fetched
(or unwrapped) on every load. Held in a short-lived in-process
cache; the TTL is set by
INDEX_KEK_CACHE_TTL_SECONDS(default 60 s). - DEK (Data-Encryption Key, 32 bytes) — per index. Generated internally at index-creation time, wrapped under the KEK with AES-GCM, and persisted alongside the index. Never leaves the service and is never returned to clients.
Supplying both
kms_name and index_key is rejected with a 400.
Supplying neither is also a 400.
YAML Registry
Per-index KMS slots live in the service YAML file (not environment variables) underkms.registry. Each slot is a named entry that
create_index(..., kms_name=<slot>) can reference.
cyborgdb.yaml
String values support env-var substitution:
${VAR} (required, fails on startup if unset) and ${VAR:-default} (uses the default when unset). Use this to keep BYOK role ARNs and external IDs out of the checked-in file.role_session_name defaults to cyborgdb-<key_id> (with any alias/ prefix removed). STS rejects session names containing / or :, so set role_session_name explicitly on any BYOK slot whose key_id is a path-style name (such as customers/acme/kek) or an ARN.Provider types
Theprovider field selects how the KEK is wrapped:
Both providers additionally accept
role_arn + external_id for
cross-account access (BYOK) — the service calls sts:AssumeRole
before reaching the key.
There is no registry slot for the SDK-supplied path. Omit
kms_name entirely from create_index and supply index_key
directly; the persisted envelope records provider: none.
Caching and revocation
The service caches plaintext KEKs in memory forINDEX_KEK_CACHE_TTL_SECONDS (default 60 s). Only KMS-derived KEKs
are cached — the SDK-supplied path always passes through. Shorter
TTLs propagate KMS revocations (key deletion, IAM policy detach,
trust-policy edit) faster but cost more KMS calls.
To force-revoke an index globally, revoke the wrap key in the KMS
provider; cached KEKs expire within INDEX_KEK_CACHE_TTL_SECONDS.
Creating a KMS-backed index
The SDK call is unchanged except for swappingindex_key= for
kms_name=:
load_index, upsert,
query, delete — the SDK omits index_key. The service looks up
the index’s persisted envelope, resolves the KEK via the named KMS
slot (cache hit, or fresh wrap/unwrap on miss), and uses it to
open the index.
Bring Your Own KMS (cross-account)
In BYOK, the wrap key lives in the customer’s AWS account. CyborgDB Service holds no long-term credentials to that account — access flows throughsts:AssumeRole with an ExternalId on every
wrap or unwrap call.
Setup is a two-party handshake:
1
Service operator: generate an ExternalId
Treat as a credential — it’s the cryptographic gate
preventing one customer’s role ARN from being abused by
another.
2
Service operator: share three values with the customer
- The service’s AWS principal ARN (the service’s AWS
identity). Get it with:
- The
ExternalIdyou generated. - The customer-facing setup steps (Step 3 below).
3
Customer: create the wrap key and IAM role
1. Create the wrap key in Secrets Manager (32 random
bytes):2. Create an IAM role with this trust policy (substituting
the operator-supplied values):3. Attach an inline permission policy for the secret:Secrets Manager appends 6 random chars to every secret ARN,
so either use the secret’s full ARN as-is, or the ARN up to
the secret name followed by
-* as shown.Send three values back to the operator: role ARN,
secret name, region. Never share AWS credentials.4
Service operator: add the slot to YAML
Under Then restart the service. On boot, look for:
kms.registry::5
(Optional) Verify the AssumeRole
From the same shell environment as the service:Prints
ASIA... → trust policy and ExternalId match.
AccessDenied → the customer’s trust policy is wrong; share
the error.Revocation
- Pause access — customer detaches the inline permission policy
from the role. The service starts failing on the next cache
miss (i.e. within
INDEX_KEK_CACHE_TTL_SECONDS). - Permanent revoke — customer deletes the role or the secret. Note: deleting the secret renders every index wrapped under it permanently unreadable.
Configuration changes after creation
Changes to the YAML registry take effect only after a service restart. Each index remembers theprovider, key_id, and region
it was created with, and on restart the service handles changes as
follows:
role_arn/external_id/role_session_namechanges — picked up automatically. Nothing else to do.provider/key_id/regionchanges — treated as a key rotation. The service re-wraps each affected index under the new key on startup. The old key must still be reachable with the slot’s newrole_arn/external_id, so change the account and the key in separate steps. If a rotation fails, the service still starts, but requests for that index fail until the issue is fixed; check the startup logs.- Slot removed — don’t remove a slot while indexes still use it. Requests for those indexes fail with HTTP 401 until the slot is restored.
Troubleshooting
AccessDenied on sts:AssumeRole
AccessDenied on sts:AssumeRole
Trust policy mismatch. Either the
Principal doesn’t match
the service’s AWS identity, or sts:ExternalId doesn’t
match the operator-provided UUID. Re-run the manual verify
in Step 5 above and share the exact error with the customer.AccessDenied on GetSecretValue
AccessDenied on GetSecretValue
The permission policy on the customer’s role doesn’t cover
the secret, or the
Resource ARN is missing the required
-* suffix (Secrets Manager appends 6 random chars).Wrap key not present in secrets manager
Wrap key not present in secrets manager
The secret was deleted, or the
region in the YAML slot
doesn’t match the region where the secret lives. Seen as
HTTP 502 on index creation (...; cannot wrap) and HTTP 401
on later requests (...; cannot unwrap).Wrap key must be 32 bytes
Wrap key must be 32 bytes
The Secrets Manager value is the wrong size. Replace with
exactly 32 random bytes.
Index creation: index_key must not be supplied alongside kms_name
Index creation: index_key must not be supplied alongside kms_name
Supplying both fields is rejected with HTTP 400 regardless
of provider type. Pick one path per index. Supplying neither
is also a 400:
exactly one of kms_name or index_key must be supplied.index '<name>' is KMS-backed ... do not supply index_key
index '<name>' is KMS-backed ... do not supply index_key
A request for a KMS-backed index included
index_key
(HTTP 400). Remove it from the call.Unknown KMS registry entry
Unknown KMS registry entry
kms_name doesn’t match any slot under kms.registry
(HTTP 400). The error lists the known slots. Check the
startup log line KMS registry loaded (...).Cached KEK after revoking access
Cached KEK after revoking access
Plaintext KEKs are cached for
INDEX_KEK_CACHE_TTL_SECONDS (default 60 s). For tighter
revocation windows, drop the TTL — at the cost of more KMS
calls. There is no live invalidation API in v0.18.See also
- Environment Variables — full configuration reference, including
INDEX_KEK_CACHE_TTL_SECONDS. - Managing Encryption Keys — the SDK-supplied (
provider: none) path and the legacy “client decrypts via KMS” pattern. - Create Index — the
kms_nameparameter and the SDK-supplied alternative.