Create Index

Creates a new encrypted index on the service and returns an EncryptedIndex bound to it.

client.create_index(
    index_name,
    index_key = None,
    kms_name = None,
    dimension = None,
    embedding_model = None,
    metric = None,
    storage_precision = None,
    metadata_schema = None,
    text_fields = None,
    bm25_k1 = None,
    bm25_b = None,
)

There is one index type, configured with flat keyword arguments. The earlier IndexIVFFlat, IndexIVFPQ, and IndexIVFSQ types and the index_config argument no longer exist.

Parameters

Parameter	Type	Default	Description
`index_name`	`str`	-	Unique name for the index.
`index_key`	`bytes`	`None`	(Optional) 32-byte encryption key (SDK-supplied KEK). Pass either `index_key` or `kms_name`, never both.
`kms_name`	`str`	`None`	(Optional) Name of a `kms.registry` entry in the service YAML. The service generates the KEK and wraps it with that provider; the SDK never sees the plaintext key.
`dimension`	`int`	`None`	(Optional) Vector dimensionality. When omitted, the service detects it from the first upsert.
`embedding_model`	`str`	`None`	(Optional) Name of one of the service’s built-in embedding models (listed by `GET /v1/embedding-models`), used for automatic embedding generation. When set, `dimension` is taken from the model.
`metric`	`str`	`None` (service default: `"euclidean"`)	(Optional) `"euclidean"`, `"squared_euclidean"`, or `"cosine"`.
`storage_precision`	`str`	`None` (service default: `"float32"`)	(Optional) Storage format for the index’s stored vector embeddings. See Storage precision.
`metadata_schema`	`Dict[str, Dict[str, bool]]`	`None`	(Optional) Per-field metadata indexing policy. Fixed at creation and cannot be changed later. See Metadata schema.
`text_fields`	`List[str]`	`None`	(Optional) Metadata fields to index for BM25 full-text search. Shorthand for `{"full_text": True}` in `metadata_schema`. Enables `text=` on `query()` and `query_metadata()`.
`bm25_k1`	`float`	`None` (service default: `1.2`)	(Optional) BM25 term-frequency saturation, `>= 0`. Requires at least one full-text field.
`bm25_b`	`float`	`None` (service default: `0.75`)	(Optional) BM25 length-normalization strength, in `[0, 1]`. Requires at least one full-text field.

Key management

Pass exactly one of index_key or kms_name. The service rejects both together with ValidationError (HTTP 400), whatever the KMS provider.

SDK-supplied KEK: pass index_key. Supply the same key again on every load_index().
KMS-backed: pass kms_name. The service generates the key, wraps it with the named provider, and stores the wrapped key. Later calls omit index_key.

Storage precision

storage_precision sets the format used to store the index’s vector embeddings. Every value works with every metric.

Value	Bits per dimension
`"float32"`	32
`"float16"`	16
`"tq12"`	12
`"tq8"`	8
`"tq6"`	6
`"tq4"`	4

The tq* values are TurboQuant tiers; lower tiers use less storage in exchange for a small recall and latency cost.

Metadata schema

metadata_schema sets how each metadata field is indexed. Metadata itself stays schemaless: this controls indexing, not what may be stored. Fields left out get the default policy (indexed and filterable). Nested fields use dot-paths such as "loc.city".

metadata_schema = {
    "title":         {"filterable": True, "pattern": True},   # regex-searchable
    "internal_blob": {"filterable": False},                   # stored, not indexed
    "body":          {"full_text": True},                     # BM25 full-text search
}

Each field takes three settings:

Setting	Default	Effect
`filterable`	`True`	Indexes the field, so filters on it resolve from the index. `False` still stores the field but doesn’t index it; see Filters the index cannot answer.
`pattern`	`False`	Also builds a regex dictionary, so `$regex` and `$contains` resolve from the index. Requires `filterable=True`.
`full_text`	`False`	Sends the field’s text through the BM25 analyzer instead of exact-match indexing. Implies `filterable=False`; an explicit `filterable=True` with it is rejected with `ValidationError` (HTTP `422`). Can’t be combined with `pattern=True`.

Filters the index cannot answer

The index can answer a filter when the field is filterable and, for $regex or $contains, the field also has pattern set. Any other filter is handled differently depending on the call:

Call	Filter the index can’t answer
`query()` without `text`	Still works, but slower: the filter is applied to the decrypted metadata after the search.
`query()` with `text`	Rejected with `ValidationError`.
`query_metadata()`	Rejected with `ValidationError`.

With the schema above:

{"title": {"$regex": "^Intro"}}    # answerable: title is filterable and has pattern
{"internal_blob": "abc"}           # not answerable: internal_blob isn't filterable
{"author": {"$regex": "^Ann"}}     # not answerable: author is filterable by default, but has no pattern

The stored policy is returned by the metadata_schema getter.

Returns

An EncryptedIndex bound to the new index. For KMS-backed indexes it holds no plaintext key.

Exceptions

cyborgdb.ValidationError

Before any request is sent: neither index_key nor kms_name is passed, index_key is not 32 bytes, or storage_precision is not one of the six values above. From the service: the index name already exists, both index_key and kms_name were passed, or the kms_name slot is unknown (HTTP 400); or metadata_schema combines settings that aren’t allowed (HTTP 422).

cyborgdb.AuthenticationError

The client’s api_key is missing or invalid (HTTP 401), or is a per-user key: creating an index requires the root key (HTTP 403).

cyborgdb.ServiceError

The service returns 5xx, including KMS wrap failures (502).

cyborgdb.TransportError

The service can’t be reached.

See Errors for the full hierarchy.

Example Usage

SDK-supplied key

from cyborgdb import Client

client = Client(base_url='http://localhost:8000', api_key='your-api-key')

index_key = client.generate_key()

# The dimension is detected from the first upsert
index = client.create_index('documents', index_key=index_key)

Store index_key securely: it is needed to load the index again. See Generate Key.

Explicit dimension and metric

index = client.create_index(
    'documents_384',
    index_key=index_key,
    dimension=384,
    metric='cosine',
)

KMS-backed index

# The service generates the key and wraps it with the 'prod-aws' registry slot.
index = client.create_index(
    'kms_documents',
    kms_name='prod-aws',
    dimension=384,
)
# Later calls, including load_index(), omit index_key.

Automatic embeddings

# The dimension is taken from the embedding model
index = client.create_index(
    'semantic_docs',
    index_key=index_key,
    embedding_model='all-MiniLM-L6-v2',
)

See Auto-Generate Embeddings.

Reduced storage with TurboQuant

index = client.create_index(
    'compact_docs',
    index_key=index_key,
    dimension=768,
    storage_precision='tq8',
)

Per-field metadata policy

index = client.create_index(
    'docs_index',
    index_key=index_key,
    dimension=4,
    metadata_schema={
        'title':         {'filterable': True, 'pattern': True},
        'internal_blob': {'filterable': False},
    },
)
index.upsert([
    {'id': 'doc1', 'vector': [0.1, 0.2, 0.3, 0.4], 'metadata': {'title': 'intro to encryption'}},
    {'id': 'doc2', 'vector': [0.5, 0.6, 0.7, 0.8], 'metadata': {'title': 'advanced setup'}},
])

# $regex resolves from the index because 'title' has pattern=True
print(index.query_metadata(filters={'title': {'$regex': '^intro'}}))
# Output: [{'id': 'doc1'}]

Full-text search fields

index = client.create_index(
    'articles',
    index_key=index_key,
    dimension=4,
    text_fields=['body'],
)
print(index.bm25)
# Output: {'k1': 1.2, 'b': 0.75, 'analyzer_version': 'en/1;sw=2de831ef7be676b6;snowball-english-utf8-3.0.1'}

Introduction

Client

Encrypted Index

Types & Helpers

Parameters

Key management

Storage precision

Metadata schema

Filters the index cannot answer

Returns

Exceptions

Example Usage

SDK-supplied key

Explicit dimension and metric

KMS-backed index

Automatic embeddings

Reduced storage with TurboQuant

Per-field metadata policy

Full-text search fields

​Parameters

​Key management

​Storage precision

​Metadata schema

​Filters the index cannot answer

​Returns

​Exceptions

​Example Usage

​SDK-supplied key

​Explicit dimension and metric

​KMS-backed index

​Automatic embeddings

​Reduced storage with TurboQuant

​Per-field metadata policy

​Full-text search fields

Parameters

Key management

Storage precision

Metadata schema

Filters the index cannot answer

Returns

Exceptions

Example Usage

SDK-supplied key

Explicit dimension and metric

KMS-backed index

Automatic embeddings

Reduced storage with TurboQuant

Per-field metadata policy

Full-text search fields