Skip to main content

​
StorageConfig

The StorageConfig class specifies the backing store for an index’s keystores. It is immutable and is built via static factory methods. A single StorageConfig is shared across all per-index keystores and is passed to the Client constructor. CyborgDB supports three backing stores: memory, disk, and s3.

​
Factories

StorageConfig.memory()
# Ephemeral, no persistence (tests / benchmarking).

StorageConfig.disk(path: str,
                   *,
                   cache_vectors: bool = False,
                   cache_metadata: bool = False,
                   cache_ids: bool = False)
# Local, RocksDB-backed persistent storage.

StorageConfig.s3(bucket: str,
                 *,
                 region: str | None = None,
                 endpoint: str | None = None,
                 prefix: str = "",
                 credentials: S3Credentials | None = None)
# AWS S3 / S3-compatible (MinIO, Ceph, R2, etc.).
FactoryDescription
memory()Ephemeral in-memory store with no persistence. Use for tests and benchmarking.
disk(path, ...)Local persistent store backed by RocksDB. Optional cache_* flags keep vectors / metadata / IDs in memory for faster reads.
s3(bucket, ...)S3 or S3-compatible object store. Set endpoint for non-AWS providers (path-style addressing is auto-selected). Omit credentials to use the AWS default credential provider chain.

​
Example Usage

import cyborgdb_core as cyborgdb

# In-memory (ephemeral)
memory_store = cyborgdb.StorageConfig.memory()

# Local disk (RocksDB-backed), caching vectors in memory
disk_store = cyborgdb.StorageConfig.disk("/tmp/cyborgdb", cache_vectors=True)

# AWS S3 with explicit credentials
s3_store = cyborgdb.StorageConfig.s3(
    "my-bucket",
    region="us-east-1",
    credentials=cyborgdb.S3Credentials(access_key="...", secret_key="..."),
)

# S3-compatible (e.g. MinIO) with default credential chain
minio_store = cyborgdb.StorageConfig.s3(
    "my-bucket",
    endpoint="http://localhost:9000",
)
For more info, you can read about supported backing stores here.

​
S3Credentials

Explicit credentials for the StorageConfig.s3(...) factory. Omit credentials= on .s3() to use the AWS default credential provider chain (environment variables, ~/.aws/credentials, EC2 instance profile, EKS IRSA). 
S3Credentials(access_key: str,
              secret_key: str,
              session_token: str | None = None)

​
Parameters

ParameterTypeDefaultDescription
access_keystr-S3 access key ID.
secret_keystr-S3 secret access key.
session_tokenstr | NoneNone(Optional) Temporary session token (for STS / assumed roles).

​
Example Usage

import cyborgdb_core as cyborgdb

creds = cyborgdb.S3Credentials(access_key="AKIA...", secret_key="...")
store = cyborgdb.StorageConfig.s3("my-bucket", region="us-east-1", credentials=creds)

​
Storage Precision

storage_precision controls the on-disk dtype of the rerank vectors for a DiskIVF index, set at create_index time.
ValueDescription
numpy.float32 (default)Full-precision rerank vectors.
numpy.float16Halves the on-disk footprint with a slight precision loss.
The strings "float32" and "float16" are also accepted. 
import cyborgdb_core as cyborgdb
import numpy as np

index = client.create_index("my_index", index_key, storage_precision=np.float16)

​
GPUConfig

The GPUConfig class configures which operations should use GPU acceleration. GPU acceleration requires CUDA support.

​
Parameters

ParameterTypeDefaultDescription
upsertboolFalse(Optional) Enable GPU for upsert operations
trainboolFalse(Optional) Enable GPU for training operations

​
Properties (Read-Only)

PropertyTypeDescription
upsertboolWhether GPU is enabled for upsert operations
trainboolWhether GPU is enabled for training operations
queryboolWhether GPU is enabled for query operations
allboolWhether all GPU operations are enabled
noneboolWhether no GPU operations are enabled

​
Example Usage

import cyborgdb_core as cyborgdb

# Enable GPU for upsert and training
gpu_config1 = cyborgdb.GPUConfig(upsert=True, train=True)

# Enable GPU only for training
gpu_config2 = cyborgdb.GPUConfig(train=True)

# Disable GPU (default)
gpu_config3 = cyborgdb.GPUConfig()

# Check GPU configuration
if gpu_config1.all:
    print("All GPU operations enabled")

if gpu_config2.train:
    print("GPU enabled for training")

​
DistanceMetric

DistanceMetric is a string representing the distance metric used for the index. Options include:
  • "cosine": Cosine similarity.
  • "euclidean": Euclidean distance.
  • "squared_euclidean": Squared Euclidean distance.

​
KMSBlob

KMSBlob describes a per-index envelope recording how the index KEK is wrapped by an external KMS. It is used with the module-level KMS functions (create_index_kms, push_index_kms, get_index_kms, delete_index_kms). This is an advanced, optional surface primarily for the service layer; embedded SDK users supplying their own KEK can ignore it or use provider="none". 
KMSBlob(kms_name: str = "",
        provider: str = "",
        key_id: str = "",
        region: str = "",
        wrapped_kek: bytes = b"",
        version: int = 1,
        created_at: int = 0)

​
Fields

FieldTypeDescription
kms_namestrHuman-readable name of the KMS configuration.
providerstrKMS provider: "aws", "aws-kms", or "none".
key_idstrIdentifier of the wrapping key within the KMS.
regionstrRegion of the KMS / key.
wrapped_kekbytesThe wrapped index KEK.
versionintEnvelope version.
created_atintCreation timestamp (unix epoch seconds).

​
Example Usage

import cyborgdb_core as cyborgdb

blob = cyborgdb.KMSBlob(
    kms_name="prod",
    provider="aws-kms",
    key_id="arn:aws:kms:us-east-1:111122223333:key/abcd",
    region="us-east-1",
    wrapped_kek=wrapped_bytes,
)