To create an encrypted index, you need to specify an index name (must be unique), an index key, and an index configuration. Here’s an example with an IVFFlat index type:

# Import cyborgdb_core or cyborgdb_lite:
import cyborgdb_core as cyborgdb
import cyborgdb_lite as cyborgdb
import secrets

# Using `memory` storage for this example
index_location = cyborgdb.DBConfig("memory") 
config_location = cyborgdb.DBConfig("memory")

# Create a client
client = cyborgdb.Client(index_location, config_location)

# Create an IVFFlat index config (can also be IVF/IVFPQ)
# Using an example vector dimension of 128, and number of lists of 1024
index_config = cyborgdb.IndexIVFFlat(dimension=128, n_lists=1024)

# Generate an encryption key for the index
index_key = secrets.token_bytes(32)

# Create an encrypted index
index = client.create_index("my_index", index_key, index_config)

This creates a new encrypted index with the IVFFlat type. For more details on IVFFlat and other index options, see Configure an Encrypted Index.

The example above creates a random 32 byte (256-bit) index key. This is fine for evaluation purposes, but for production use, we recommend that you use an HSM or KMS solution. For more details, see Managing Encryption Keys.

​
Encrypted Index Caching

For improved query performance, you can enable encrypted index caching by setting a max_cache_size:

# Import cyborgdb_core or cyborgdb_lite:
import cyborgdb_core as cyborgdb
import cyborgdb_lite as cyborgdb
import secrets

# Using `memory` storage for this example
index_location = cyborgdb.DBConfig("memory") 
config_location = cyborgdb.DBConfig("memory")

# Create a client
client = cyborgdb.Client(index_location, config_location)

# Example index config
index_config = cyborgdb.IndexIVFFlat(dimension=128, n_lists=1024)

# Generate an encryption key for the index
index_key = secrets.token_bytes(32)

# Set max cache size at 1MB
max_cache_size = 1000000

# Create an encrypted index
index = client.create_index("my_index", index_key, index_config, max_cache_size)

​
Automatic Embedding Generation

This feature is only available in Python and is experimental as of v0.9.0.

In the Python version of CyborgDB, you can enable automatic embedding generation for the encrypted index by setting embedding_model in create_index():

Python
# Import cyborgdb_core or cyborgdb_lite:
import cyborgdb_core as cyborgdb
import cyborgdb_lite as cyborgdb

# Using `memory` storage for this example
index_location = cyborgdb.DBConfig("memory")
config_location = cyborgdb.DBConfig("memory")
items_location = cyborgdb.DBConfig("memory")

# Create a client
client = cyborgdb.Client(index_location, config_location, items_location)

# Example index config
index_config = cyborgdb.IndexIVFFlat(dimension=128, n_lists=1024)

# Generate an encryption key for the index
index_key = secrets.token_bytes(32)

# Set embedding model (from HuggingFace)
embedding_model = "all-MiniLM-L6-v2"

# Create an encrypted index with managed embedding generation
index = client.create_index("my_index", index_key, index_config, embedding_model=embedding_model)

​
API Reference

For more information on creating encrypted indexes, refer to the API reference:

Python API Reference

API reference for create_index() in Python

C++ API Reference

API reference for CreateIndex() in C++

Was this page helpful?

Suggest edits