To create an encrypted index, you need to specify an index name (must be unique), an index key, and an index configuration. Here’s an example with an IVFFlat index type:
Copy
Ask AI
# Import cyborgdb_core or cyborgdb_lite:import cyborgdb_core as cyborgdbimport cyborgdb_lite as cyborgdbimport secrets# Using `memory` storage for this exampleindex_location = cyborgdb.DBConfig("memory") config_location = cyborgdb.DBConfig("memory")# Get your API keyapi_key = "your_api_key_here" # Replace with your actual API key# Create a clientclient = cyborgdb.Client(api_key, index_location, config_location)# Create an IVFFlat index config (can also be IVF/IVFPQ)# Using an example vector dimension of 128, and number of lists of 1024index_config = cyborgdb.IndexIVFFlat(dimension=128, n_lists=1024)# Generate an encryption key for the indexindex_key = secrets.token_bytes(32)# Create an encrypted indexindex = client.create_index("my_index", index_key, index_config)
This creates a new encrypted index with the IVFFlat type. For more details on IVFFlat and other index options, see Configure an Encrypted Index.
The example above creates a random 32 byte (256-bit) index key.
This is fine for evaluation purposes, but for production use, we recommend that you use an HSM or KMS solution.
For more details, see Managing Encryption Keys.
For improved query performance, you can enable encrypted index caching by setting a max_cache_size:
Copy
Ask AI
# Import cyborgdb_core or cyborgdb_lite:import cyborgdb_core as cyborgdbimport cyborgdb_lite as cyborgdbimport secrets# Using `memory` storage for this exampleindex_location = cyborgdb.DBConfig("memory") config_location = cyborgdb.DBConfig("memory")# Get your API keyapi_key = "your_api_key_here" # Replace with your actual API key# Create a clientclient = cyborgdb.Client(api_key, index_location, config_location)# Example index configindex_config = cyborgdb.IndexIVFFlat(dimension=128, n_lists=1024)# Generate an encryption key for the indexindex_key = secrets.token_bytes(32)# Set max cache size at 1MBmax_cache_size = 1000000# Create an encrypted indexindex = client.create_index("my_index", index_key, index_config, max_cache_size)
This feature is only available in Python and is experimental as of v0.9.0.
In the Python version of CyborgDB, you can enable automatic embedding generation for the encrypted index by setting embedding_model in create_index():
Python
Copy
Ask AI
# Import cyborgdb_core or cyborgdb_lite:import cyborgdb_core as cyborgdbimport cyborgdb_lite as cyborgdb# Using `memory` storage for this exampleindex_location = cyborgdb.DBConfig("memory")config_location = cyborgdb.DBConfig("memory")items_location = cyborgdb.DBConfig("memory")# Get your API keyapi_key = "your_api_key_here" # Replace with your actual API key# Create a clientclient = cyborgdb.Client(api_key, index_location, config_location, items_location)# Example index configindex_config = cyborgdb.IndexIVFFlat(dimension=128, n_lists=1024)# Generate an encryption key for the indexindex_key = secrets.token_bytes(32)# Set embedding model (from HuggingFace)embedding_model = "all-MiniLM-L6-v2"# Create an encrypted index with managed embedding generationindex = client.create_index("my_index", index_key, index_config, embedding_model=embedding_model)