CyborgDB supports metadata tagging and filtering, enabling fine-grained search filters in addition to encrypted vector search. Metadata, like vectors, are end-to-end encrypted and never exposed in plaintext outside of the client.
The following metadata types are supported:
- Boolean
- String
- List of strings
- Numbers (will be stored as fp64)
To add items with metadata tags, you can pass a dictionary of key-value pairs to the metadata field during upsert:
# Example data in the format:
# [{id: int, vector: List[float], metadata: Dict)]
data = [
{"id": "item_1", "vector": [0.1, 0.1, 0.1, 0.1], "metadata": {"category": "dog"}},
{"id": "item_2", "vector": [0.2, 0.2, 0.2, 0.2], "metadata": {"category": "cat"}}
]
# Upsert data with metadata
index.upsert(data)
// Example data
std::vector<std::string> ids = {"item_1", "item_2"};
cyborg::Array2D<float> vectors{{0.1, 0.1, 0.1, 0.1}, {0.2, 0.2, 0.2, 0.2}};
std::vector<std::string> metadata = {R"({"category": "dog"})", R"({"category": "cat"})"};
// Upsert data with metadata
index->Upsert(ids, vectors, {}, metadata);
Metadata field names and string values are case-sensitive. Ensure that the metadata field names and values are consistent when filtering queries.
filters field during query:
# Example query
query_vectors = [0.5, 0.9, 0.2, 0.7]
top_k = 10
# Example filters
filters = {
"category": {"$in": ["dog", "cat"]} # Will match either 'dog' or 'cat'
}
# Perform query
results = index.query(query_vectors=query_vectors, top_k=top_k, filters=filters)
// Example query
cyborg::Array2D<float> query_vectors{{0.5, 0.9, 0.2, 0.7}};
int top_k = 10;
// Example filters
std::string filters = R"({"category": {"$in": ["dog", "cat"]}})";
// Create QueryParams
cyborg::QueryParams query_params(top_k);
query_params.filters = filters;
// Perform query
cyborg::QueryResults results = index->Query(query_vectors, query_params);
| Filter | Types | Description |
|---|
$exists | any | Matches vectors that have this field |
$eq | Boolean, Number, String | Matches vectors with the metadata that is equal to the filter value |
$ne | Boolean, Number, String | Matches vectors with the metadata that is not equal to the filter value |
$in | Number, String | Matches vectors with metadata that is in the filter array |
$nin | Number, String | Matches vectors with metadata that is not in the filter array |
$gt | Number | Matches vectors with metadata that is greater than the filter value |
$gte | Number | Matches vectors with metadata that is greater than or equal to the filter value |
$lt | Number | Matches vectors with metadata that is less than the filter value |
$lte | Number | Matches vectors with metadata that is less than or equal to the filter value |
