Skip to main content
Trains the encrypted index to optimize it for efficient similarity search queries. Training is essential for IVF-based indexes to achieve optimal query performance and accuracy.
In CyborgDB Service v0.17, training is auto-triggered server-side once upserts cross the configured RETRAIN_THRESHOLD. The Go SDK’s Upsert response only carries Status and Message — to observe training, call IsTraining. Calling Train explicitly forces immediate clustering and is useful when you want to block until the index is ready (for example, before benchmarking queries). Auto-training can be disabled service-side with the AUTO_TRAIN_DISABLED setting.
func (e *EncryptedIndex) Train(ctx context.Context, params TrainParams) error

​
Parameters

ParameterTypeDescription
ctxcontext.ContextContext for cancellation and timeouts (training can take time)
paramsTrainParamsTraining parameters specifying optimization options

​
TrainParams Fields

FieldTypeDescription
BatchSize*int32(Optional) Number of vectors processed per training batch (default: 2048)
MaxIters*int32(Optional) Maximum training iterations (default: 100)
Tolerance*float64(Optional) Convergence tolerance for training (default: 1e-6)
MaxMemory*int32(Optional) Maximum memory usage in MB, 0 = no limit (default: 0)
NLists*int32(Optional) Number of IVF clusters, 0 = auto-determine (default: 0)
Training is a compute-intensive operation that may take several seconds to minutes depending on the index size and configuration.

​
Returns

  • error: Any error encountered during training, or nil if successful

​
Error Handling

  • Returns error if the API request fails due to network connectivity issues
  • Returns error if authentication fails (invalid API key)
  • Returns error if the encryption key is invalid for the specified index
  • Returns error if there are internal server errors during training
  • Returns error if the index has insufficient data for training
  • Returns error if training parameters are invalid or out of range
  • Returns error if training fails due to memory constraints

​
Example Usage

package main

import (
    "context"
    "encoding/hex"
    "fmt"
    "log"
    "time"

    "github.com/cyborginc/cyborgdb-go"
)

func main() {
    client, err := cyborgdb.NewClient("http://localhost:8000", "your-api-key")
    if err != nil {
        log.Fatal(err)
    }

    // Load an existing index
    indexKeyHex := "your-64-character-hex-key-here"
    indexKey, err := hex.DecodeString(indexKeyHex)
    if err != nil {
        log.Fatal(err)
    }

    index, err := client.LoadIndex(context.Background(), "my-vector-index", indexKey)
    if err != nil {
        log.Fatal(err)
    }

    // Basic training with default parameters
    params := cyborgdb.TrainParams{}

    fmt.Println("Starting index training...")
    err = index.Train(context.Background(), params)
    if err != nil {
        log.Fatalf("Training failed: %v", err)
    }

    fmt.Println("Training completed successfully!")

    trained, err := index.IsTrained(context.Background())
    if err != nil {
        log.Fatalf("Failed to check training status: %v", err)
    }
    fmt.Printf("Index is now trained: %t\n", trained)
}

​
Training with Custom Parameters

// Training with custom parameters for better control
batchSize := int32(1024)    // Smaller batches for limited memory
maxIters := int32(200)      // More iterations for better convergence
tolerance := 1e-7           // Tighter convergence tolerance
maxMemory := int32(4096)    // Limit to 4GB RAM
nLists := int32(256)        // Specific number of clusters

params := cyborgdb.TrainParams{
    BatchSize: &batchSize,
    MaxIters:  &maxIters,
    Tolerance: &tolerance,
    MaxMemory: &maxMemory,
    NLists:    &nLists,
}

// Use longer timeout for training large datasets
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
defer cancel()

fmt.Println("Starting training with custom parameters...")
err := index.Train(ctx, params)
if err != nil {
    if ctx.Err() == context.DeadlineExceeded {
        log.Fatal("Training timed out after 30 minutes")
    }
    log.Fatalf("Training failed: %v", err)
}

fmt.Println("Custom training completed!")