Train

Clusters the index’s vectors into IVF lists so queries only scan the closest lists. Train waits until training finishes before returning.

func (e *EncryptedIndex) Train(ctx context.Context, params TrainParams) error

The service also trains indexes automatically. An untrained index is trained once it holds more than AUTO_TRAIN_MIN_VECTORS vectors (default 65536), and a trained index is retrained when its size calls for more lists than it has. Set AUTO_TRAIN_DISABLED=true on the service to turn this off. Call Train yourself to train a smaller index, or to make sure training is finished before running queries or benchmarks.

Parameters

TrainParams

All fields are pointers; nil uses the service default. Pass cyborgdb.TrainParams{} for all defaults.

Field	Type	Default	Description
`NLists`	`*int32`	`nil` (auto)	Number of IVF lists. When `nil`, the square root of the vector count, rounded to a power of two.
`BatchSize`	`*int32`	`nil` (ignored)	Vectors processed per training batch.
`MaxIters`	`*int32`	`nil` (ignored)	Maximum clustering iterations.
`Tolerance`	`*float64`	`nil` (ignored)	Convergence tolerance.
`MaxMemory`	`*int32`	`nil` (ignored)	Memory cap for training, in MB.

Only NLists affects training today. The service accepts BatchSize, MaxIters, Tolerance, and MaxMemory but doesn’t pass them to training, so they have no effect.

Parameter values aren’t range-checked: an out-of-range value such as a negative list count is accepted, and training is skipped as described in Requirements.

Requirements

Training only runs when all three hold:

The index has at least 10,000 vectors.
It has at least twice as many vectors as NLists.
The vector dimension is a multiple of 32, such as 384, 768, or 1536.

Train doesn’t report a skipped or failed training run. If a requirement isn’t met, or training fails on the service, Train still returns nil and the index stays untrained. Check IsTrained after Train returns: an untrained index has NLists of 1. Queries on an untrained index still work.

Returns

error only. If the index is already training, Train waits for that run to finish, then trains again.

Errors

*AuthenticationError

The client isn’t using the root key (HTTP 401/403). Training requires the root key.

*NotFoundError

The index doesn’t exist (HTTP 404). A wrong key gives *NotFoundError or *AuthenticationError, as described in Load Index.

*ServiceError / *TransportError

The service returns 5xx or can’t be reached. A failed training run doesn’t return an error; see Requirements.

See Errors.

Example Usage

Default settings

if err := index.Train(ctx, cyborgdb.TrainParams{}); err != nil {
    log.Fatal(err)
}

// Train doesn't return an error if training is skipped, so check the result
trained, err := index.IsTrained(ctx)
if err != nil {
    log.Fatal(err)
}
nLists, err := index.NLists(ctx)
if err != nil {
    log.Fatal(err)
}
fmt.Println(trained, nLists)
// Output: true 128

Custom settings

nLists := int32(128)
batchSize := int32(4096)
maxIters := int32(200)
tolerance := 1e-7

err := index.Train(ctx, cyborgdb.TrainParams{
    NLists:    &nLists,
    BatchSize: &batchSize,
    MaxIters:  &maxIters,
    Tolerance: &tolerance,
})
if err != nil {
    log.Fatal(err)
}

Introduction

Client

Encrypted Index

Types & Helpers

Parameters

TrainParams

Requirements

Returns

Errors

Example Usage

Default settings

Custom settings

​Parameters

​TrainParams

​Requirements

​Returns

​Errors

​Example Usage

​Default settings

​Custom settings

Parameters

TrainParams

Requirements

Returns

Errors

Example Usage

Default settings

Custom settings