Record Class Config

java.lang.Object
java.lang.Record
com.apple.foundationdb.async.guardiann.Config
Record Components:
metric - the metric in use for this Guardiann structure
numDimensions - the number of dimensions of the vectors stored
primaryClusterMin - minimum number of primary vectors in a cluster, underflow will result in a merge task to be enqueued
primaryClusterMax - maximum number of primary vectors in a cluster, overflow will result in a split task to be enqueued
underreplicatedPrimaryClusterMax - maximum number of under-replicated primary vectors in a cluster, overflow will result in a reassign task to be enqueued
replicatedClusterMaxWrites - maximum number of writes of replicated vectors to a cluster
replicatedClusterTarget - the number of replicated clusters we target whenever a split/merge or a reassign task is executed
replicationPriorityMin - minimum threshold for the replication priority score
replicationDistanceRatioWeight - weight of the border-proximity distance ratio term in the replication priority score
replicationZScoreWeight - weight of the distance z-score term in the replication priority score
replicationStatsMinSampleSize - minimum number of primary vectors in a cluster before its distance statistics (mean/standard deviation) are trusted for the replication priority z-score term
sampleVectorStatsProbability - probability of sampling a vector write for statistics computation
maintainStatsProbability - probability of maintaining statistics when inserting a vector
statsThreshold - number of sampled vectors that triggers centroid computation
useRaBitQ - indicator if we should use RaBitQ quantization
raBitQNumExBits - number of extra bits per dimension for RaBitQ encoding
deterministicRandomness - whether randomness should always be deterministic (for debugging/replay)
sampleBatchSize - number of sampled vectors consumed per statistics-computation pass
insertMaxCandidateClusters - maximum clusters evaluated as insertion targets
deleteMaxCandidateClusters - maximum clusters probed when locating a vector's references during delete
deleteConcurrency - concurrency for parallel operations during delete
splitNumNearestClusters - number of nearest clusters fetched from HNSW for split candidate evaluation
mergeNumNearestClusters - number of nearest clusters fetched from HNSW for merge candidate evaluation
kMeansMaxIterations - maximum Lloyd's iterations per k-means restart
kMeansMaxRestarts - maximum number of random restarts for bounded k-means during split/merge
reassignNumNeighboringClusters - outer clusters considered as replication/migration targets during reassign
collapseMinDuplicates - minimum identical vectors sharing a signature before collapse
splitMergeConcurrency - concurrency for parallel operations during split/merge tasks
reassignConcurrency - concurrency for parallel operations during reassign tasks
collapseConcurrency - concurrency for parallel operations during collapse tasks
bounceConcurrency - concurrency for parallel operations during bounce tasks
constructionSearchConfig - centroid-walk tuning (SearchConfig) for the non-search insert/delete/maintenance paths, which probe the centroid HNSW without a per-query SearchConfig; only its centroidEf* knobs are consulted there
All Implemented Interfaces:
VectorEncodingConfig

public record Config(@Nonnull Metric metric, int numDimensions, int primaryClusterMin, int primaryClusterMax, int underreplicatedPrimaryClusterMax, int replicatedClusterMaxWrites, int replicatedClusterTarget, double replicationPriorityMin, double replicationDistanceRatioWeight, double replicationZScoreWeight, int replicationStatsMinSampleSize, double sampleVectorStatsProbability, double maintainStatsProbability, int statsThreshold, boolean useRaBitQ, int raBitQNumExBits, boolean deterministicRandomness, int sampleBatchSize, int insertMaxCandidateClusters, int deleteMaxCandidateClusters, int deleteConcurrency, int splitNumNearestClusters, int mergeNumNearestClusters, int kMeansMaxIterations, int kMeansMaxRestarts, int reassignNumNeighboringClusters, int collapseMinDuplicates, int splitMergeConcurrency, int reassignConcurrency, int collapseConcurrency, int bounceConcurrency, @Nonnull SearchConfig constructionSearchConfig) extends Record implements VectorEncodingConfig
Configuration for the Guardiann vector structure.
  • Field Details

    • DEFAULT_METRIC

      @Nonnull public static final Metric DEFAULT_METRIC
    • DEFAULT_PRIMARY_CLUSTER_MIN

      public static final int DEFAULT_PRIMARY_CLUSTER_MIN
      See Also:
    • DEFAULT_PRIMARY_CLUSTER_MAX

      public static final int DEFAULT_PRIMARY_CLUSTER_MAX
      See Also:
    • DEFAULT_UNDERREPLICATED_PRIMARY_CLUSTER_MAX

      public static final int DEFAULT_UNDERREPLICATED_PRIMARY_CLUSTER_MAX
      See Also:
    • DEFAULT_REPLICATED_CLUSTER_MAX_WRITES

      public static final int DEFAULT_REPLICATED_CLUSTER_MAX_WRITES
      See Also:
    • DEFAULT_REPLICATED_CLUSTER_TARGET

      public static final int DEFAULT_REPLICATED_CLUSTER_TARGET
      See Also:
    • DEFAULT_REPLICATION_PRIORITY_MIN

      public static final double DEFAULT_REPLICATION_PRIORITY_MIN
      See Also:
    • DEFAULT_REPLICATION_DISTANCE_RATIO_WEIGHT

      public static final double DEFAULT_REPLICATION_DISTANCE_RATIO_WEIGHT
      See Also:
    • DEFAULT_REPLICATION_Z_SCORE_WEIGHT

      public static final double DEFAULT_REPLICATION_Z_SCORE_WEIGHT
      See Also:
    • DEFAULT_REPLICATION_STATS_MIN_SAMPLE_SIZE

      public static final int DEFAULT_REPLICATION_STATS_MIN_SAMPLE_SIZE
      See Also:
    • DEFAULT_SAMPLE_VECTOR_STATS_PROBABILITY

      public static final double DEFAULT_SAMPLE_VECTOR_STATS_PROBABILITY
      See Also:
    • DEFAULT_MAINTAIN_STATS_PROBABILITY

      public static final double DEFAULT_MAINTAIN_STATS_PROBABILITY
      See Also:
    • DEFAULT_STATS_THRESHOLD

      public static final int DEFAULT_STATS_THRESHOLD
      See Also:
    • DEFAULT_USE_RABITQ

      public static final boolean DEFAULT_USE_RABITQ
      See Also:
    • DEFAULT_RABITQ_NUM_EX_BITS

      public static final int DEFAULT_RABITQ_NUM_EX_BITS
      See Also:
    • DEFAULT_DETERMINISTIC_RANDOMNESS

      public static final boolean DEFAULT_DETERMINISTIC_RANDOMNESS
      See Also:
    • DEFAULT_SAMPLE_BATCH_SIZE

      public static final int DEFAULT_SAMPLE_BATCH_SIZE
      See Also:
    • DEFAULT_INSERT_MAX_CANDIDATE_CLUSTERS

      public static final int DEFAULT_INSERT_MAX_CANDIDATE_CLUSTERS
      See Also:
    • DEFAULT_DELETE_MAX_CANDIDATE_CLUSTERS

      public static final int DEFAULT_DELETE_MAX_CANDIDATE_CLUSTERS
      See Also:
    • DEFAULT_DELETE_CONCURRENCY

      public static final int DEFAULT_DELETE_CONCURRENCY
      See Also:
    • DEFAULT_SPLIT_NUM_NEAREST_CLUSTERS

      public static final int DEFAULT_SPLIT_NUM_NEAREST_CLUSTERS
      See Also:
    • DEFAULT_MERGE_NUM_NEAREST_CLUSTERS

      public static final int DEFAULT_MERGE_NUM_NEAREST_CLUSTERS
      See Also:
    • DEFAULT_KMEANS_MAX_ITERATIONS

      public static final int DEFAULT_KMEANS_MAX_ITERATIONS
      See Also:
    • DEFAULT_KMEANS_MAX_RESTARTS

      public static final int DEFAULT_KMEANS_MAX_RESTARTS
      See Also:
    • DEFAULT_REASSIGN_NUM_NEIGHBORING_CLUSTERS

      public static final int DEFAULT_REASSIGN_NUM_NEIGHBORING_CLUSTERS
      See Also:
    • DEFAULT_COLLAPSE_MIN_DUPLICATES

      public static final int DEFAULT_COLLAPSE_MIN_DUPLICATES
      See Also:
    • DEFAULT_SPLIT_MERGE_CONCURRENCY

      public static final int DEFAULT_SPLIT_MERGE_CONCURRENCY
      See Also:
    • DEFAULT_REASSIGN_CONCURRENCY

      public static final int DEFAULT_REASSIGN_CONCURRENCY
      See Also:
    • DEFAULT_COLLAPSE_CONCURRENCY

      public static final int DEFAULT_COLLAPSE_CONCURRENCY
      See Also:
    • DEFAULT_BOUNCE_CONCURRENCY

      public static final int DEFAULT_BOUNCE_CONCURRENCY
      See Also:
    • DEFAULT_CONSTRUCTION_SEARCH_CONFIG

      @Nonnull public static final SearchConfig DEFAULT_CONSTRUCTION_SEARCH_CONFIG
  • Constructor Details

    • Config

      public Config(@Nonnull Metric metric, int numDimensions, int primaryClusterMin, int primaryClusterMax, int underreplicatedPrimaryClusterMax, int replicatedClusterMaxWrites, int replicatedClusterTarget, double replicationPriorityMin, double replicationDistanceRatioWeight, double replicationZScoreWeight, int replicationStatsMinSampleSize, double sampleVectorStatsProbability, double maintainStatsProbability, int statsThreshold, boolean useRaBitQ, int raBitQNumExBits, boolean deterministicRandomness, int sampleBatchSize, int insertMaxCandidateClusters, int deleteMaxCandidateClusters, int deleteConcurrency, int splitNumNearestClusters, int mergeNumNearestClusters, int kMeansMaxIterations, int kMeansMaxRestarts, int reassignNumNeighboringClusters, int collapseMinDuplicates, int splitMergeConcurrency, int reassignConcurrency, int collapseConcurrency, int bounceConcurrency, @Nonnull SearchConfig constructionSearchConfig)
      Creates an instance of a Config record class.
      Parameters:
      metric - the value for the metric record component
      numDimensions - the value for the numDimensions record component
      primaryClusterMin - the value for the primaryClusterMin record component
      primaryClusterMax - the value for the primaryClusterMax record component
      underreplicatedPrimaryClusterMax - the value for the underreplicatedPrimaryClusterMax record component
      replicatedClusterMaxWrites - the value for the replicatedClusterMaxWrites record component
      replicatedClusterTarget - the value for the replicatedClusterTarget record component
      replicationPriorityMin - the value for the replicationPriorityMin record component
      replicationDistanceRatioWeight - the value for the replicationDistanceRatioWeight record component
      replicationZScoreWeight - the value for the replicationZScoreWeight record component
      replicationStatsMinSampleSize - the value for the replicationStatsMinSampleSize record component
      sampleVectorStatsProbability - the value for the sampleVectorStatsProbability record component
      maintainStatsProbability - the value for the maintainStatsProbability record component
      statsThreshold - the value for the statsThreshold record component
      useRaBitQ - the value for the useRaBitQ record component
      raBitQNumExBits - the value for the raBitQNumExBits record component
      deterministicRandomness - the value for the deterministicRandomness record component
      sampleBatchSize - the value for the sampleBatchSize record component
      insertMaxCandidateClusters - the value for the insertMaxCandidateClusters record component
      deleteMaxCandidateClusters - the value for the deleteMaxCandidateClusters record component
      deleteConcurrency - the value for the deleteConcurrency record component
      splitNumNearestClusters - the value for the splitNumNearestClusters record component
      mergeNumNearestClusters - the value for the mergeNumNearestClusters record component
      kMeansMaxIterations - the value for the kMeansMaxIterations record component
      kMeansMaxRestarts - the value for the kMeansMaxRestarts record component
      reassignNumNeighboringClusters - the value for the reassignNumNeighboringClusters record component
      collapseMinDuplicates - the value for the collapseMinDuplicates record component
      splitMergeConcurrency - the value for the splitMergeConcurrency record component
      reassignConcurrency - the value for the reassignConcurrency record component
      collapseConcurrency - the value for the collapseConcurrency record component
      bounceConcurrency - the value for the bounceConcurrency record component
      constructionSearchConfig - the value for the constructionSearchConfig record component
  • Method Details