Package com.apple.foundationdb.async.guardiann

Class Guardiann

java.lang.Object
com.apple.foundationdb.async.guardiann.Guardiann
@API(EXPERIMENTAL) public class Guardiann extends Object
A transactional approximate nearest neighbor (ANN) vector structure built on top of FoundationDB. Guardiann organizes vectors into clusters, each represented by a centroid stored in an HNSW graph for fast centroid lookup, and maintains per-cluster metadata (vector counts, running statistics, replication state) to support incremental maintenance.

Architecture

Vectors are partitioned into clusters. Each cluster stores a set of primary vector references (the authoritative copies) and replicated references (copies from neighboring clusters kept for improved search recall). Cluster centroids are indexed in an HNSW graph so that the nearest clusters to a query vector can be found efficiently.

Operations

Maintenance

Structural maintenance is performed lazily via deferred tasks that are executed piggy-backed on insert and delete operations:

  • Split/Merge — when a cluster grows beyond Config.primaryClusterMax() or shrinks below Config.primaryClusterMin(), a SplitMergeTask repartitions the affected clusters using bounded k-means and updates the HNSW centroid index.
  • Reassign — a ReassignTask recomputes vector assignments and replication for a cluster, fixing underreplicated vectors and cleaning up excess replicas.
  • Collapse — a CollapseTask detects large groups of identical vectors and replaces them with a single collapsed representative to reduce storage.
  • Bounce — a BounceTask coordinates execution order between dependent tasks, ensuring prerequisite tasks complete before follow-up tasks are created.

Usage

This class is the primary entry point for interacting with the Guardiann vector structure. It delegates to specialized operation classes (Insert, Search, Delete) which implement the respective algorithms. All operations are asynchronous and designed for use within FoundationDB transactions.

See Also:

  • Constructor Details

    • Guardiann

      public Guardiann(@Nonnull Subspace subspace, @Nonnull Executor executor, @Nonnull Config config, @Nonnull OnWriteListener onWriteListener, @Nonnull OnReadListener onReadListener)
      Constructs a new Guardiann vector structure instance.

      Initializes the structure with the necessary components for storage, execution, configuration, and event handling. All parameters are mandatory and must not be null.

      Parameters:
      subspace - the Subspace where the vector structure data is stored
      executor - the Executor service to use for concurrent operations
      config - the Config object containing algorithm and storage parameters
      onWriteListener - a listener to be notified of write events
      onReadListener - a listener to be notified of read events

  • Method Details

    • newConfigBuilder

      public static Config.ConfigBuilder newConfigBuilder()
      Start building a Config.
      Returns:
      a new Config that can be altered and then built for use with a Guardiann
      See Also:

    • defaultConfig

      @Nonnull public static Config defaultConfig(int numDimensions)
      Returns a default Config.
      Parameters:
      numDimensions - number of dimensions
      Returns:
      a new default Config.
      See Also:

    • getLocator

      @Nonnull public Locator getLocator()

    • getSubspace

      @Nonnull public Subspace getSubspace()
      Gets the subspace associated with this object.
      Returns:
      the non-null subspace

    • getExecutor

      @Nonnull public Executor getExecutor()
      Get the executor used by this object.
      Returns:
      executor used when running asynchronous tasks

    • getConfig

      @Nonnull public Config getConfig()
      Get this object's configuration.
      Returns:
      the configuration

    • getOnWriteListener

      @Nonnull public OnWriteListener getOnWriteListener()
      Get the on-write listener.
      Returns:
      the on-write listener

    • getOnReadListener

      @Nonnull public OnReadListener getOnReadListener()
      Get the on-read listener.
      Returns:
      the on-read listener

    • kNearestNeighborsSearch

      @Nonnull public CompletableFuture<List<? extends ResultEntry>> kNearestNeighborsSearch(@Nonnull ReadTransaction readTransaction, int k, @Nonnull SearchConfig searchConfig, boolean includeVectors, @Nonnull RealVector queryVector)
      Performs a search for the k-nearest neighbors for a given query vector.
      Parameters:
      readTransaction - the transaction to use for reading from the database
      k - the number of nearest neighbors to return
      searchConfig - the performance/recall tuning knobs for this search (see SearchConfig)
      includeVectors - indicator if the caller would like the search to also include vectors in the result set
      queryVector - the vector to find the nearest neighbors of
      Returns:
      a CompletableFuture that will complete with a list of the k nearest neighbors, sorted by distance in ascending order.

    • insert

      @Nonnull public CompletableFuture<Void> insert(@Nonnull Transaction transaction, @Nonnull Tuple newPrimaryKey, @Nonnull RealVector newVector, @Nullable Tuple additionalValues)
      Inserts a new vector with its associated primary key into the vector structure.

      Finds the nearest cluster(s) for the new vector by querying the HNSW centroid index, then writes a primary vector reference to the nearest cluster and replicated references to neighboring clusters based on replication priority scoring. Updates cluster metadata and enqueues deferred maintenance tasks (split, merge, reassign) if cluster size invariants are violated.

      Parameters:
      transaction - the Transaction context for all database operations
      newPrimaryKey - the unique Tuple primary key for the new vector being inserted
      newVector - the RealVector data to be inserted
      additionalValues - additional values to be associated with the new vector/record, or null
      Returns:
      a CompletableFuture that completes when the insertion operation is finished

    • delete

      @Nonnull public CompletableFuture<Void> delete(@Nonnull Transaction transaction, @Nonnull Tuple primaryKey, @Nonnull RealVector vector)
      Deletes a vector identified by its primary key from the Guardiann index.

      Removes the vector's references from all clusters it belongs to (primary and replicated), updates cluster metadata, and enqueues maintenance tasks if needed. The caller must provide the vector data to locate which clusters contain the vector.

      Parameters:
      transaction - the Transaction context for all database operations
      primaryKey - the unique Tuple primary key of the vector to delete
      vector - the RealVector data of the vector being deleted
      Returns:
      a CompletableFuture that completes when the deletion is finished