Package com.apple.foundationdb.record.lucene

Class LucenePartitioner

java.lang.Object
com.apple.foundationdb.record.lucene.LucenePartitioner
@API(EXPERIMENTAL) public class LucenePartitioner extends Object
Manage partitioning info for a logical, partitioned lucene index, in which each partition is a separate physical lucene index.

  • Field Details

    • DEFAULT_PARTITION_LOW_WATERMARK

      public static final int DEFAULT_PARTITION_LOW_WATERMARK
      See Also:

    • PARTITION_META_SUBSPACE

      public static final int PARTITION_META_SUBSPACE
      See Also:

    • PARTITION_DATA_SUBSPACE

      public static final int PARTITION_DATA_SUBSPACE
      See Also:

  • Constructor Details

  • Method Details

    • selectQueryPartitionId

      @Nullable public Integer selectQueryPartitionId(@Nonnull Tuple groupKey)
      return the partition ID on which to run a query, given a grouping key. For now, the most recent partition is returned.
      Parameters:
      groupKey - group key
      Returns:
      partition id, or null if partitioning isn't enabled

    • selectQueryPartition

      public com.apple.foundationdb.record.lucene.LucenePartitioner.PartitionedQueryHint selectQueryPartition(@Nonnull Tuple groupKey, @Nullable LuceneScanQuery luceneScanQuery)
      Synchronous counterpart of selectQueryPartitionAsync(Tuple, LuceneScanQuery).
      Parameters:
      groupKey - group key
      luceneScanQuery - query
      Returns:
      partition query hint, or null if partitioning isn't enabled or no partitioning metadata exist for the given query

    • selectQueryPartitionAsync

      public CompletableFuture<com.apple.foundationdb.record.lucene.LucenePartitioner.PartitionedQueryHint> selectQueryPartitionAsync(@Nonnull Tuple groupKey, @Nullable LuceneScanQuery luceneScanQuery)
      Return the partition ID on which to run a query, given a grouping key. If no partitioning field predicate can be used to determine a particular starting partition, the most recent partition is returned, unless the query is sorted by the partitioning field in ascending order, in which case the oldest partition is returned. If the query contains a top level partitioning field predicate in conjunction with the rest of the predicates, and the predicate can be used to determine a specific starting partition, that partition will be computed and returned.
      Parameters:
      groupKey - group key
      luceneScanQuery - query
      Returns:
      partition query hint, or null if partitioning isn't enabled or no partitioning metadata exist for the given query

    • isSortedByPartitionField

      @Nonnull public com.apple.foundationdb.record.lucene.LucenePartitioner.PartitionedSortContext isSortedByPartitionField(@Nonnull org.apache.lucene.search.Sort sort)
      checks whether the provided Sort is by the partitioning field and whether it's in reverse order.
      Parameters:
      sort - sort
      Returns:
      PartitionedSortContext object

    • isPartitioningEnabled

      public boolean isPartitioningEnabled()
      get whether this index has partitioning enabled.
      Returns:
      true if partitioning is enabled

    • getPartitionFieldNameInLucene

      @Nullable public String getPartitionFieldNameInLucene()
      get the document field name the contains the document partition field, as it is stored in Lucene.
      Returns:
      Lucene document field name, or null

    • addToAndSavePartitionMetadata

      @Nonnull public <M extends Message> CompletableFuture<Integer> addToAndSavePartitionMetadata(@Nonnull FDBIndexableRecord<M> newRecord, @Nonnull Tuple groupingKey, @Nullable Integer assignedPartitionId)
      add a new written record to its partition metadata.
      Type Parameters:
      M - message
      Parameters:
      newRecord - record to be written
      groupingKey - grouping key
      assignedPartitionId - assigned partition override, if not null
      Returns:
      partition id or null if partitioning isn't enabled on index

    • rebalancePartitions

      @Nonnull public CompletableFuture<RecordCursorContinuation> rebalancePartitions(RecordCursorContinuation start, int documentCount, LucenePartitioner.RepartitioningLogMessages logMessages)
      Re-balance full partitions, if applicable.
      Parameters:
      start - The continuation at which to resume rebalancing, as returned from a previous call to rebalancePartitions.
      documentCount - max number of documents to move in each transaction
      logMessages - FDBDatabaseRunner additional log messages
      Returns:
      a continuation at which to resume rebalancing in another call to rebalancePartitions

    • processPartitionRebalancing

      @Nonnull public CompletableFuture<Integer> processPartitionRebalancing(@Nonnull Tuple groupingKey, int repartitionDocumentCount, LucenePartitioner.RepartitioningLogMessages logMessages)
      Re-balance the first partition in a given grouping key by moving documents out of it. Note that in order to finish the task within the bounds of a single transaction, only the first partition needing re-balancing will be processed. If there are other partitions that need re-balancing, they will be processed during subsequent calls.
      Parameters:
      groupingKey - grouping key
      repartitionDocumentCount - max number of documents to move in each transaction
      logMessages - FDBDatabaseRunner additional log messages
      Returns:
      future count of documents rebalanced in this group. If zero, no more re-balancing needed

    • getNewestNDocuments

      @Nonnull public LuceneRecordCursor getNewestNDocuments(@Nonnull com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo partitionInfo, @Nonnull Tuple groupingKey, int count)
      get the newest N index entries in a given Lucene partition.
      Parameters:
      partitionInfo - partition metadata
      groupingKey - grouping key
      count - count of index entries to return
      Returns:
      cursor over the N (or fewer) newest index entries

    • getOldestNDocuments

      @Nonnull public LuceneRecordCursor getOldestNDocuments(@Nonnull com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo partitionInfo, @Nonnull Tuple groupingKey, int count)
      get the oldest N index entries in a given Lucene partition.
      Parameters:
      partitionInfo - partition metadata
      groupingKey - grouping key
      count - count of index entries to return
      Returns:
      cursor over the N (or fewer) oldest index entries

    • getAllPartitionMetaInfo

      public CompletableFuture<List<com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo>> getAllPartitionMetaInfo(@Nonnull Tuple groupingKey)
      Get all partition metadata for a given grouping key.
      Parameters:
      groupingKey - grouping key
      Returns:
      future list of partition metadata

    • getPartitionMetaInfoById

      @Nonnull public CompletableFuture<com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo> getPartitionMetaInfoById(int partitionId, @Nonnull Tuple groupingKey)
      find the partition metadata for a given partition id.
      Parameters:
      partitionId - partition id
      groupingKey - grouping key
      Returns:
      future of: partition info, or null if not found

    • getNextOlderPartitionInfo

      @Nonnull public static CompletableFuture<com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo> getNextOlderPartitionInfo(@Nonnull FDBRecordContext context, @Nonnull Tuple groupingKey, @Nullable Tuple previousKey, @Nonnull Subspace indexSubspace)
      get the next "older" partition for a given partition key.
      Parameters:
      context - FDB record context
      groupingKey - grouping key
      previousKey - partition key
      indexSubspace - index subspace
      Returns:
      partition future

    • getNextNewerPartitionInfo

      @Nonnull public static CompletableFuture<com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo> getNextNewerPartitionInfo(@Nonnull FDBRecordContext context, @Nonnull Tuple groupingKey, @Nullable Tuple currentPartitionKey, @Nonnull Subspace indexSubspace)
      get the next "newer" partition for a given partition key.
      Parameters:
      context - FDB record context
      groupingKey - grouping key
      currentPartitionKey - current partition key
      indexSubspace - index subspace
      Returns:
      partition future

    • toPartitionKey

      @Nonnull public Tuple toPartitionKey(@Nonnull Object partitioningFieldValue, @Nonnull Tuple primaryKey)
      convenience function that builds a partitioning key value from a partitioning field value and a primary key.
      Parameters:
      partitioningFieldValue - partitioning field value
      primaryKey - record primary key
      Returns:
      partitioning key value tuple

    • isOlderThan

      public static boolean isOlderThan(@Nonnull Tuple key, @Nonnull com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo partitionInfo)
      convenience function that evaluates whether a given partitioning key tuple is "older" than the given partitioning metadata.
      Parameters:
      key - partitioning key tuple
      partitionInfo - partitioning meta data
      Returns:
      true if key is "older" than partitionInfo

    • isPrefixOlderThanPartition

      public static boolean isPrefixOlderThanPartition(@Nonnull Tuple prefix, @Nonnull com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo partitionInfo)

    • isNewerThan

      public static boolean isNewerThan(@Nonnull Tuple key, @Nonnull com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo partitionInfo)
      convenience function that evaluates whether a given partitioning key tuple is "newer" than the given partitioning metadata.
      Parameters:
      key - partitioning key tuple
      partitionInfo - partitioning meta data
      Returns:
      true if key is "newer" than partitionInfo

    • getPartitionKey

      @Nonnull public static Tuple getPartitionKey(@Nonnull com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo partitionInfo)
      convenience function that returns the partitioning key of a given partition metadata object.
      Parameters:
      partitionInfo - partition metadata
      Returns:
      partition key tuple

    • getToTuple

      @Nonnull public static Tuple getToTuple(@Nonnull com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo partitionInfo)
      convenience function that returns the to value of a given partition metadata object.
      Parameters:
      partitionInfo - partition metadata
      Returns:
      to tuple

    • getPartitionNeighbors

      @Nonnull public static com.apple.foundationdb.record.util.pair.Pair<com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo,com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo> getPartitionNeighbors(@Nonnull List<com.apple.foundationdb.record.lucene.LucenePartitionInfoProto.LucenePartitionInfo> allPartitions, int currentPartitionPosition)
      return the neighbors of a given partition in a list.
      Parameters:
      allPartitions - all partitions
      currentPartitionPosition - current partition's position in the list
      Returns:
      pair of left and right neighbors