Package com.apple.foundationdb.linear

Class StoredVecsIterator<N extends Number,T>

java.lang.Object

com.google.common.collect.UnmodifiableIterator<T>

com.google.common.collect.AbstractIterator<T>

com.apple.foundationdb.linear.StoredVecsIterator<N,T>

Type Parameters:

N - the component type of the vectors which must extends Number

T - the type of object this iterator creates and uses to represent a stored vector in memory

All Implemented Interfaces:

Iterator<T>

Direct Known Subclasses:

StoredVecsIterator.StoredFVecsIterator, StoredVecsIterator.StoredIVecsIterator

public abstract class StoredVecsIterator<N extends Number,T>
extends com.google.common.collect.AbstractIterator<T>

Abstract iterator implementation to read the IVecs/FVecs data format that is used by publicly available embedding datasets.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	StoredVecsIterator.StoredFVecsIterator	Iterator to read floating point vectors from a FileChannel providing an iterator of DoubleRealVectors.
static class	StoredVecsIterator.StoredIVecsIterator	Iterator to read vectors from a FileChannel into a list of integers.

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	StoredVecsIterator(FileChannel fileChannel)

Method Summary

Modifier and Type	Method	Description
protected T	computeNext()
protected abstract N[]	newComponentArray(int size)
protected abstract N	toComponent(ByteBuffer byteBuffer)
protected abstract T	toTarget(N[] components)

Methods inherited from class com.google.common.collect.AbstractIterator

endOfData, hasNext, next, peek

Methods inherited from class com.google.common.collect.UnmodifiableIterator

remove

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Constructor Details

StoredVecsIterator

protected StoredVecsIterator(@Nonnull
 FileChannel fileChannel)

Method Details

newComponentArray

@Nonnull
protected abstract N[] newComponentArray(int size)

toComponent

@Nonnull
protected abstract N toComponent(@Nonnull
 ByteBuffer byteBuffer)

toTarget

@Nonnull
protected abstract T toTarget(@Nonnull
 N[] components)

computeNext

@Nullable
protected T computeNext()

Specified by:

computeNext in class com.google.common.collect.AbstractIterator<T>