org.apache.arrow.algorithm.dictionary

Class HashTableBasedDictionaryBuilder<V extends ElementAddressableVector>

java.lang.Object

org.apache.arrow.algorithm.dictionary.HashTableBasedDictionaryBuilder<V>

Type Parameters:

V - the dictionary vector type.

public class HashTableBasedDictionaryBuilder<V extends ElementAddressableVector>
extends Object

A dictionary builder is intended for the scenario frequently encountered in practice: the dictionary is not known a priori, so it is generated dynamically. In particular, when a new value arrives, it is tested to check if it is already in the dictionary. If so, it is simply neglected, otherwise, it is added to the dictionary.

This class builds the dictionary based on a hash table. Each add operation can be finished in O(1) time, where n is the current dictionary size.

The dictionary builder is intended to build a single dictionary. So it cannot be used for different dictionaries.

Below gives the sample code for using the dictionary builder

 HashTableBasedDictionaryBuilder dictionaryBuilder = ...
 ...
 dictionaryBuild.addValue(newValue);
 ...
 

With the above code, the dictionary vector will be populated, and it can be retrieved by the getDictionary() method. After that, dictionary encoding can proceed with the populated dictionary encoder.

Constructor Summary

Constructors

Constructor and Description
HashTableBasedDictionaryBuilder(V dictionary) Constructs a hash table based dictionary builder.
HashTableBasedDictionaryBuilder(V dictionary, boolean encodeNull) Constructs a hash table based dictionary builder.
HashTableBasedDictionaryBuilder(V dictionary, boolean encodeNull, ArrowBufHasher hasher) Constructs a hash table based dictionary builder.

Method Summary

Modifier and Type	Method and Description
int	addValue(V targetVector, int targetIndex) Try to add an element from the target vector to the dictionary.
int	addValues(V targetVector) Try to add all values from the target vector to the dictionary.
V	getDictionary() Gets the dictionary built.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HashTableBasedDictionaryBuilder

public HashTableBasedDictionaryBuilder(V dictionary)

Constructs a hash table based dictionary builder.

Parameters:

dictionary - the dictionary to populate.

HashTableBasedDictionaryBuilder

public HashTableBasedDictionaryBuilder(V dictionary,
                                       boolean encodeNull)

Constructs a hash table based dictionary builder.

Parameters:

dictionary - the dictionary to populate.

encodeNull - if null values should be added to the dictionary.

HashTableBasedDictionaryBuilder

public HashTableBasedDictionaryBuilder(V dictionary,
                                       boolean encodeNull,
                                       ArrowBufHasher hasher)

Constructs a hash table based dictionary builder.

Parameters:

dictionary - the dictionary to populate.

encodeNull - if null values should be added to the dictionary.

hasher - the hasher used to compute the hash code.

Method Detail

getDictionary

public V getDictionary()

Gets the dictionary built.

Returns:

the dictionary.

addValues

public int addValues(V targetVector)

Try to add all values from the target vector to the dictionary.

Parameters:

targetVector - the target vector containing values to probe.

Returns:

the number of values actually added to the dictionary.

addValue

public int addValue(V targetVector,
                    int targetIndex)

Try to add an element from the target vector to the dictionary.

Parameters:

targetVector - the target vector containing new element.

targetIndex - the index of the new element in the target vector.

Returns:

the index of the new element in the dictionary.