Package org.apache.pinot.tools.segment.converter

Class DictionaryToRawIndexConverter

java.lang.Object

org.apache.pinot.tools.segment.converter.DictionaryToRawIndexConverter

public class DictionaryToRawIndexConverter
extends Object

Class to convert segment with dictionary encoded column to raw index (without dictionary).

Constructor Summary

Constructors

Constructor	Description
DictionaryToRawIndexConverter()

Method Summary

Modifier and Type	Method	Description
boolean	convert()	Method to perform the conversion for a set of segments in the _dataDir
boolean	convertSegment(File segmentDir, String[] columns, File outputDir, boolean compressOutput)	This method converts the specified columns of the given segment from dictionary encoded forward index to raw index without dictionary.
static void	main(String[] args)	Main method for the class.
void	printUsage()	Helper method to print usage at the command line interface.
DictionaryToRawIndexConverter	setColumns(String columns)	Setter for columns to convert.
DictionaryToRawIndexConverter	setDataDir(String dataDir)	Setter for _dataDir
DictionaryToRawIndexConverter	setOutputDir(String outputDir)	Setter for _outputDir
DictionaryToRawIndexConverter	setOverwrite(boolean overwrite)	Setter for _overwrite When set to true, already existing output directory is overwritten.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DictionaryToRawIndexConverter

public DictionaryToRawIndexConverter()

Method Detail

setDataDir

public DictionaryToRawIndexConverter setDataDir(String dataDir)

Setter for _dataDir

Parameters:

dataDir - Data directory containing un-tarred segments.

Returns:

this

setOutputDir

public DictionaryToRawIndexConverter setOutputDir(String outputDir)

Setter for _outputDir

Parameters:

outputDir - Directory where output segments should be written

Returns:

this

setColumns

public DictionaryToRawIndexConverter setColumns(String columns)

Setter for columns to convert.

Parameters:

columns - Comma separated list of columns

Returns:

this

setOverwrite

public DictionaryToRawIndexConverter setOverwrite(boolean overwrite)

Setter for _overwrite When set to true, already existing output directory is overwritten.

Parameters:

overwrite - True for overwriting existing output dir, False otherwise

Returns:

this

convert

public boolean convert()
                throws Exception

Method to perform the conversion for a set of segments in the _dataDir

Returns:

True if successful, False otherwise

Throws:

Exception

convertSegment

public boolean convertSegment(File segmentDir,
                              String[] columns,
                              File outputDir,
                              boolean compressOutput)
                       throws Exception

This method converts the specified columns of the given segment from dictionary encoded forward index to raw index without dictionary.

Parameters:

segmentDir - Segment directory

columns - Columns to convert

outputDir - Directory for writing output segment

compressOutput - Tar/gzip the output segment

Returns:

True if successful, False otherwise

Throws:

Exception

printUsage

public void printUsage()

Helper method to print usage at the command line interface.

main

public static void main(String[] args)
                 throws Exception

Main method for the class.

Parameters:

args - Arguments for the converter

Throws:

Exception