org.apache.flink.connectors.hive

Class HiveSourceFileEnumerator

java.lang.Object

org.apache.flink.connectors.hive.HiveSourceFileEnumerator

All Implemented Interfaces:

org.apache.flink.connector.file.src.enumerate.FileEnumerator

public class HiveSourceFileEnumerator
extends Object
implements org.apache.flink.connector.file.src.enumerate.FileEnumerator

A FileEnumerator implementation for hive source, which generates splits based on HiveTablePartitions.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HiveSourceFileEnumerator.Provider A factory to create HiveSourceFileEnumerator.

Constructor Summary

Constructors

Constructor and Description
HiveSourceFileEnumerator(List<HiveTablePartition> partitions, org.apache.hadoop.mapred.JobConf jobConf)

Method Summary

Modifier and Type	Method and Description
static List<HiveSourceSplit>	createInputSplits(int minNumSplits, List<HiveTablePartition> partitions, org.apache.hadoop.mapred.JobConf jobConf)
Collection<org.apache.flink.connector.file.src.FileSourceSplit>	enumerateSplits(org.apache.flink.core.fs.Path[] paths, int minDesiredSplits)
static int	getNumFiles(List<HiveTablePartition> partitions, org.apache.hadoop.mapred.JobConf jobConf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HiveSourceFileEnumerator

public HiveSourceFileEnumerator(List<HiveTablePartition> partitions,
                                org.apache.hadoop.mapred.JobConf jobConf)

Method Detail

enumerateSplits

public Collection<org.apache.flink.connector.file.src.FileSourceSplit> enumerateSplits(org.apache.flink.core.fs.Path[] paths,
                                                                                       int minDesiredSplits)
                                                                                throws IOException

Specified by:

enumerateSplits in interface org.apache.flink.connector.file.src.enumerate.FileEnumerator

Throws:

IOException

createInputSplits

public static List<HiveSourceSplit> createInputSplits(int minNumSplits,
                                                      List<HiveTablePartition> partitions,
                                                      org.apache.hadoop.mapred.JobConf jobConf)
                                               throws IOException

Throws:

IOException

getNumFiles

public static int getNumFiles(List<HiveTablePartition> partitions,
                              org.apache.hadoop.mapred.JobConf jobConf)
                       throws IOException

Throws:

IOException