org.apache.flink.connectors.hive

Class ContinuousHiveSplitEnumerator<T extends Comparable<T>>

java.lang.Object

org.apache.flink.connectors.hive.ContinuousHiveSplitEnumerator<T>

All Implemented Interfaces:

AutoCloseable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.api.connector.source.SplitEnumerator<HiveSourceSplit,org.apache.flink.connector.file.src.PendingSplitsCheckpoint<HiveSourceSplit>>

public class ContinuousHiveSplitEnumerator<T extends Comparable<T>>
extends Object
implements org.apache.flink.api.connector.source.SplitEnumerator<HiveSourceSplit,org.apache.flink.connector.file.src.PendingSplitsCheckpoint<HiveSourceSplit>>

A continuously monitoring SplitEnumerator for hive source.

Constructor Summary

Constructors

Constructor and Description

ContinuousHiveSplitEnumerator(org.apache.flink.api.connector.source.SplitEnumeratorContext<HiveSourceSplit> enumeratorContext, T currentReadOffset, Collection<List<String>> seenPartitionsSinceOffset, org.apache.flink.connector.file.src.assigners.FileSplitAssigner splitAssigner, long discoveryInterval, org.apache.hadoop.mapred.JobConf jobConf, org.apache.flink.table.catalog.ObjectPath tablePath, org.apache.flink.table.filesystem.ContinuousPartitionFetcher<org.apache.hadoop.hive.metastore.api.Partition,T> fetcher, HiveTableSource.HiveContinuousPartitionFetcherContext<T> fetcherContext)

Method Summary

Modifier and Type	Method and Description
void	addReader(int subtaskId)
void	addSplitsBack(List<HiveSourceSplit> splits, int subtaskId)
void	close()
void	handleSourceEvent(int subtaskId, org.apache.flink.api.connector.source.SourceEvent sourceEvent)
void	handleSplitRequest(int subtaskId, String hostName)
org.apache.flink.connector.file.src.PendingSplitsCheckpoint<HiveSourceSplit>	snapshotState()
void	start()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.connector.source.SplitEnumerator

notifyCheckpointComplete

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted

Constructor Detail

ContinuousHiveSplitEnumerator

public ContinuousHiveSplitEnumerator(org.apache.flink.api.connector.source.SplitEnumeratorContext<HiveSourceSplit> enumeratorContext,
                                     T currentReadOffset,
                                     Collection<List<String>> seenPartitionsSinceOffset,
                                     org.apache.flink.connector.file.src.assigners.FileSplitAssigner splitAssigner,
                                     long discoveryInterval,
                                     org.apache.hadoop.mapred.JobConf jobConf,
                                     org.apache.flink.table.catalog.ObjectPath tablePath,
                                     org.apache.flink.table.filesystem.ContinuousPartitionFetcher<org.apache.hadoop.hive.metastore.api.Partition,T> fetcher,
                                     HiveTableSource.HiveContinuousPartitionFetcherContext<T> fetcherContext)

Method Detail

start

public void start()

Specified by:

start in interface org.apache.flink.api.connector.source.SplitEnumerator<HiveSourceSplit,org.apache.flink.connector.file.src.PendingSplitsCheckpoint<HiveSourceSplit>>

handleSplitRequest

public void handleSplitRequest(int subtaskId,
                               @Nullable
                               String hostName)

Specified by:

handleSplitRequest in interface org.apache.flink.api.connector.source.SplitEnumerator<HiveSourceSplit,org.apache.flink.connector.file.src.PendingSplitsCheckpoint<HiveSourceSplit>>

handleSourceEvent

public void handleSourceEvent(int subtaskId,
                              org.apache.flink.api.connector.source.SourceEvent sourceEvent)

Specified by:

handleSourceEvent in interface org.apache.flink.api.connector.source.SplitEnumerator<HiveSourceSplit,org.apache.flink.connector.file.src.PendingSplitsCheckpoint<HiveSourceSplit>>

addSplitsBack

public void addSplitsBack(List<HiveSourceSplit> splits,
                          int subtaskId)

Specified by:

addSplitsBack in interface org.apache.flink.api.connector.source.SplitEnumerator<HiveSourceSplit,org.apache.flink.connector.file.src.PendingSplitsCheckpoint<HiveSourceSplit>>

addReader

public void addReader(int subtaskId)

Specified by:

addReader in interface org.apache.flink.api.connector.source.SplitEnumerator<HiveSourceSplit,org.apache.flink.connector.file.src.PendingSplitsCheckpoint<HiveSourceSplit>>

snapshotState

public org.apache.flink.connector.file.src.PendingSplitsCheckpoint<HiveSourceSplit> snapshotState()
                                                                                           throws Exception

Specified by:

snapshotState in interface org.apache.flink.api.connector.source.SplitEnumerator<HiveSourceSplit,org.apache.flink.connector.file.src.PendingSplitsCheckpoint<HiveSourceSplit>>

Throws:

Exception

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface org.apache.flink.api.connector.source.SplitEnumerator<HiveSourceSplit,org.apache.flink.connector.file.src.PendingSplitsCheckpoint<HiveSourceSplit>>

Throws:

IOException