org.apache.flink.connectors.hive.read

类 HiveContinuousMonitoringFunction

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.source.RichSourceFunction<TimestampedHiveInputSplit>

org.apache.flink.connectors.hive.read.HiveContinuousMonitoringFunction

所有已实现的接口:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.streaming.api.checkpoint.CheckpointedFunction, org.apache.flink.streaming.api.functions.source.SourceFunction<TimestampedHiveInputSplit>

public class HiveContinuousMonitoringFunction
extends org.apache.flink.streaming.api.functions.source.RichSourceFunction<TimestampedHiveInputSplit>
implements org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

This is the single (non-parallel) monitoring task which takes a HiveTableInputFormat, it is responsible for:

1. Monitoring partitions of hive meta store.
2. Deciding which partitions should be further read and processed.
3. Creating the splits corresponding to those partitions.
4. Assigning them to downstream tasks for further processing.

The splits to be read are forwarded to the downstream ContinuousFileReaderOperator which can have parallelism greater than one.

IMPORTANT NOTE: Splits are forwarded downstream for reading in ascending partition time order, based on the partition time of the partitions they belong to.

另请参阅:

序列化表格

嵌套类概要

从接口继承的嵌套类/接口 org.apache.flink.streaming.api.functions.source.SourceFunction

org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<T>

构造器概要

构造器

构造器和说明
HiveContinuousMonitoringFunction(HiveShim hiveShim, org.apache.hadoop.mapred.JobConf conf, org.apache.flink.table.catalog.ObjectPath tablePath, org.apache.flink.table.catalog.CatalogTable catalogTable, int readerParallelism, ConsumeOrder consumeOrder, String consumeOffset, String extractorKind, String extractorClass, String extractorPattern, long interval)

方法概要

限定符和类型	方法和说明
void	cancel()
void	close()
void	initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
void	run(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<TimestampedHiveInputSplit> context)
void	snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)

从类继承的方法 org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, open, setRuntimeContext

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

HiveContinuousMonitoringFunction

public HiveContinuousMonitoringFunction(HiveShim hiveShim,
                                        org.apache.hadoop.mapred.JobConf conf,
                                        org.apache.flink.table.catalog.ObjectPath tablePath,
                                        org.apache.flink.table.catalog.CatalogTable catalogTable,
                                        int readerParallelism,
                                        ConsumeOrder consumeOrder,
                                        String consumeOffset,
                                        String extractorKind,
                                        String extractorClass,
                                        String extractorPattern,
                                        long interval)

方法详细资料

initializeState

public void initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
                     throws Exception

指定者:

initializeState 在接口中 org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

抛出:

Exception

run

public void run(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<TimestampedHiveInputSplit> context)
         throws Exception

指定者:

run 在接口中 org.apache.flink.streaming.api.functions.source.SourceFunction<TimestampedHiveInputSplit>

抛出:

Exception

snapshotState

public void snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)
                   throws Exception

指定者:

snapshotState 在接口中 org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

抛出:

Exception

close

public void close()

指定者:

close 在接口中 org.apache.flink.api.common.functions.RichFunction

覆盖:

close 在类中 org.apache.flink.api.common.functions.AbstractRichFunction

cancel

public void cancel()

指定者:

cancel 在接口中 org.apache.flink.streaming.api.functions.source.SourceFunction<TimestampedHiveInputSplit>