org.apache.flink.connectors.hive

Class HiveTableSource.HiveContinuousPartitionFetcherContext<T extends Comparable<T>>

java.lang.Object

org.apache.flink.connectors.hive.read.HivePartitionFetcherContextBase<org.apache.hadoop.hive.metastore.api.Partition>

org.apache.flink.connectors.hive.HiveTableSource.HiveContinuousPartitionFetcherContext<T>

All Implemented Interfaces:

Serializable, org.apache.flink.table.filesystem.ContinuousPartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition,T>, org.apache.flink.table.filesystem.PartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition>

Enclosing class:

HiveTableSource

public static class HiveTableSource.HiveContinuousPartitionFetcherContext<T extends Comparable<T>>
extends HivePartitionFetcherContextBase<org.apache.hadoop.hive.metastore.api.Partition>
implements org.apache.flink.table.filesystem.ContinuousPartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition,T>

PartitionFetcher.Context for ContinuousPartitionFetcher.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.table.filesystem.PartitionFetcher.Context

org.apache.flink.table.filesystem.PartitionFetcher.Context.ComparablePartitionValue<P,T extends Comparable<T>>

Field Summary

Fields inherited from class org.apache.flink.connectors.hive.read.HivePartitionFetcherContextBase

configuration, confWrapper, consumeOrder, defaultPartitionName, fieldNames, fieldTypes, fs, hiveShim, metaStoreClient, partitionKeys, tableLocation, tablePath, tableProps, tableSd

Constructor Summary

Constructors

Constructor and Description
HiveContinuousPartitionFetcherContext(org.apache.flink.table.catalog.ObjectPath tablePath, HiveShim hiveShim, JobConfWrapper confWrapper, List<String> partitionKeys, org.apache.flink.table.types.DataType[] fieldTypes, String[] fieldNames, org.apache.flink.configuration.Configuration configuration, String defaultPartitionName)

Method Summary

Modifier and Type	Method and Description
void	close()
T	getConsumeStartOffset()
long	getModificationTime(org.apache.hadoop.hive.metastore.api.Partition partition, T partitionOffset) Get the partition modified time.
Optional<org.apache.hadoop.hive.metastore.api.Partition>	getPartition(List<String> partValues)
org.apache.flink.table.catalog.ObjectPath	getTablePath()
org.apache.flink.api.common.typeutils.TypeSerializer<T>	getTypeSerializer()
HiveTablePartition	toHiveTablePartition(org.apache.hadoop.hive.metastore.api.Partition partition) Convert partition to HiveTablePartition.

Methods inherited from class org.apache.flink.connectors.hive.read.HivePartitionFetcherContextBase

getComparablePartitionValueList, open

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.filesystem.PartitionFetcher.Context

getComparablePartitionValueList, open

Constructor Detail

HiveContinuousPartitionFetcherContext

public HiveContinuousPartitionFetcherContext(org.apache.flink.table.catalog.ObjectPath tablePath,
                                             HiveShim hiveShim,
                                             JobConfWrapper confWrapper,
                                             List<String> partitionKeys,
                                             org.apache.flink.table.types.DataType[] fieldTypes,
                                             String[] fieldNames,
                                             org.apache.flink.configuration.Configuration configuration,
                                             String defaultPartitionName)

Method Detail

getPartition

public Optional<org.apache.hadoop.hive.metastore.api.Partition> getPartition(List<String> partValues)
                                                                      throws org.apache.thrift.TException

Specified by:

getPartition in interface org.apache.flink.table.filesystem.PartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition>

Throws:

org.apache.thrift.TException

getTablePath

public org.apache.flink.table.catalog.ObjectPath getTablePath()

Specified by:

getTablePath in interface org.apache.flink.table.filesystem.ContinuousPartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition,T extends Comparable<T>>

getModificationTime

public long getModificationTime(org.apache.hadoop.hive.metastore.api.Partition partition,
                                T partitionOffset)

Get the partition modified time.

the time is the the folder/file modification time in filesystem when fetched in create-time order, the time is extracted from partition name when fetched in partition-time order, the time is partion create time in metaStore when fetched in partition-name order.

toHiveTablePartition

public HiveTablePartition toHiveTablePartition(org.apache.hadoop.hive.metastore.api.Partition partition)

Convert partition to HiveTablePartition.

getTypeSerializer

public org.apache.flink.api.common.typeutils.TypeSerializer<T> getTypeSerializer()

Specified by:

getTypeSerializer in interface org.apache.flink.table.filesystem.ContinuousPartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition,T extends Comparable<T>>

getConsumeStartOffset

public T getConsumeStartOffset()

Specified by:

getConsumeStartOffset in interface org.apache.flink.table.filesystem.ContinuousPartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition,T extends Comparable<T>>

close

public void close()
           throws Exception

Specified by:

close in interface org.apache.flink.table.filesystem.PartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition>

Overrides:

close in class HivePartitionFetcherContextBase<org.apache.hadoop.hive.metastore.api.Partition>

Throws:

Exception