org.apache.flink.connectors.hive

Class HiveLookupTableSource

java.lang.Object

org.apache.flink.connectors.hive.HiveTableSource

org.apache.flink.connectors.hive.HiveLookupTableSource

All Implemented Interfaces:

org.apache.flink.table.connector.source.abilities.SupportsLimitPushDown, org.apache.flink.table.connector.source.abilities.SupportsPartitionPushDown, org.apache.flink.table.connector.source.abilities.SupportsProjectionPushDown, org.apache.flink.table.connector.source.DynamicTableSource, org.apache.flink.table.connector.source.LookupTableSource, org.apache.flink.table.connector.source.ScanTableSource

public class HiveLookupTableSource
extends HiveTableSource
implements org.apache.flink.table.connector.source.LookupTableSource

Hive Table Source that has lookup ability.

Hive Table source has both lookup and continuous read ability, when it acts as continuous read source it does not have the lookup ability but can be a temporal table just like other stream sources. When it acts as bounded table, it has the lookup ability.

A common user case is use hive table as dimension table and always lookup the latest partition data, in this case, hive table source is a continuous read source but currently we implements it by LookupFunction. Because currently TableSource can not tell the downstream when the latest partition has been read finished. This is a temporarily workaround and will re-implement in the future.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.connectors.hive.HiveTableSource

HiveTableSource.HiveContinuousPartitionFetcherContext<T extends Comparable<T>>

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.LookupTableSource

org.apache.flink.table.connector.source.LookupTableSource.LookupContext, org.apache.flink.table.connector.source.LookupTableSource.LookupRuntimeProvider

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.DynamicTableSource

org.apache.flink.table.connector.source.DynamicTableSource.Context, org.apache.flink.table.connector.source.DynamicTableSource.DataStructureConverter

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.ScanTableSource

org.apache.flink.table.connector.source.ScanTableSource.ScanContext, org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider

Field Summary

Fields inherited from class org.apache.flink.connectors.hive.HiveTableSource

catalogTable, flinkConf, hiveShim, hiveVersion, jobConf, projectedFields, tablePath

Constructor Summary

Constructors

Constructor and Description
HiveLookupTableSource(org.apache.hadoop.mapred.JobConf jobConf, org.apache.flink.configuration.ReadableConfig flinkConf, org.apache.flink.table.catalog.ObjectPath tablePath, org.apache.flink.table.catalog.CatalogTable catalogTable)

Method Summary

Modifier and Type	Method and Description
org.apache.flink.table.connector.source.LookupTableSource.LookupRuntimeProvider	getLookupRuntimeProvider(org.apache.flink.table.connector.source.LookupTableSource.LookupContext context)

Methods inherited from class org.apache.flink.connectors.hive.HiveTableSource

applyLimit, applyPartitions, applyProjection, asSummaryString, copy, getChangelogMode, getDataStream, getProducedTableSchema, getScanRuntimeProvider, getTableSchema, isStreamingSource, listPartitions, supportsNestedProjection

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.connector.source.DynamicTableSource

asSummaryString, copy

Constructor Detail

HiveLookupTableSource

public HiveLookupTableSource(org.apache.hadoop.mapred.JobConf jobConf,
                             org.apache.flink.configuration.ReadableConfig flinkConf,
                             org.apache.flink.table.catalog.ObjectPath tablePath,
                             org.apache.flink.table.catalog.CatalogTable catalogTable)

Method Detail

getLookupRuntimeProvider

public org.apache.flink.table.connector.source.LookupTableSource.LookupRuntimeProvider getLookupRuntimeProvider(org.apache.flink.table.connector.source.LookupTableSource.LookupContext context)

Specified by:

getLookupRuntimeProvider in interface org.apache.flink.table.connector.source.LookupTableSource