org.apache.flink.connectors.hive

Class HiveTableSource

java.lang.Object

org.apache.flink.connectors.hive.HiveTableSource

All Implemented Interfaces:

org.apache.flink.table.connector.source.abilities.SupportsLimitPushDown, org.apache.flink.table.connector.source.abilities.SupportsPartitionPushDown, org.apache.flink.table.connector.source.abilities.SupportsProjectionPushDown, org.apache.flink.table.connector.source.DynamicTableSource, org.apache.flink.table.connector.source.ScanTableSource

Direct Known Subclasses:

HiveLookupTableSource

public class HiveTableSource
extends Object
implements org.apache.flink.table.connector.source.ScanTableSource, org.apache.flink.table.connector.source.abilities.SupportsPartitionPushDown, org.apache.flink.table.connector.source.abilities.SupportsProjectionPushDown, org.apache.flink.table.connector.source.abilities.SupportsLimitPushDown

A TableSource implementation to read data from Hive tables.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HiveTableSource.HiveContinuousPartitionFetcherContext<T extends Comparable<T>> PartitionFetcher.Context for ContinuousPartitionFetcher.

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.ScanTableSource

org.apache.flink.table.connector.source.ScanTableSource.ScanContext, org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.DynamicTableSource

org.apache.flink.table.connector.source.DynamicTableSource.Context, org.apache.flink.table.connector.source.DynamicTableSource.DataStructureConverter

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.flink.table.catalog.CatalogTable	catalogTable
protected org.apache.flink.configuration.ReadableConfig	flinkConf
protected HiveShim	hiveShim
protected String	hiveVersion
protected org.apache.hadoop.mapred.JobConf	jobConf
protected int[]	projectedFields
protected org.apache.flink.table.catalog.ObjectPath	tablePath

Constructor Summary

Constructors

Constructor and Description
HiveTableSource(org.apache.hadoop.mapred.JobConf jobConf, org.apache.flink.configuration.ReadableConfig flinkConf, org.apache.flink.table.catalog.ObjectPath tablePath, org.apache.flink.table.catalog.CatalogTable catalogTable)

Method Summary

Modifier and Type	Method and Description
void	applyLimit(long limit)
void	applyPartitions(List<Map<String,String>> remainingPartitions)
void	applyProjection(int[][] projectedFields)
String	asSummaryString()
org.apache.flink.table.connector.source.DynamicTableSource	copy()
org.apache.flink.table.connector.ChangelogMode	getChangelogMode()
protected org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData>	getDataStream(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment execEnv)
protected org.apache.flink.table.api.TableSchema	getProducedTableSchema()
org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider	getScanRuntimeProvider(org.apache.flink.table.connector.source.ScanTableSource.ScanContext runtimeProviderContext)
protected org.apache.flink.table.api.TableSchema	getTableSchema()
protected boolean	isStreamingSource()
Optional<List<Map<String,String>>>	listPartitions()
boolean	supportsNestedProjection()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

jobConf

protected final org.apache.hadoop.mapred.JobConf jobConf

flinkConf

protected final org.apache.flink.configuration.ReadableConfig flinkConf

tablePath

protected final org.apache.flink.table.catalog.ObjectPath tablePath

catalogTable

protected final org.apache.flink.table.catalog.CatalogTable catalogTable

hiveVersion

protected final String hiveVersion

hiveShim

protected final HiveShim hiveShim

projectedFields

protected int[] projectedFields

Constructor Detail

HiveTableSource

public HiveTableSource(org.apache.hadoop.mapred.JobConf jobConf,
                       org.apache.flink.configuration.ReadableConfig flinkConf,
                       org.apache.flink.table.catalog.ObjectPath tablePath,
                       org.apache.flink.table.catalog.CatalogTable catalogTable)

Method Detail

getScanRuntimeProvider

public org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider getScanRuntimeProvider(org.apache.flink.table.connector.source.ScanTableSource.ScanContext runtimeProviderContext)

Specified by:

getScanRuntimeProvider in interface org.apache.flink.table.connector.source.ScanTableSource

getDataStream

@VisibleForTesting
protected org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData> getDataStream(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment execEnv)

isStreamingSource

protected boolean isStreamingSource()

getTableSchema

protected org.apache.flink.table.api.TableSchema getTableSchema()

getProducedTableSchema

protected org.apache.flink.table.api.TableSchema getProducedTableSchema()

applyLimit

public void applyLimit(long limit)

Specified by:

applyLimit in interface org.apache.flink.table.connector.source.abilities.SupportsLimitPushDown

listPartitions

public Optional<List<Map<String,String>>> listPartitions()

Specified by:

listPartitions in interface org.apache.flink.table.connector.source.abilities.SupportsPartitionPushDown

applyPartitions

public void applyPartitions(List<Map<String,String>> remainingPartitions)

Specified by:

applyPartitions in interface org.apache.flink.table.connector.source.abilities.SupportsPartitionPushDown

supportsNestedProjection

public boolean supportsNestedProjection()

Specified by:

supportsNestedProjection in interface org.apache.flink.table.connector.source.abilities.SupportsProjectionPushDown

applyProjection

public void applyProjection(int[][] projectedFields)

Specified by:

applyProjection in interface org.apache.flink.table.connector.source.abilities.SupportsProjectionPushDown

asSummaryString

public String asSummaryString()

Specified by:

asSummaryString in interface org.apache.flink.table.connector.source.DynamicTableSource

getChangelogMode

public org.apache.flink.table.connector.ChangelogMode getChangelogMode()

Specified by:

getChangelogMode in interface org.apache.flink.table.connector.source.ScanTableSource

copy

public org.apache.flink.table.connector.source.DynamicTableSource copy()

Specified by:

copy in interface org.apache.flink.table.connector.source.DynamicTableSource