Type Members

1.  trait AWSConnector extends Connector
2.  class BatchKinesisWriter extends KinesisWriter
3.  case class KinesisDataConnector(streamName: String, region: String = "us-east-1", partitionKey: Option[String], partitionKeyIndex: Option[Int], separator: String = ",", name: String, credentialName: Option[String], credential: Option[Credential]) extends StreamingDataConnector with AWSConnector with Product with Serializable

   Data Connector implementation that works with Kinesis.

   Data Connector implementation that works with Kinesis. Each row produced will be formatted to a string using the separator character provided.

   streamName

   The name of the Kinesis stream.

   region

   The region containing the Kinesis stream

   partitionKey

   The optional static partition key to use

   partitionKeyIndex

   The optional field index in the DataFrame row containing the value to use as the partition key

   separator

   The field separator to use when formatting the row data

   name

   The name of the connector

   credentialName

   The optional name of the credential to use when authorizing to the Kinesis stream

   credential

   The optional credential to use when authorizing to the Kinesis stream

4.  trait KinesisWriter extends ConnectorWriter

   Write a batch DataFrame to Kinesis using record batching.

   Write a batch DataFrame to Kinesis using record batching. The following parameters are expected: streamName The Kinesis stream name region The region of the Kinesis stream dataFrame The DataFrame to write partitionKey The static partition key to use partitionKeyIndex The field index in the DataFrame row containing the value to use as the partition key separator The field separator to use when formatting the row data credential An optional credential to use to authenticate to Kinesis

5.  case class S3DataConnector(name: String, credentialName: Option[String], credential: Option[Credential]) extends BatchDataConnector with AWSConnector with Product with Serializable
6.  case class S3FileConnector(region: String, bucket: String, name: String, credentialName: Option[String], credential: Option[Credential]) extends FileConnector with AWSConnector with Product with Serializable

   Provides an implementation of theFileConnector that works with S3.

   Provides an implementation of theFileConnector that works with S3.

   region

   The AWS region

   bucket

   The S3 bucket

   name

   The name of this connector

   credentialName

   The optional name of the credential to provide the CredentialProvider

   credential

   The optional credential to use. credentialName takes precedence if provided.

7.  class StructuredStreamingKinesisSink extends ForeachWriter[Row] with KinesisWriter