Value Members

1.  final def !=(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

2.  final def ##(): Int

   Definition Classes

   AnyRef → Any

3.  final def ==(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

4.  val MULTIPART_COPY_SIZE: Long
5.  val MULTIPART_UPLOAD_SIZE: Int
6.  final def asInstanceOf[T0]: T0

   Definition Classes

   Any

7.  def clone(): AnyRef

   Attributes

   protected[lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... ) @native()

8.  def configureSparkSession(pipelineContext: PipelineContext): Unit

   Prepares Spark for reading/writing of DataFrames

   Prepares Spark for reading/writing of DataFrames

   pipelineContext

   The PipelineContext containing the SparkSession

9.  def deriveProtocol(path: String): String

   Given a path, this function will attempt to derive the protocol.

   Given a path, this function will attempt to derive the protocol. If the protocol cannot be determined, then it will default to s3a.

   path

   A valid path

   returns

   The protocol to use for authentication.

10.  final def eq(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

11.  def equals(arg0: Any): Boolean

    Definition Classes

    AnyRef → Any

12.  def finalize(): Unit

    Attributes

    protected[lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

13.  final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

14.  def hashCode(): Int

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

15.  final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

16.  final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

17.  final def notify(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

18.  final def notifyAll(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

19.  def prepareS3FilePath(path: String, bucket: Option[String] = None): String

    This function will take the given path and strip any protocol information.

    This function will take the given path and strip any protocol information.

    path

    A valid path

    returns

    A raw path with no protocol information

20.  def replaceProtocol(path: String, protocol: String): String

    This function will attempt to add or replace the protocol in the given path.

    This function will attempt to add or replace the protocol in the given path.

    path

    A calid path

    protocol

    The protocol to use

    returns

    The path with the proper protocol.

21.  def setS3Authorization(path: String, accessKeyId: Option[String], secretAccessKey: Option[String], pipelineContext: PipelineContext): Unit

    This function will attempt to set the authorization used when reading or writing a DataFrame.

    This function will attempt to set the authorization used when reading or writing a DataFrame.

    path

    A valid path

    accessKeyId

    The access key

    secretAccessKey

    The secret

    pipelineContext

    The PipelineContext

22.  final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

23.  def toString(): String

    Definition Classes

    AnyRef → Any

24.  final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

25.  final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

26.  final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... ) @native()