alluxio.underfs.s3a

Class S3AInputStream

java.lang.Object

java.io.InputStream

alluxio.underfs.s3a.S3AInputStream

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

@NotThreadSafe
public class S3AInputStream
extends java.io.InputStream

A wrapper around an S3ObjectInputStream which handles skips efficiently.

Field Summary

Fields

Modifier and Type	Field and Description
protected java.lang.String	mBucketName Name of the bucket the object resides in.
protected com.amazonaws.services.s3.AmazonS3	mClient Client for operations with s3.
protected com.amazonaws.services.s3.model.S3ObjectInputStream	mIn The backing input stream from s3.
protected java.lang.String	mKey The path of the object to read.
protected long	mPos The current position of the stream.
protected RetryPolicy	mRetryPolicy Policy determining the retry behavior in case the key does not exist.

Constructor Summary

Constructors

Constructor and Description
S3AInputStream(java.lang.String bucketName, java.lang.String key, com.amazonaws.services.s3.AmazonS3 client, long position, RetryPolicy retryPolicy) Constructor for an input stream of an object in s3 using the aws-sdk implementation to read the data.
S3AInputStream(java.lang.String bucketName, java.lang.String key, com.amazonaws.services.s3.AmazonS3 client, RetryPolicy retryPolicy) Constructor for an input stream of an object in s3 using the aws-sdk implementation to read the data.

Method Summary

Modifier and Type	Method and Description
void	close()
protected com.amazonaws.services.s3.AmazonS3	getClient()
int	read()
int	read(byte[] b)
int	read(byte[] b, int offset, int length)
long	skip(long n)

Methods inherited from class java.io.InputStream

available, mark, markSupported, reset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

mClient

protected com.amazonaws.services.s3.AmazonS3 mClient

Client for operations with s3.

mBucketName

protected final java.lang.String mBucketName

Name of the bucket the object resides in.

mKey

protected final java.lang.String mKey

The path of the object to read.

mIn

protected com.amazonaws.services.s3.model.S3ObjectInputStream mIn

The backing input stream from s3.

mPos

protected long mPos

The current position of the stream.

mRetryPolicy

protected final RetryPolicy mRetryPolicy

Policy determining the retry behavior in case the key does not exist. The key may not exist because of eventual consistency.

Constructor Detail

S3AInputStream

public S3AInputStream(java.lang.String bucketName,
                      java.lang.String key,
                      com.amazonaws.services.s3.AmazonS3 client,
                      RetryPolicy retryPolicy)

Constructor for an input stream of an object in s3 using the aws-sdk implementation to read the data. The stream will be positioned at the start of the file.

Parameters:

bucketName - the bucket the object resides in

key - the path of the object to read

client - the s3 client to use for operations

retryPolicy - retry policy in case the key does not exist

S3AInputStream

public S3AInputStream(java.lang.String bucketName,
                      java.lang.String key,
                      com.amazonaws.services.s3.AmazonS3 client,
                      long position,
                      RetryPolicy retryPolicy)

Constructor for an input stream of an object in s3 using the aws-sdk implementation to read the data. The stream will be positioned at the specified position.

Parameters:

bucketName - the bucket the object resides in

key - the path of the object to read

client - the s3 client to use for operations

position - the position to begin reading from

retryPolicy - retry policy in case the key does not exist

Method Detail

close

public void close()

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Overrides:

close in class java.io.InputStream

read

public int read()
         throws java.io.IOException

Specified by:

read in class java.io.InputStream

Throws:

java.io.IOException

read

public int read(byte[] b)
         throws java.io.IOException

Overrides:

read in class java.io.InputStream

Throws:

java.io.IOException

read

public int read(byte[] b,
                int offset,
                int length)
         throws java.io.IOException

Overrides:

read in class java.io.InputStream

Throws:

java.io.IOException

skip

public long skip(long n)
          throws java.io.IOException

Overrides:

skip in class java.io.InputStream

Throws:

java.io.IOException

getClient

protected com.amazonaws.services.s3.AmazonS3 getClient()

Returns:

the client