Package org.apache.pinot.common.utils

Class FileUploadDownloadClient

java.lang.Object

org.apache.pinot.common.utils.FileUploadDownloadClient

All Implemented Interfaces:

Closeable, AutoCloseable

public class FileUploadDownloadClient
extends Object
implements Closeable

The FileUploadDownloadClient class provides methods to upload schema/segment, download segment or send segment completion protocol request through HTTP/HTTPS.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	FileUploadDownloadClient.CustomHeaders
static class	FileUploadDownloadClient.FileUploadType
static class	FileUploadDownloadClient.QueryParameters

Field Summary

Fields

Modifier and Type	Field	Description
static SSLContext	_defaultSSLContext	optional default SSL context for FileUploadDownloadClient operations
static int	DEFAULT_SOCKET_TIMEOUT_MS
static int	DELETE_REQUEST_SOCKET_TIMEOUT_MS
static int	GET_REQUEST_SOCKET_TIMEOUT_MS

Constructor Summary

Constructors

Constructor	Description
FileUploadDownloadClient()	Construct the client with default settings.
FileUploadDownloadClient(SSLContext sslContext)	Construct the client with optional SSLContext to handle HTTPS request properly.

Method Summary

Modifier and Type	Method	Description
SimpleHttpResponse	addSchema(URI uri, String schemaName, File schemaFile)	Deprecated.
SimpleHttpResponse	addSchema(URI uri, String schemaName, File schemaFile, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters)	Add schema.
void	close()
int	downloadFile(URI uri, int socketTimeoutMs, File dest)	Deprecated.
int	downloadFile(URI uri, int socketTimeoutMs, File dest, String authToken, List<org.apache.http.Header> httpHeaders)	Download a file using default settings, with an optional auth token
int	downloadFile(URI uri, File dest)	Deprecated.
int	downloadFile(URI uri, File dest, String authToken)	Download a file.
int	downloadFile(URI uri, File dest, String authToken, List<org.apache.http.Header> httpHeaders)	Download a file.
SimpleHttpResponse	endReplaceSegments(URI uri, int socketTimeoutMs, String authToken)	End replace segments with default settings.
static URI	getDeleteSegmentHttpUri(String host, int port, String rawTableName, String segmentName, String tableType)	Deprecated.
static URI	getEndReplaceSegmentsURI(URI controllerURI, String rawTableName, String tableType, String segmentLineageEntryId)
static URI	getOldUploadSegmentHttpsURI(String host, int port)	Deprecated.
static URI	getOldUploadSegmentHttpURI(String host, int port)	Deprecated.
static URI	getRetrieveAllSegmentWithTableTypeHttpUri(String host, int port, String rawTableName, String tableType)	Deprecated.
static URI	getRetrieveSchemaHttpURI(String host, int port, String schemaName)	Deprecated.
static URI	getRetrieveSchemaURI(String protocol, String host, int port, String schemaName)
static URI	getRetrieveTableConfigHttpURI(String host, int port, String rawTableName)	Deprecated.
static URI	getRetrieveTableConfigURI(String protocol, String host, int port, String rawTableName)
static URI	getRevertReplaceSegmentsURI(URI controllerURI, String rawTableName, String tableType, String segmentLineageEntryId, boolean forceRevert)
static URI	getStartReplaceSegmentsURI(URI controllerURI, String rawTableName, String tableType, boolean forceCleanup)
static URI	getUploadSchemaHttpsURI(String host, int port)	Deprecated.
static URI	getUploadSchemaHttpURI(String host, int port)	Deprecated.
static URI	getUploadSchemaURI(String protocol, String host, int port)
static URI	getUploadSchemaURI(URI controllerURI)
static URI	getUploadSegmentHttpsURI(String host, int port)	Deprecated.
static URI	getUploadSegmentHttpURI(String host, int port)	Deprecated.
static URI	getUploadSegmentURI(String protocol, String host, int port)
static URI	getUploadSegmentURI(URI controllerURI)
static URI	getURI(String protocol, String host, int port, String path)
static URI	getURI(String protocol, String host, int port, String path, String query)
static void	installDefaultSSLContext(SSLContext sslContext)	Install a default SSLContext for all FileUploadDownloadClients instantiated.
static List<org.apache.http.Header>	makeAuthHeader(String authToken)	Generate an (optional) HTTP Authorization header given an auth token.
static List<org.apache.http.NameValuePair>	makeTableParam(String tableName)	Generate a param list with a table name attribute.
SimpleHttpResponse	postJsonRequest(URI uri, String jsonRequestBody, String authToken)
SimpleHttpResponse	revertReplaceSegments(URI uri)	Revert replace segments with default settings.
SimpleHttpResponse	sendDeleteRequest(URI uri)	Deprecated.
SimpleHttpResponse	sendDeleteRequest(URI uri, String authToken)
SimpleHttpResponse	sendGetRequest(URI uri)	Deprecated.
SimpleHttpResponse	sendGetRequest(URI uri, String authToken)
SimpleHttpResponse	sendSegmentCompletionProtocolRequest(URI uri, int socketTimeoutMs)	Deprecated.
SimpleHttpResponse	sendSegmentCompletionProtocolRequest(URI uri, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int socketTimeoutMs)	Send segment completion protocol request.
SimpleHttpResponse	sendSegmentJson(URI uri, String jsonString)	Deprecated.
SimpleHttpResponse	sendSegmentJson(URI uri, String jsonString, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int socketTimeoutMs)	Send segment json.
SimpleHttpResponse	sendSegmentUri(URI uri, String downloadUri, String rawTableName)	Deprecated.
SimpleHttpResponse	sendSegmentUri(URI uri, String downloadUri, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int socketTimeoutMs)	Send segment uri.
SimpleHttpResponse	startReplaceSegments(URI uri, StartReplaceSegmentsRequest startReplaceSegmentsRequest, String authToken)	Start replace segments with default settings.
SimpleHttpResponse	updateSchema(URI uri, String schemaName, File schemaFile)	Deprecated.
SimpleHttpResponse	updateSchema(URI uri, String schemaName, File schemaFile, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters)	Update schema.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, File segmentFile, String tableName)	Deprecated.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, File segmentFile, String tableName, TableType tableType)	Upload segment with segment file using default settings.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, File segmentFile, String tableName, TableType tableType, boolean enableParallelPushProtection, boolean allowRefresh)	Upload segment with segment file using table name, type, enableParallelPushProtection and allowRefresh as request parameters.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, File segmentFile, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int socketTimeoutMs)	Upload segment with segment file.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, InputStream inputStream, String rawTableName)	Upload segment with segment file input stream using default settings.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, InputStream inputStream, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int socketTimeoutMs)	Upload segment with segment file input stream.
SimpleHttpResponse	uploadSegment(URI uri, String segmentName, InputStream inputStream, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, String tableName, TableType tableType)	Upload segment with segment file input stream.
SimpleHttpResponse	uploadSegmentMetadata(URI uri, String segmentName, File segmentMetadataFile, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int socketTimeoutMs)	Upload segment by sending a zip of creation.meta and metadata.properties.
SimpleHttpResponse	uploadSegmentMetadataFiles(URI uri, Map<String,File> metadataFiles, int segmentUploadRequestTimeoutMs)	Deprecated.
SimpleHttpResponse	uploadSegmentMetadataFiles(URI uri, Map<String,File> metadataFiles, List<org.apache.http.Header> headers, List<org.apache.http.NameValuePair> parameters, int segmentUploadRequestTimeoutMs)
String	uploadToSegmentStore(String uri)	Used by controllers to send requests to servers: Controller periodic task uses this endpoint to ask servers to upload committed llc segment to segment store if missing.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_defaultSSLContext

public static SSLContext _defaultSSLContext

optional default SSL context for FileUploadDownloadClient operations

DEFAULT_SOCKET_TIMEOUT_MS

public static final int DEFAULT_SOCKET_TIMEOUT_MS

See Also:

Constant Field Values

GET_REQUEST_SOCKET_TIMEOUT_MS

public static final int GET_REQUEST_SOCKET_TIMEOUT_MS

See Also:

Constant Field Values

DELETE_REQUEST_SOCKET_TIMEOUT_MS

public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS

See Also:

Constant Field Values

Constructor Detail

FileUploadDownloadClient

public FileUploadDownloadClient()

Construct the client with default settings.

FileUploadDownloadClient

public FileUploadDownloadClient(@Nullable
                                SSLContext sslContext)

Construct the client with optional SSLContext to handle HTTPS request properly.

Parameters:

sslContext - SSL context

Method Detail

getURI

public static URI getURI(String protocol,
                         String host,
                         int port,
                         String path)
                  throws URISyntaxException

Throws:

URISyntaxException

getURI

public static URI getURI(String protocol,
                         String host,
                         int port,
                         String path,
                         String query)
                  throws URISyntaxException

Throws:

URISyntaxException

getRetrieveTableConfigHttpURI

@Deprecated
public static URI getRetrieveTableConfigHttpURI(String host,
                                                int port,
                                                String rawTableName)
                                         throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

getRetrieveTableConfigURI(String, String, int, String)

getRetrieveTableConfigURI

public static URI getRetrieveTableConfigURI(String protocol,
                                            String host,
                                            int port,
                                            String rawTableName)
                                     throws URISyntaxException

Throws:

URISyntaxException

getDeleteSegmentHttpUri

@Deprecated
public static URI getDeleteSegmentHttpUri(String host,
                                          int port,
                                          String rawTableName,
                                          String segmentName,
                                          String tableType)
                                   throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled This method calls the old segment endpoint. We will deprecate this behavior soon.

Throws:

URISyntaxException

getRetrieveAllSegmentWithTableTypeHttpUri

@Deprecated
public static URI getRetrieveAllSegmentWithTableTypeHttpUri(String host,
                                                            int port,
                                                            String rawTableName,
                                                            String tableType)
                                                     throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled This method calls the old segment endpoint. We will deprecate this behavior soon.

Throws:

URISyntaxException

getRetrieveSchemaHttpURI

@Deprecated
public static URI getRetrieveSchemaHttpURI(String host,
                                           int port,
                                           String schemaName)
                                    throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

getRetrieveSchemaURI(String, String, int, String)

getRetrieveSchemaURI

public static URI getRetrieveSchemaURI(String protocol,
                                       String host,
                                       int port,
                                       String schemaName)
                                throws URISyntaxException

Throws:

URISyntaxException

getUploadSchemaHttpURI

@Deprecated
public static URI getUploadSchemaHttpURI(String host,
                                         int port)
                                  throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

getUploadSchemaURI(String, String, int)

getUploadSchemaHttpsURI

@Deprecated
public static URI getUploadSchemaHttpsURI(String host,
                                          int port)
                                   throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

getUploadSchemaURI(String, String, int)

getUploadSchemaURI

public static URI getUploadSchemaURI(String protocol,
                                     String host,
                                     int port)
                              throws URISyntaxException

Throws:

URISyntaxException

getUploadSchemaURI

public static URI getUploadSchemaURI(URI controllerURI)
                              throws URISyntaxException

Throws:

URISyntaxException

getOldUploadSegmentHttpURI

@Deprecated
public static URI getOldUploadSegmentHttpURI(String host,
                                             int port)
                                      throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

This method calls the old segment upload endpoint. We will deprecate this behavior soon. Please call getUploadSegmentHttpURI to construct your request.

getOldUploadSegmentHttpsURI

@Deprecated
public static URI getOldUploadSegmentHttpsURI(String host,
                                              int port)
                                       throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

This method calls the old segment upload endpoint. We will deprecate this behavior soon. Please call getUploadSegmentHttpsURI to construct your request.

getUploadSegmentHttpURI

@Deprecated
public static URI getUploadSegmentHttpURI(String host,
                                          int port)
                                   throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

getUploadSegmentURI(String, String, int)

getUploadSegmentHttpsURI

@Deprecated
public static URI getUploadSegmentHttpsURI(String host,
                                           int port)
                                    throws URISyntaxException

Deprecated.

Deprecated due to lack of protocol/scheme support. May break for deployments with TLS/SSL enabled

Throws:

URISyntaxException

See Also:

getUploadSegmentURI(String, String, int)

getUploadSegmentURI

public static URI getUploadSegmentURI(String protocol,
                                      String host,
                                      int port)
                               throws URISyntaxException

Throws:

URISyntaxException

getUploadSegmentURI

public static URI getUploadSegmentURI(URI controllerURI)
                               throws URISyntaxException

Throws:

URISyntaxException

getStartReplaceSegmentsURI

public static URI getStartReplaceSegmentsURI(URI controllerURI,
                                             String rawTableName,
                                             String tableType,
                                             boolean forceCleanup)
                                      throws URISyntaxException

Throws:

URISyntaxException

getEndReplaceSegmentsURI

public static URI getEndReplaceSegmentsURI(URI controllerURI,
                                           String rawTableName,
                                           String tableType,
                                           String segmentLineageEntryId)
                                    throws URISyntaxException

Throws:

URISyntaxException

getRevertReplaceSegmentsURI

public static URI getRevertReplaceSegmentsURI(URI controllerURI,
                                              String rawTableName,
                                              String tableType,
                                              String segmentLineageEntryId,
                                              boolean forceRevert)
                                       throws URISyntaxException

Throws:

URISyntaxException

sendGetRequest

@Deprecated
public SimpleHttpResponse sendGetRequest(URI uri)
                                  throws IOException,
                                         HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled

Throws:

IOException

HttpErrorStatusException

See Also:

sendGetRequest(URI, String)

sendGetRequest

public SimpleHttpResponse sendGetRequest(URI uri,
                                         @Nullable
                                         String authToken)
                                  throws IOException,
                                         HttpErrorStatusException

Throws:

IOException

HttpErrorStatusException

sendDeleteRequest

@Deprecated
public SimpleHttpResponse sendDeleteRequest(URI uri)
                                     throws IOException,
                                            HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled

Throws:

IOException

HttpErrorStatusException

See Also:

sendDeleteRequest(URI, String)

sendDeleteRequest

public SimpleHttpResponse sendDeleteRequest(URI uri,
                                            @Nullable
                                            String authToken)
                                     throws IOException,
                                            HttpErrorStatusException

Throws:

IOException

HttpErrorStatusException

postJsonRequest

public SimpleHttpResponse postJsonRequest(URI uri,
                                          @Nullable
                                          String jsonRequestBody,
                                          @Nullable
                                          String authToken)
                                   throws HttpErrorStatusException,
                                          IOException

Throws:

HttpErrorStatusException

IOException

addSchema

@Deprecated
public SimpleHttpResponse addSchema(URI uri,
                                    String schemaName,
                                    File schemaFile)
                             throws IOException,
                                    HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Add schema.

Parameters:

uri - URI

schemaName - Schema name

schemaFile - Schema file

Returns:

Response

Throws:

IOException

HttpErrorStatusException

See Also:

addSchema(URI, String, File, List, List)

addSchema

public SimpleHttpResponse addSchema(URI uri,
                                    String schemaName,
                                    File schemaFile,
                                    @Nullable
                                    List<org.apache.http.Header> headers,
                                    @Nullable
                                    List<org.apache.http.NameValuePair> parameters)
                             throws IOException,
                                    HttpErrorStatusException

Add schema.

Parameters:

uri - URI

schemaName - Schema name

schemaFile - Schema file

headers - HTTP headers

parameters - HTTP parameters

Returns:

Response

Throws:

IOException

HttpErrorStatusException

updateSchema

@Deprecated
public SimpleHttpResponse updateSchema(URI uri,
                                       String schemaName,
                                       File schemaFile)
                                throws IOException,
                                       HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Update schema.

Parameters:

uri - URI

schemaName - Schema name

schemaFile - Schema file

Returns:

Response

Throws:

IOException

HttpErrorStatusException

See Also:

updateSchema(URI, String, File, List, List)

updateSchema

public SimpleHttpResponse updateSchema(URI uri,
                                       String schemaName,
                                       File schemaFile,
                                       @Nullable
                                       List<org.apache.http.Header> headers,
                                       @Nullable
                                       List<org.apache.http.NameValuePair> parameters)
                                throws IOException,
                                       HttpErrorStatusException

Update schema.

Parameters:

uri - URI

schemaName - Schema name

schemaFile - Schema file

headers - HTTP headers

parameters - HTTP parameters

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegmentMetadata

public SimpleHttpResponse uploadSegmentMetadata(URI uri,
                                                String segmentName,
                                                File segmentMetadataFile,
                                                @Nullable
                                                List<org.apache.http.Header> headers,
                                                @Nullable
                                                List<org.apache.http.NameValuePair> parameters,
                                                int socketTimeoutMs)
                                         throws IOException,
                                                HttpErrorStatusException

Upload segment by sending a zip of creation.meta and metadata.properties.

Parameters:

uri - URI

segmentName - Segment name

segmentMetadataFile - Segment metadata file

headers - Optional http headers

parameters - Optional query parameters

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegmentMetadataFiles

@Deprecated
public SimpleHttpResponse uploadSegmentMetadataFiles(URI uri,
                                                     Map<String,File> metadataFiles,
                                                     int segmentUploadRequestTimeoutMs)
                                              throws IOException,
                                                     HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled

Throws:

IOException

HttpErrorStatusException

See Also:

uploadSegment(URI, String, InputStream, List, List, int)

uploadSegmentMetadataFiles

public SimpleHttpResponse uploadSegmentMetadataFiles(URI uri,
                                                     Map<String,File> metadataFiles,
                                                     @Nullable
                                                     List<org.apache.http.Header> headers,
                                                     @Nullable
                                                     List<org.apache.http.NameValuePair> parameters,
                                                     int segmentUploadRequestTimeoutMs)
                                              throws IOException,
                                                     HttpErrorStatusException

Throws:

IOException

HttpErrorStatusException

uploadSegment

public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        File segmentFile,
                                        @Nullable
                                        List<org.apache.http.Header> headers,
                                        @Nullable
                                        List<org.apache.http.NameValuePair> parameters,
                                        int socketTimeoutMs)
                                 throws IOException,
                                        HttpErrorStatusException

Upload segment with segment file. Note: table name needs to be added as a parameter except for the case where this gets called during realtime segment commit protocol. TODO: fix the realtime segment commit protocol to add table name as a parameter.

Parameters:

uri - URI

segmentName - Segment name

segmentFile - Segment file

headers - Optional http headers

parameters - Optional query parameters

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegment

@Deprecated
public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        File segmentFile,
                                        String tableName)
                                 throws IOException,
                                        HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Upload segment with segment file using default settings. Include table name as a request parameter.

Parameters:

uri - URI

segmentName - Segment name

segmentFile - Segment file

tableName - Table name with or without type suffix

Returns:

Response

Throws:

IOException

HttpErrorStatusException

See Also:

uploadSegment(URI, String, InputStream, List, List, int)

uploadSegment

public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        File segmentFile,
                                        String tableName,
                                        TableType tableType)
                                 throws IOException,
                                        HttpErrorStatusException

Upload segment with segment file using default settings. Include table name and type as a request parameters.

Parameters:

uri - URI

segmentName - Segment name

segmentFile - Segment file

tableName - Table name with or without type suffix

tableType - Table type

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegment

public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        File segmentFile,
                                        String tableName,
                                        TableType tableType,
                                        boolean enableParallelPushProtection,
                                        boolean allowRefresh)
                                 throws IOException,
                                        HttpErrorStatusException

Upload segment with segment file using table name, type, enableParallelPushProtection and allowRefresh as request parameters.

Parameters:

uri - URI

segmentName - Segment name

segmentFile - Segment file

tableName - Table name with or without type suffix

tableType - Table type

enableParallelPushProtection - enable protection against concurrent segment uploads for the same segment

allowRefresh - whether to refresh a segment if it already exists

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegment

public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        InputStream inputStream,
                                        @Nullable
                                        List<org.apache.http.Header> headers,
                                        @Nullable
                                        List<org.apache.http.NameValuePair> parameters,
                                        String tableName,
                                        TableType tableType)
                                 throws IOException,
                                        HttpErrorStatusException

Upload segment with segment file input stream. Note: table name has to be set as a parameter.

Parameters:

uri - URI

segmentName - Segment name

inputStream - Segment file input stream

headers - Optional http headers

parameters - Optional query parameters

tableName - Table name with or without type suffix

tableType - Table type

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegment

public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        InputStream inputStream,
                                        @Nullable
                                        List<org.apache.http.Header> headers,
                                        @Nullable
                                        List<org.apache.http.NameValuePair> parameters,
                                        int socketTimeoutMs)
                                 throws IOException,
                                        HttpErrorStatusException

Upload segment with segment file input stream. Note: table name has to be set as a parameter.

Parameters:

uri - URI

segmentName - Segment name

inputStream - Segment file input stream

headers - Optional http headers

parameters - Optional query parameters

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadSegment

public SimpleHttpResponse uploadSegment(URI uri,
                                        String segmentName,
                                        InputStream inputStream,
                                        String rawTableName)
                                 throws IOException,
                                        HttpErrorStatusException

Upload segment with segment file input stream using default settings. Include table name as a request parameter.

Parameters:

uri - URI

segmentName - Segment name

inputStream - Segment file input stream

rawTableName - Raw table name

Returns:

Response

Throws:

IOException

HttpErrorStatusException

uploadToSegmentStore

public String uploadToSegmentStore(String uri)
                            throws URISyntaxException,
                                   IOException,
                                   HttpErrorStatusException

Used by controllers to send requests to servers: Controller periodic task uses this endpoint to ask servers to upload committed llc segment to segment store if missing.

Parameters:

uri - The uri to ask servers to upload segment to segment store

Returns:

the uploaded segment download url from segment store

Throws:

URISyntaxException

IOException

HttpErrorStatusException - TODO: migrate this method to another class

sendSegmentUri

public SimpleHttpResponse sendSegmentUri(URI uri,
                                         String downloadUri,
                                         @Nullable
                                         List<org.apache.http.Header> headers,
                                         @Nullable
                                         List<org.apache.http.NameValuePair> parameters,
                                         int socketTimeoutMs)
                                  throws IOException,
                                         HttpErrorStatusException

Send segment uri. Note: table name has to be set as a parameter.

Parameters:

uri - URI

downloadUri - Segment download uri

headers - Optional http headers

parameters - Optional query parameters

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

sendSegmentUri

@Deprecated
public SimpleHttpResponse sendSegmentUri(URI uri,
                                         String downloadUri,
                                         String rawTableName)
                                  throws IOException,
                                         HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Send segment uri using default settings. Include table name as a request parameter.

Parameters:

uri - URI

downloadUri - Segment download uri

rawTableName - Raw table name

Returns:

Response

Throws:

IOException

HttpErrorStatusException

See Also:

sendSegmentUri(URI, String, List, List, int)

sendSegmentJson

public SimpleHttpResponse sendSegmentJson(URI uri,
                                          String jsonString,
                                          @Nullable
                                          List<org.apache.http.Header> headers,
                                          @Nullable
                                          List<org.apache.http.NameValuePair> parameters,
                                          int socketTimeoutMs)
                                   throws IOException,
                                          HttpErrorStatusException

Send segment json.

Parameters:

uri - URI

jsonString - Segment json string

headers - Optional http headers

parameters - Optional query parameters

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

sendSegmentJson

@Deprecated
public SimpleHttpResponse sendSegmentJson(URI uri,
                                          String jsonString)
                                   throws IOException,
                                          HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Send segment json using default settings.

Parameters:

uri - URI

jsonString - Segment json string

Returns:

Response

Throws:

IOException

HttpErrorStatusException

See Also:

sendSegmentJson(URI, String, List, List, int)

startReplaceSegments

public SimpleHttpResponse startReplaceSegments(URI uri,
                                               StartReplaceSegmentsRequest startReplaceSegmentsRequest,
                                               @Nullable
                                               String authToken)
                                        throws IOException,
                                               HttpErrorStatusException

Start replace segments with default settings.

Parameters:

uri - URI

startReplaceSegmentsRequest - request

authToken - auth token

Returns:

Response

Throws:

IOException

HttpErrorStatusException

endReplaceSegments

public SimpleHttpResponse endReplaceSegments(URI uri,
                                             int socketTimeoutMs,
                                             @Nullable
                                             String authToken)
                                      throws IOException,
                                             HttpErrorStatusException

End replace segments with default settings.

Parameters:

uri - URI

authToken - auth token

Returns:

Response

Throws:

IOException

HttpErrorStatusException

revertReplaceSegments

public SimpleHttpResponse revertReplaceSegments(URI uri)
                                         throws IOException,
                                                HttpErrorStatusException

Revert replace segments with default settings.

Parameters:

uri - URI

Returns:

Response

Throws:

IOException

HttpErrorStatusException

sendSegmentCompletionProtocolRequest

@Deprecated
public SimpleHttpResponse sendSegmentCompletionProtocolRequest(URI uri,
                                                               int socketTimeoutMs)
                                                        throws IOException,
                                                               HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Send segment completion protocol request.

Parameters:

uri - URI

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

See Also:

sendSegmentCompletionProtocolRequest(URI, List, List, int)

sendSegmentCompletionProtocolRequest

public SimpleHttpResponse sendSegmentCompletionProtocolRequest(URI uri,
                                                               @Nullable
                                                               List<org.apache.http.Header> headers,
                                                               @Nullable
                                                               List<org.apache.http.NameValuePair> parameters,
                                                               int socketTimeoutMs)
                                                        throws IOException,
                                                               HttpErrorStatusException

Send segment completion protocol request.

Parameters:

uri - URI

headers - Optional http headers

parameters - Optional query parameters

socketTimeoutMs - Socket timeout in milliseconds

Returns:

Response

Throws:

IOException

HttpErrorStatusException

downloadFile

@Deprecated
public int downloadFile(URI uri,
                        int socketTimeoutMs,
                        File dest)
                 throws IOException,
                        HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Download a file using default settings, with an optional auth token

Parameters:

uri - URI

socketTimeoutMs - Socket timeout in milliseconds

dest - File destination

Returns:

Response status code

Throws:

IOException

HttpErrorStatusException

See Also:

downloadFile(URI, int, File, String, List)

downloadFile

public int downloadFile(URI uri,
                        int socketTimeoutMs,
                        File dest,
                        String authToken,
                        List<org.apache.http.Header> httpHeaders)
                 throws IOException,
                        HttpErrorStatusException

Download a file using default settings, with an optional auth token

Parameters:

uri - URI

socketTimeoutMs - Socket timeout in milliseconds

dest - File destination

authToken - auth token

httpHeaders - http headers

Returns:

Response status code

Throws:

IOException

HttpErrorStatusException

downloadFile

@Deprecated
public int downloadFile(URI uri,
                        File dest)
                 throws IOException,
                        HttpErrorStatusException

Deprecated.

Deprecated due to lack of auth header support. May break for deployments with auth enabled Download a file.

Parameters:

uri - URI

dest - File destination

Returns:

Response status code

Throws:

IOException

HttpErrorStatusException

See Also:

downloadFile(URI, File, String)

downloadFile

public int downloadFile(URI uri,
                        File dest,
                        String authToken)
                 throws IOException,
                        HttpErrorStatusException

Download a file.

Parameters:

uri - URI

dest - File destination

authToken - auth token

Returns:

Response status code

Throws:

IOException

HttpErrorStatusException

downloadFile

public int downloadFile(URI uri,
                        File dest,
                        String authToken,
                        List<org.apache.http.Header> httpHeaders)
                 throws IOException,
                        HttpErrorStatusException

Download a file.

Parameters:

uri - URI

dest - File destination

authToken - auth token

httpHeaders - http headers

Returns:

Response status code

Throws:

IOException

HttpErrorStatusException

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException

installDefaultSSLContext

public static void installDefaultSSLContext(SSLContext sslContext)

Install a default SSLContext for all FileUploadDownloadClients instantiated.

Parameters:

sslContext - default ssl context

makeAuthHeader

public static List<org.apache.http.Header> makeAuthHeader(String authToken)

Generate an (optional) HTTP Authorization header given an auth token.

Parameters:

authToken - auth token

Returns:

list of 0 or 1 "Authorization" headers

makeTableParam

public static List<org.apache.http.NameValuePair> makeTableParam(String tableName)

Generate a param list with a table name attribute.

Parameters:

tableName - table name

Returns:

param list