org.finra.herd.service.helper

Class StorageFileHelper

java.lang.Object

org.finra.herd.service.helper.StorageFileHelper

@Component
public class StorageFileHelper
extends Object

A helper class for StorageFile related code.

Constructor Summary

Constructors

Constructor and Description
StorageFileHelper()

Method Summary

Modifier and Type	Method and Description
StorageFile	createStorageFileFromEntity(StorageFileEntity storageFileEntity) Creates a storage file from the storage file entity.
List<StorageFile>	createStorageFilesFromEntities(Collection<StorageFileEntity> storageFileEntities) Creates a list of storage files from the collection of storage file entities.
List<StorageFile>	createStorageFilesFromS3ObjectSummaries(List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries) Creates a list of storage files from the list of S3 object summaries.
List<String>	getFilePathsFromS3ObjectSummaries(List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries) Returns a list of file paths extracted from the specified list of S3 object summaries.
List<String>	getFilePathsFromStorageFiles(List<StorageFile> storageFiles) Returns a list of file paths extracted from the specified list of storage files.
List<File>	getFiles(List<StorageFile> storageFiles) Returns a list of files extracted from the specified list of storage files.
Map<String,StorageFileEntity>	getStorageFileEntitiesMap(Collection<StorageFileEntity> storageFileEntities) Returns a map of file paths to the storage file entities build from the list of storage file entities.
Map<String,StorageFile>	getStorageFilesMapFromS3ObjectSummaries(List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries) Returns a map of file paths to the storage files build from the list of S3 object summaries with map iteration order matching the original list order.
Map<String,StorageFile>	getStorageFilesMapFromStorageFiles(List<StorageFile> storageFiles) Returns a map of file paths to the storage files build from the list of storage files with map iteration order matching the original list order.
long	getStorageFilesSizeBytes(Collection<StorageFile> storageFiles) Get the total size in bytes of all storage files in the specified list.
void	validateArchivedS3Files(List<StorageFile> expectedStorageFiles, List<com.amazonaws.services.s3.model.S3ObjectSummary> actualS3Files, String storageName, BusinessObjectDataKey businessObjectDataKey) Validates archived S3 files per list of expected storage files.
void	validateCopiedS3Files(List<StorageFile> expectedStorageFiles, List<com.amazonaws.services.s3.model.S3ObjectSummary> actualS3Files, String storageName, BusinessObjectDataKey businessObjectDataKey) Validates copied S3 files per list of expected storage files.
void	validateDownloadedS3Files(String baseDirectory, String s3KeyPrefix, List<StorageFile> storageFiles) Validate downloaded S3 files per specified list of storage files.
void	validateDownloadedS3Files(String baseDirectory, String s3KeyPrefix, StorageUnit storageUnit) Validate downloaded S3 files per storage unit information.
void	validateRestoredS3Files(List<StorageFile> expectedStorageFiles, List<com.amazonaws.services.s3.model.S3ObjectSummary> actualS3Files, String storageName, BusinessObjectDataKey businessObjectDataKey) Validates restored S3 files per list of expected storage files.
void	validateSourceS3Files(List<StorageFile> expectedStorageFiles, List<com.amazonaws.services.s3.model.S3ObjectSummary> actualS3Files, String storageName, BusinessObjectDataKey businessObjectDataKey) Validates registered S3 files per list of expected storage files.
void	validateStorageFile(StorageFile storageFile, String s3BucketName, Map<String,StorageFile> actualS3Keys, boolean validateFileSize) Validates storage file against the actual S3 objects reported by S3.
void	validateStorageFileEntity(StorageFileEntity storageFileEntity, String s3BucketName, Map<String,StorageFile> actualS3Keys, boolean validateFileSize) Validates storage file entity against the actual S3 objects reported by S3.
void	validateStorageFiles(Collection<String> storageFilePaths, String s3KeyPrefix, BusinessObjectDataEntity businessObjectDataEntity, String storageName) Validates a list of storage files registered with specified business object data at specified storage.
void	validateStorageUnitS3Files(StorageUnit storageUnit, List<String> actualS3Files, String s3KeyPrefix) Validates S3 files per storage unit information.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StorageFileHelper

public StorageFileHelper()

Method Detail

createStorageFileFromEntity

public StorageFile createStorageFileFromEntity(StorageFileEntity storageFileEntity)

Creates a storage file from the storage file entity.

Parameters:

storageFileEntity - the storage file entity

Returns:

the storage file

createStorageFilesFromEntities

public List<StorageFile> createStorageFilesFromEntities(Collection<StorageFileEntity> storageFileEntities)

Creates a list of storage files from the collection of storage file entities.

Parameters:

storageFileEntities - the collection of storage file entities

Returns:

the list of storage files

createStorageFilesFromS3ObjectSummaries

public List<StorageFile> createStorageFilesFromS3ObjectSummaries(List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries)

Creates a list of storage files from the list of S3 object summaries.

Parameters:

s3ObjectSummaries - the list of S3 object summaries

Returns:

the list of storage files

getFilePathsFromS3ObjectSummaries

public List<String> getFilePathsFromS3ObjectSummaries(List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries)

Returns a list of file paths extracted from the specified list of S3 object summaries.

Parameters:

s3ObjectSummaries - the list of of S3 object summaries

Returns:

the list of file paths

getFilePathsFromStorageFiles

public List<String> getFilePathsFromStorageFiles(List<StorageFile> storageFiles)

Returns a list of file paths extracted from the specified list of storage files.

Parameters:

storageFiles - the list of storage files

Returns:

the list of file paths

getFiles

public List<File> getFiles(List<StorageFile> storageFiles)

Returns a list of files extracted from the specified list of storage files.

Parameters:

storageFiles - the list of storage files

Returns:

the list of files

getStorageFileEntitiesMap

public Map<String,StorageFileEntity> getStorageFileEntitiesMap(Collection<StorageFileEntity> storageFileEntities)

Returns a map of file paths to the storage file entities build from the list of storage file entities.

Parameters:

storageFileEntities - the collection of storage file entities

Returns:

the map of file paths to storage file entities

getStorageFilesMapFromS3ObjectSummaries

public Map<String,StorageFile> getStorageFilesMapFromS3ObjectSummaries(List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries)

Returns a map of file paths to the storage files build from the list of S3 object summaries with map iteration order matching the original list order.

Parameters:

s3ObjectSummaries - the list of S3 object summaries

Returns:

the map of file paths to storage files

getStorageFilesMapFromStorageFiles

public Map<String,StorageFile> getStorageFilesMapFromStorageFiles(List<StorageFile> storageFiles)

Returns a map of file paths to the storage files build from the list of storage files with map iteration order matching the original list order.

Parameters:

storageFiles - the list of storage files

Returns:

the map of file paths to storage files

getStorageFilesSizeBytes

public long getStorageFilesSizeBytes(Collection<StorageFile> storageFiles)

Get the total size in bytes of all storage files in the specified list.

Parameters:

storageFiles - the list of storage files

Returns:

the total size of storage files in bytes

validateArchivedS3Files

public void validateArchivedS3Files(List<StorageFile> expectedStorageFiles,
                                    List<com.amazonaws.services.s3.model.S3ObjectSummary> actualS3Files,
                                    String storageName,
                                    BusinessObjectDataKey businessObjectDataKey)

Validates archived S3 files per list of expected storage files.

Parameters:

expectedStorageFiles - the list of expected S3 files represented by storage files

actualS3Files - the list of actual S3 files represented by S3 object summaries

storageName - the storage name

businessObjectDataKey - the business object data key

validateCopiedS3Files

public void validateCopiedS3Files(List<StorageFile> expectedStorageFiles,
                                  List<com.amazonaws.services.s3.model.S3ObjectSummary> actualS3Files,
                                  String storageName,
                                  BusinessObjectDataKey businessObjectDataKey)

Validates copied S3 files per list of expected storage files.

Parameters:

expectedStorageFiles - the list of expected S3 files represented by storage files

actualS3Files - the list of actual S3 files represented by S3 object summaries

storageName - the storage name

businessObjectDataKey - the business object data key

validateDownloadedS3Files

public void validateDownloadedS3Files(String baseDirectory,
                                      String s3KeyPrefix,
                                      StorageUnit storageUnit)
                               throws IllegalStateException

Validate downloaded S3 files per storage unit information.

Parameters:

baseDirectory - the local parent directory path, relative to which the files are expected to be located

s3KeyPrefix - the S3 key prefix that was prepended to the S3 file paths, when they were uploaded to S3

storageUnit - the storage unit that contains a list of storage files to be validated

Throws:

IllegalStateException - if files are not valid

validateDownloadedS3Files

public void validateDownloadedS3Files(String baseDirectory,
                                      String s3KeyPrefix,
                                      List<StorageFile> storageFiles)
                               throws IllegalStateException

Validate downloaded S3 files per specified list of storage files.

Parameters:

baseDirectory - the local parent directory path, relative to which the files are expected to be located

s3KeyPrefix - the S3 key prefix that was prepended to the S3 file paths, when they were uploaded to S3

storageFiles - the list of storage files

Throws:

IllegalStateException - if files are not valid

validateRestoredS3Files

public void validateRestoredS3Files(List<StorageFile> expectedStorageFiles,
                                    List<com.amazonaws.services.s3.model.S3ObjectSummary> actualS3Files,
                                    String storageName,
                                    BusinessObjectDataKey businessObjectDataKey)

Validates restored S3 files per list of expected storage files.

Parameters:

expectedStorageFiles - the list of expected S3 files represented by storage files

actualS3Files - the list of actual S3 files represented by S3 object summaries

storageName - the storage name

businessObjectDataKey - the business object data key

validateSourceS3Files

public void validateSourceS3Files(List<StorageFile> expectedStorageFiles,
                                  List<com.amazonaws.services.s3.model.S3ObjectSummary> actualS3Files,
                                  String storageName,
                                  BusinessObjectDataKey businessObjectDataKey)

Validates registered S3 files per list of expected storage files.

Parameters:

expectedStorageFiles - the list of expected S3 files represented by storage files

actualS3Files - the list of actual S3 files represented by S3 object summaries

storageName - the storage name

businessObjectDataKey - the business object data key

validateStorageFile

public void validateStorageFile(StorageFile storageFile,
                                String s3BucketName,
                                Map<String,StorageFile> actualS3Keys,
                                boolean validateFileSize)

Validates storage file against the actual S3 objects reported by S3.

Parameters:

storageFile - the storage file to be validated

s3BucketName - the S3 bucket name

actualS3Keys - the map of storage file paths to storage files as reported by S3

validateFileSize - specifies whether file size validation is required or not

validateStorageFileEntity

public void validateStorageFileEntity(StorageFileEntity storageFileEntity,
                                      String s3BucketName,
                                      Map<String,StorageFile> actualS3Keys,
                                      boolean validateFileSize)

Validates storage file entity against the actual S3 objects reported by S3.

Parameters:

storageFileEntity - the storage file to be validated

s3BucketName - the S3 bucket name

actualS3Keys - the map of storage file paths to storage files as reported by S3

validateFileSize - specifies whether file size validation is required or not

validateStorageFiles

public void validateStorageFiles(Collection<String> storageFilePaths,
                                 String s3KeyPrefix,
                                 BusinessObjectDataEntity businessObjectDataEntity,
                                 String storageName)
                          throws IllegalArgumentException

Validates a list of storage files registered with specified business object data at specified storage. This method makes sure that all storage files match the expected s3 key prefix value.

Parameters:

storageFilePaths - the storage file paths to be validated

s3KeyPrefix - the S3 key prefix that storage file paths are expected to start with

businessObjectDataEntity - the business object data entity

storageName - the name of the storage that storage files are stored in

Throws:

IllegalArgumentException - if a storage file doesn't match the expected S3 key prefix.

validateStorageUnitS3Files

public void validateStorageUnitS3Files(StorageUnit storageUnit,
                                       List<String> actualS3Files,
                                       String s3KeyPrefix)

Validates S3 files per storage unit information.

Parameters:

storageUnit - the storage unit that contains S3 files to be validated

actualS3Files - the list of the actual S3 files

s3KeyPrefix - the S3 key prefix that was prepended to the S3 file paths, when they were uploaded to S3