org.finra.herd.rest

Class EmrRestController

java.lang.Object

org.finra.herd.rest.HerdBaseController

org.finra.herd.rest.EmrRestController

@RestController
 @RequestMapping(value="/rest",
                produces={"application/xml","application/json"})
public class EmrRestController
extends HerdBaseController

The REST controller that handles EMR REST requests.

Constructor Summary

Constructors

Constructor and Description
EmrRestController()

Method Summary

Modifier and Type	Method and Description
EmrMasterSecurityGroup	addGroupsToEmrClusterMaster(EmrMasterSecurityGroupAddRequest request) Adds security groups to the master node of an existing cluster
EmrHadoopJarStep	addHadoopJarStepToEmrCluster(EmrHadoopJarStepAddRequest request) Adds a Hadoop Jar step to the existing cluster
EmrHiveStep	addHiveStepToEmrCluster(EmrHiveStepAddRequest request) Adds a hive step to the existing cluster
EmrOozieStep	addOozieStepToEmrCluster(EmrOozieStepAddRequest request) Adds a oozie step to the existing cluster
EmrPigStep	addPigStepToEmrCluster(EmrPigStepAddRequest request) Adds a Pig step to the existing cluster
EmrShellStep	addShellStepToEmrCluster(EmrShellStepAddRequest request) Adds a shell step to the existing cluster
EmrCluster	createEmrCluster(EmrClusterCreateRequest request) Creates a new EMR cluster.
EmrCluster	getEmrCluster(String namespace, String emrClusterDefinitionName, String emrClusterName, String emrClusterId, String emrStepId, Boolean verbose, Boolean retrieveOozieJobs, String accountId) Gets an existing EMR cluster details.
OozieWorkflowJob	getEmrOozieWorkflow(String namespace, String emrClusterDefinitionName, String emrClusterName, String oozieWorkflowJobId, Boolean verbose, String emrClusterId, String accountId) Retrieves an existing Oozie workflow job for the specified EMR cluster and workflow job ID.
OozieWorkflowJob	runOozieJobToEmrCluster(RunOozieWorkflowRequest request) Submits an oozie workflow to the existing cluster.
EmrCluster	terminateEmrCluster(String namespace, String emrClusterDefinitionName, String emrClusterName, Boolean overrideTerminationProtection, String emrClusterId, String accountId) Terminates an existing EMR cluster.

Methods inherited from class org.finra.herd.rest.HerdBaseController

getList, validateNoDuplicateQueryStringParams

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

EmrRestController

public EmrRestController()

Method Detail

getEmrCluster

@RequestMapping(value="/emrClusters/namespaces/{namespace}/emrClusterDefinitionNames/{emrClusterDefinitionName}/emrClusterNames/{emrClusterName}",
                method=GET)
 @Secured(value="FN_EMR_CLUSTERS_GET")
public EmrCluster getEmrCluster(@PathVariable(value="namespace")
                                                                                                                                                                                                                                                      String namespace,
                                                                                                                                                                                                                                                      @PathVariable(value="emrClusterDefinitionName")
                                                                                                                                                                                                                                                      String emrClusterDefinitionName,
                                                                                                                                                                                                                                                      @PathVariable(value="emrClusterName")
                                                                                                                                                                                                                                                      String emrClusterName,
                                                                                                                                                                                                                                                      @RequestParam(value="emrClusterId",required=false)
                                                                                                                                                                                                                                                      String emrClusterId,
                                                                                                                                                                                                                                                      @RequestParam(value="emrStepId",required=false)
                                                                                                                                                                                                                                                      String emrStepId,
                                                                                                                                                                                                                                                      @RequestParam(value="verbose",required=false,defaultValue="false")
                                                                                                                                                                                                                                                      Boolean verbose,
                                                                                                                                                                                                                                                      @RequestParam(value="retrieveOozieJobs",required=false,defaultValue="false")
                                                                                                                                                                                                                                                      Boolean retrieveOozieJobs,
                                                                                                                                                                                                                                                      @RequestParam(value="accountId",required=false)
                                                                                                                                                                                                                                                      String accountId)
                                                                                                                                                                                                                                               throws Exception

Gets an existing EMR cluster details.

Requires READ permission on namespace

Parameters:

namespace - the namespace

emrClusterDefinitionName - the EMR cluster definition name

emrClusterName - the EMR cluster name

emrClusterId - the cluster id of the cluster to get details

emrStepId - the step id of the step to get details

verbose - parameter for whether to return detailed information

retrieveOozieJobs - parameter for whether to retrieve oozie job information

accountId - the account Id

Returns:

the EMR Cluster object with details.

Throws:

Exception - if there was an error getting the EMR cluster.

createEmrCluster

@RequestMapping(value="/emrClusters",
                method=POST,
                consumes={"application/xml","application/json"})
 @Secured(value="FN_EMR_CLUSTERS_POST")
public EmrCluster createEmrCluster(@RequestBody
                                                                                                                                                                                                               EmrClusterCreateRequest request)
                                                                                                                                                                                                        throws Exception

Creates a new EMR cluster.

Requires EXECUTE permission on namespace

Parameters:

request - the information needed to create the EMR cluster.

Returns:

the created EMR cluster.

Throws:

Exception

terminateEmrCluster

@RequestMapping(value="/emrClusters/namespaces/{namespace}/emrClusterDefinitionNames/{emrClusterDefinitionName}/emrClusterNames/{emrClusterName}",
                method=DELETE)
 @Secured(value="FN_EMR_CLUSTERS_DELETE")
public EmrCluster terminateEmrCluster(@PathVariable(value="namespace")
                                                                                                                                                                                                                                                                  String namespace,
                                                                                                                                                                                                                                                                  @PathVariable(value="emrClusterDefinitionName")
                                                                                                                                                                                                                                                                  String emrClusterDefinitionName,
                                                                                                                                                                                                                                                                  @PathVariable(value="emrClusterName")
                                                                                                                                                                                                                                                                  String emrClusterName,
                                                                                                                                                                                                                                                                  @RequestParam(value="overrideTerminationProtection",required=false,defaultValue="false")
                                                                                                                                                                                                                                                                  Boolean overrideTerminationProtection,
                                                                                                                                                                                                                                                                  @RequestParam(value="emrClusterId",required=false)
                                                                                                                                                                                                                                                                  String emrClusterId,
                                                                                                                                                                                                                                                                  @RequestParam(value="accountId",required=false)
                                                                                                                                                                                                                                                                  String accountId)
                                                                                                                                                                                                                                                           throws Exception

Terminates an existing EMR cluster.

Requires EXECUTE permission on namespace

Parameters:

namespace - the namespace

emrClusterDefinitionName - the EMR cluster definition name

emrClusterName - the EMR cluster name

overrideTerminationProtection - parameter for whether to override termination protection

emrClusterId - EMR cluster ID

accountId - account Id

Returns:

the EMR cluster that was terminated

Throws:

Exception - if there was an error terminating the EMR cluster.

addShellStepToEmrCluster

@RequestMapping(value="/emrShellSteps",
                method=POST,
                consumes={"application/xml","application/json"})
 @Secured(value="FN_EMR_SHELL_STEPS_POST")
public EmrShellStep addShellStepToEmrCluster(@RequestBody
                                                                                                                                                                                                                              EmrShellStepAddRequest request)
                                                                                                                                                                                                                       throws Exception

Adds a shell step to the existing cluster

Requires EXECUTE permission on namespace

Parameters:

request - the information needed to add shell step to the EMR cluster.

Returns:

the created EMR shell step.

Throws:

Exception - if a shell step couldn't be added to the EMR cluster.

addHiveStepToEmrCluster

@RequestMapping(value="/emrHiveSteps",
                method=POST,
                consumes={"application/xml","application/json"})
 @Secured(value="FN_EMR_HIVE_STEPS_POST")
public EmrHiveStep addHiveStepToEmrCluster(@RequestBody
                                                                                                                                                                                                                          EmrHiveStepAddRequest request)
                                                                                                                                                                                                                   throws Exception

Adds a hive step to the existing cluster

Requires EXECUTE permission on namespace

Parameters:

request - the information needed to add hive step to the EMR cluster.

Returns:

the created EMR hive step.

Throws:

Exception

addPigStepToEmrCluster

@RequestMapping(value="/emrPigSteps",
                method=POST,
                consumes={"application/xml","application/json"})
 @Secured(value="FN_EMR_PIG_STEPS_POST")
public EmrPigStep addPigStepToEmrCluster(@RequestBody
                                                                                                                                                                                                                      EmrPigStepAddRequest request)
                                                                                                                                                                                                               throws Exception

Adds a Pig step to the existing cluster

Requires EXECUTE permission on namespace

Parameters:

request - the information needed to add Pig step to the EMR cluster.

Returns:

the created EMR Pig step.

Throws:

Exception

addOozieStepToEmrCluster

@RequestMapping(value="/emrOozieSteps",
                method=POST,
                consumes={"application/xml","application/json"})
 @Secured(value="FN_EMR_OOZIE_STEPS_POST")
public EmrOozieStep addOozieStepToEmrCluster(@RequestBody
                                                                                                                                                                                                                              EmrOozieStepAddRequest request)
                                                                                                                                                                                                                       throws Exception

Adds a oozie step to the existing cluster

Requires EXECUTE permission on namespace

Parameters:

request - the information needed to add oozie step to the EMR cluster.

Returns:

the created EMR oozie step.

Throws:

Exception - if an Oozie step couldn't be added to an EMR cluster.

addHadoopJarStepToEmrCluster

@RequestMapping(value="/emrHadoopJarSteps",
                method=POST,
                consumes={"application/xml","application/json"})
 @Secured(value="FN_EMR_HADOOP_JAR_STEPS_POST")
public EmrHadoopJarStep addHadoopJarStepToEmrCluster(@RequestBody
                                                                                                                                                                                                                                               EmrHadoopJarStepAddRequest request)
                                                                                                                                                                                                                                        throws Exception

Adds a Hadoop Jar step to the existing cluster

Requires EXECUTE permission on namespace

Parameters:

request - the information needed to add Hadoop Jar step to the EMR cluster.

Returns:

the created EMR Hadoop Jar step.

Throws:

Exception

addGroupsToEmrClusterMaster

@RequestMapping(value="/emrMasterSecurityGroups",
                method=POST,
                consumes={"application/xml","application/json"})
 @Secured(value="FN_EMR_MASTER_SECURITY_GROUPS_POST")
public EmrMasterSecurityGroup addGroupsToEmrClusterMaster(@RequestBody
                                                                                                                                                                                                                                                                EmrMasterSecurityGroupAddRequest request)
                                                                                                                                                                                                                                                         throws Exception

Adds security groups to the master node of an existing cluster

Requires WRITE permission on namespace

Parameters:

request - the information needed to add security groups to master node of the EMR cluster.

Returns:

the created EMR master groups.

Throws:

Exception

runOozieJobToEmrCluster

@RequestMapping(value="/emrOozieWorkflows",
                method=POST,
                consumes={"application/xml","application/json"})
 @Secured(value="FN_EMR_OOZIE_STEPS_POST")
public OozieWorkflowJob runOozieJobToEmrCluster(@RequestBody
                                                                                                                                                                                                                                     RunOozieWorkflowRequest request)
                                                                                                                                                                                                                              throws Exception

Submits an oozie workflow to the existing cluster.

Requires EXECUTE permission on namespace

Parameters:

request - the information needed to run oozie workflow to the EMR cluster.

Returns:

the oozie workflow job that was submitted.

Throws:

Exception - if an Oozie step couldn't be added to an EMR cluster.

getEmrOozieWorkflow

@RequestMapping(value="/emrOozieWorkflows/namespaces/{namespace}/emrClusterDefinitionNames/{emrClusterDefinitionName}/emrClusterNames/{emrClusterName}/oozieWorkflowJobIds/{oozieWorkflowJobId}",
                method=GET)
 @ResponseBody
 @Secured(value="FN_EMR_OOZIE_WORKFLOW_GET")
public OozieWorkflowJob getEmrOozieWorkflow(@PathVariable(value="namespace")
                                                                                                                                                                                                                                                                                                                                      String namespace,
                                                                                                                                                                                                                                                                                                                                      @PathVariable(value="emrClusterDefinitionName")
                                                                                                                                                                                                                                                                                                                                      String emrClusterDefinitionName,
                                                                                                                                                                                                                                                                                                                                      @PathVariable(value="emrClusterName")
                                                                                                                                                                                                                                                                                                                                      String emrClusterName,
                                                                                                                                                                                                                                                                                                                                      @PathVariable(value="oozieWorkflowJobId")
                                                                                                                                                                                                                                                                                                                                      String oozieWorkflowJobId,
                                                                                                                                                                                                                                                                                                                                      @RequestParam(value="verbose",required=false,defaultValue="false")
                                                                                                                                                                                                                                                                                                                                      Boolean verbose,
                                                                                                                                                                                                                                                                                                                                      @RequestParam(value="emrClusterId",required=false)
                                                                                                                                                                                                                                                                                                                                      String emrClusterId,
                                                                                                                                                                                                                                                                                                                                      @RequestParam(value="accountId",required=false)
                                                                                                                                                                                                                                                                                                                                      String accountId)
                                                                                                                                                                                                                                                                                                                               throws Exception

Retrieves an existing Oozie workflow job for the specified EMR cluster and workflow job ID.

Requires READ permission on namespace

Parameters:

namespace - Namespace of the EMR cluster

emrClusterDefinitionName - EMR cluster definition name

emrClusterName - EMR cluster name

oozieWorkflowJobId - Oozie workflow job ID

verbose - true to return more detailed information, false or null otherwise

emrClusterId - The EMR cluster ID

accountId - the account Id

Returns:

the Oozie workflow job information

Throws:

Exception - when errors occur, whether user or system