org.apache.hadoop.tools.dynamometer.workloadgenerator

Class CreateFileMapper

java.lang.Object

org.apache.hadoop.mapreduce.Mapper<KEYIN,VALUEIN,KEYOUT,VALUEOUT>

org.apache.hadoop.tools.dynamometer.workloadgenerator.WorkloadMapper<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable>

org.apache.hadoop.tools.dynamometer.workloadgenerator.CreateFileMapper

public class CreateFileMapper
extends WorkloadMapper<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable>

CreateFileMapper continuously creates 1 byte files for the specified duration to increase the number of file objects on the NN.

Configuration options available:

• "createfile.num-mappers" (required): Number of mappers to launch.
• "createfile.duration-min" (required): Number of minutes to induce workload for.
• "createfile.should-delete" (default: false): If true, delete the files after creating them. This can be useful for generating constant load without increasing the number of file objects.
• "createfile.file-parent-path" (default: "/tmp/createFileMapper"): The root directory in which to create files.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CreateFileMapper.CREATEFILECOUNTERS Custom Counter definitions.

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper

org.apache.hadoop.mapreduce.Mapper.Context

Field Summary

Fields

Modifier and Type	Field and Description
static String	DURATION_MIN_KEY
static String	FILE_PARENT_PATH_DEFAULT
static String	FILE_PARENT_PATH_KEY
static String	NUM_MAPPERS_KEY
static boolean	SHOULD_DELETE_DEFAULT
static String	SHOULD_DELETE_KEY

Constructor Summary

Constructors

Constructor and Description
CreateFileMapper()

Method Summary

Modifier and Type	Method and Description
List<String>	getConfigDescriptions() Get a list of the description of each configuration that this mapper accepts.
String	getDescription() Get the description of the behavior of this mapper.
void	map(org.apache.hadoop.io.NullWritable key, org.apache.hadoop.io.NullWritable value, org.apache.hadoop.mapreduce.Mapper.Context mapperContext)
boolean	verifyConfigurations(org.apache.hadoop.conf.Configuration confToVerify) Verify that the provided configuration contains all configurations required by this mapper.

Methods inherited from class org.apache.hadoop.tools.dynamometer.workloadgenerator.WorkloadMapper

configureJob

Methods inherited from class org.apache.hadoop.mapreduce.Mapper

cleanup, run, setup

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

NUM_MAPPERS_KEY

public static final String NUM_MAPPERS_KEY

See Also:

Constant Field Values

DURATION_MIN_KEY

public static final String DURATION_MIN_KEY

See Also:

Constant Field Values

FILE_PARENT_PATH_KEY

public static final String FILE_PARENT_PATH_KEY

See Also:

Constant Field Values

FILE_PARENT_PATH_DEFAULT

public static final String FILE_PARENT_PATH_DEFAULT

See Also:

Constant Field Values

SHOULD_DELETE_KEY

public static final String SHOULD_DELETE_KEY

See Also:

Constant Field Values

SHOULD_DELETE_DEFAULT

public static final boolean SHOULD_DELETE_DEFAULT

See Also:

Constant Field Values

Constructor Detail

CreateFileMapper

public CreateFileMapper()

Method Detail

getDescription

public String getDescription()

Description copied from class: WorkloadMapper

Get the description of the behavior of this mapper.

Specified by:

getDescription in class WorkloadMapper<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable>

Returns:

description string.

getConfigDescriptions

public List<String> getConfigDescriptions()

Description copied from class: WorkloadMapper

Get a list of the description of each configuration that this mapper accepts.

Specified by:

getConfigDescriptions in class WorkloadMapper<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable>

Returns:

list of the description of each configuration.

verifyConfigurations

public boolean verifyConfigurations(org.apache.hadoop.conf.Configuration confToVerify)

Description copied from class: WorkloadMapper

Verify that the provided configuration contains all configurations required by this mapper.

Specified by:

verifyConfigurations in class WorkloadMapper<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable>

Parameters:

confToVerify - configuration.

Returns:

whether or not all configurations required are provided.

map

public void map(org.apache.hadoop.io.NullWritable key,
                org.apache.hadoop.io.NullWritable value,
                org.apache.hadoop.mapreduce.Mapper.Context mapperContext)
         throws IOException,
                InterruptedException

Overrides:

map in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable>

Throws:

IOException

InterruptedException