AuditReplayMapper (Apache Hadoop Dynamometer Workload Simulator 3.3.5 API)

java.lang.Object
- org.apache.hadoop.mapreduce.Mapper<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
- - org.apache.hadoop.tools.dynamometer.workloadgenerator.WorkloadMapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
  - - org.apache.hadoop.tools.dynamometer.workloadgenerator.audit.AuditReplayMapper

```
public class AuditReplayMapper
extends WorkloadMapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
```
AuditReplayMapper replays the given audit trace against the NameNode under test. Each mapper spawns a number of threads equal to the "auditreplay.num-threads" configuration (by default 1) to use for replaying. Each mapper reads a single input file which will be consumed by all of the available threads. A FileInputFormat with splitting disabled is used so any files present in the input path directory (given by the "auditreplay.input-path" configuration) will be used as input; one file per mapper. The expected format of these files is determined by the value of the "auditreplay.command-parser.class" configuration, which defaults to AuditLogDirectParser.
This generates a number of Counter values which can be used to get information into the replay, including the number of commands replayed, how many of them were "invalid" (threw an exception), how many were "late" (replayed later than they should have been), and the latency (from client perspective) of each command. If there are a large number of "late" commands, you likely need to increase the number of threads used and/or the number of mappers.
By default, commands will be replayed at the same rate as they were originally performed. However a rate factor can be specified via the "auditreplay.rate-factor" configuration; all of the (relative) timestamps will be divided by this rate factor, effectively changing the rate at which they are replayed. For example, a rate factor of 2 would make the replay occur twice as fast, and a rate factor of 0.5 would make it occur half as fast.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`AuditReplayMapper.CommandType` Define the type of command, either read or write.
`static class`	`AuditReplayMapper.ReplayCommand` Definitions of the various commands that can be replayed.
`static class`	`AuditReplayMapper.REPLAYCOUNTERS` `Counter` definitions for replay.

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper
org.apache.hadoop.mapreduce.Mapper.Context

Field Summary

Fields
Modifier and Type	Field and Description
`static Class<AuditLogDirectParser>`	`COMMAND_PARSER_DEFAULT`
`static String`	`COMMAND_PARSER_KEY`
`static boolean`	`CREATE_BLOCKS_DEFAULT`
`static String`	`CREATE_BLOCKS_KEY`
`static String`	`INDIVIDUAL_COMMANDS_COUNT_SUFFIX`
`static String`	`INDIVIDUAL_COMMANDS_COUNTER_GROUP`
`static String`	`INDIVIDUAL_COMMANDS_INVALID_SUFFIX`
`static String`	`INDIVIDUAL_COMMANDS_LATENCY_SUFFIX`
`static String`	`INPUT_PATH_KEY`
`static int`	`NUM_THREADS_DEFAULT`
`static String`	`NUM_THREADS_KEY`
`static String`	`OUTPUT_PATH_KEY`
`static double`	`RATE_FACTOR_DEFAULT`
`static String`	`RATE_FACTOR_KEY`

Constructor Summary

Constructors
Constructor and Description

AuditReplayMapper()

Constructors
Constructor and Description
`AuditReplayMapper()`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`cleanup(org.apache.hadoop.mapreduce.Mapper.Context context)`
`void`	`configureJob(org.apache.hadoop.mapreduce.Job job)` Setup input and output formats and optional reducer.
`List<String>`	`getConfigDescriptions()` Get a list of the description of each configuration that this mapper accepts.
`String`	`getDescription()` Get the description of the behavior of this mapper.
`void`	`map(org.apache.hadoop.io.LongWritable lineNum, org.apache.hadoop.io.Text inputLine, org.apache.hadoop.mapreduce.Mapper.Context context)`
`void`	`setup(org.apache.hadoop.mapreduce.Mapper.Context context)`
`boolean`	`verifyConfigurations(org.apache.hadoop.conf.Configuration conf)` Verify that the provided configuration contains all configurations required by this mapper.

Methods inherited from class org.apache.hadoop.mapreduce.Mapper
run

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - INPUT_PATH_KEY
```
public static final String INPUT_PATH_KEY
```
    See Also:
    
    Constant Field Values
  - OUTPUT_PATH_KEY
```
public static final String OUTPUT_PATH_KEY
```
    See Also:
    
    Constant Field Values
  - NUM_THREADS_KEY
```
public static final String NUM_THREADS_KEY
```
    See Also:
    
    Constant Field Values
  - NUM_THREADS_DEFAULT
```
public static final int NUM_THREADS_DEFAULT
```
    See Also:
    
    Constant Field Values
  - CREATE_BLOCKS_KEY
```
public static final String CREATE_BLOCKS_KEY
```
    See Also:
    
    Constant Field Values
  - CREATE_BLOCKS_DEFAULT
```
public static final boolean CREATE_BLOCKS_DEFAULT
```
    See Also:
    
    Constant Field Values
  - RATE_FACTOR_KEY
```
public static final String RATE_FACTOR_KEY
```
    See Also:
    
    Constant Field Values
  - RATE_FACTOR_DEFAULT
```
public static final double RATE_FACTOR_DEFAULT
```
    See Also:
    
    Constant Field Values
  - COMMAND_PARSER_KEY
```
public static final String COMMAND_PARSER_KEY
```
    See Also:
    
    Constant Field Values
  - COMMAND_PARSER_DEFAULT
```
public static final Class<AuditLogDirectParser> COMMAND_PARSER_DEFAULT
```
  - INDIVIDUAL_COMMANDS_COUNTER_GROUP
```
public static final String INDIVIDUAL_COMMANDS_COUNTER_GROUP
```
    See Also:
    
    Constant Field Values
  - INDIVIDUAL_COMMANDS_LATENCY_SUFFIX
```
public static final String INDIVIDUAL_COMMANDS_LATENCY_SUFFIX
```
    See Also:
    
    Constant Field Values
  - INDIVIDUAL_COMMANDS_INVALID_SUFFIX
```
public static final String INDIVIDUAL_COMMANDS_INVALID_SUFFIX
```
    See Also:
    
    Constant Field Values
  - INDIVIDUAL_COMMANDS_COUNT_SUFFIX
```
public static final String INDIVIDUAL_COMMANDS_COUNT_SUFFIX
```
    See Also:
    
    Constant Field Values
- Constructor Detail
  - AuditReplayMapper
```
public AuditReplayMapper()
```
- Method Detail
  - getDescription
```
public String getDescription()
```
    Description copied from class: WorkloadMapper
    
    Get the description of the behavior of this mapper.
    
    Specified by:
    
    getDescription in class WorkloadMapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
    
    Returns:
    
    description string.
  - getConfigDescriptions
```
public List<String> getConfigDescriptions()
```
    Description copied from class: WorkloadMapper
    
    Get a list of the description of each configuration that this mapper accepts.
    
    Specified by:
    
    getConfigDescriptions in class WorkloadMapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
    
    Returns:
    
    list of the description of each configuration.
  - verifyConfigurations
```
public boolean verifyConfigurations(org.apache.hadoop.conf.Configuration conf)
```
    Description copied from class: WorkloadMapper
    
    Verify that the provided configuration contains all configurations required by this mapper.
    
    Specified by:
    
    verifyConfigurations in class WorkloadMapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
    
    Parameters:
    
    conf - configuration.
    
    Returns:
    
    whether or not all configurations required are provided.
  - setup
```
public void setup(org.apache.hadoop.mapreduce.Mapper.Context context)
           throws IOException
```
    Overrides:
    
    setup in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
    
    Throws:
    
    IOException
  - map
```
public void map(org.apache.hadoop.io.LongWritable lineNum,
                org.apache.hadoop.io.Text inputLine,
                org.apache.hadoop.mapreduce.Mapper.Context context)
         throws IOException,
                InterruptedException
```
    Overrides:
    
    map in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
    
    Throws:
    
    IOException
    
    InterruptedException
  - cleanup
```
public void cleanup(org.apache.hadoop.mapreduce.Mapper.Context context)
             throws InterruptedException,
                    IOException
```
    Overrides:
    
    cleanup in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
    
    Throws:
    
    InterruptedException
    
    IOException
  - configureJob
```
public void configureJob(org.apache.hadoop.mapreduce.Job job)
```
    Description copied from class: WorkloadMapper
    
    Setup input and output formats and optional reducer.
    
    Overrides:
    
    configureJob in class WorkloadMapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>

Class AuditReplayMapper

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper

Field Summary

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.mapreduce.Mapper

Methods inherited from class java.lang.Object

Field Detail

INPUT_PATH_KEY

OUTPUT_PATH_KEY

NUM_THREADS_KEY

NUM_THREADS_DEFAULT

CREATE_BLOCKS_KEY

CREATE_BLOCKS_DEFAULT

RATE_FACTOR_KEY

RATE_FACTOR_DEFAULT

COMMAND_PARSER_KEY

COMMAND_PARSER_DEFAULT

INDIVIDUAL_COMMANDS_COUNTER_GROUP

INDIVIDUAL_COMMANDS_LATENCY_SUFFIX

INDIVIDUAL_COMMANDS_INVALID_SUFFIX

INDIVIDUAL_COMMANDS_COUNT_SUFFIX

Constructor Detail

AuditReplayMapper

Method Detail

getDescription

getConfigDescriptions

verifyConfigurations

setup

map

cleanup

configureJob