org.cloudgraph.hbase.mapreduce

Class GraphRecordRecognizer

java.lang.Object

org.cloudgraph.hbase.mapreduce.GraphRecordRecognizer

public class GraphRecordRecognizer
extends Object

Iterates over HBase root table data for the current TableSplit, assembling data graphs based on the detailed selection criteria within a given query. Partially or fully assembled data graphs may be passed to binary expression recognizer and potentially screened from client GraphMapper extensions, potentially eliminating business logic dedicated to identifying specific records.

The recognizer uses a binary expression tree assembler which constructs an operator precedence map, then visits (traverses) the given predicate expression syntax tree depth-first using an adapted shunting-yard algorithm and assembles a resulting binary tree structure with expression nodes specific for detecting i.e. recognizing a data graph. In typical usage scenarios, a single expression tree is assembled once, and then used to evaluate (recognize) any number of data graphs.

The adapted shunting-yard algorithm in general uses a stack of operators and operands, and as new binary tree nodes are detected and created they are pushed onto the operand stack based on operator precedence. The resulting binary expression tree reflects the syntax of the underlying query expression including the precedence of its operators.

Since:

0.5.8

Author:

Scott Cinnamond

See Also:

GraphRecognizerSyntaxTreeAssembler, GraphRecognizerContext, GraphWritable, HBaseGraphAssembler

Field Summary

Fields

Modifier and Type	Field and Description
static String	LOG_PER_ROW_COUNT

Constructor Summary

Constructors

Constructor and Description
GraphRecordRecognizer(org.apache.hadoop.conf.Configuration configuration)

Method Summary

Modifier and Type	Method and Description
void	close() Closes the split.
org.apache.hadoop.hbase.io.ImmutableBytesWritable	getCurrentKey() Returns the current key.
GraphWritable	getCurrentValue() Returns the current value.
float	getProgress() The current progress of the record reader through its data.
void	initialize(org.apache.hadoop.mapreduce.InputSplit inputsplit, org.apache.hadoop.mapreduce.TaskAttemptContext context) Not done in constructor to allow for extension.
boolean	nextKeyValue() Positions the record reader to the next recognized record, scanning forward past any unrecognized records, not passing these records to Mapper clients.
void	restart(byte[] firstRow) Restart from survivable exceptions by creating a new scanner.
void	setScan(org.apache.hadoop.hbase.client.Scan scan) Sets the scan defining the actual details like columns etc.
void	setTable(org.apache.hadoop.hbase.client.Table table) Sets the HBase table.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG_PER_ROW_COUNT

public static final String LOG_PER_ROW_COUNT

See Also:

Constant Field Values

Constructor Detail

GraphRecordRecognizer

public GraphRecordRecognizer(org.apache.hadoop.conf.Configuration configuration)

Method Detail

restart

public void restart(byte[] firstRow)
             throws IOException

Restart from survivable exceptions by creating a new scanner.

Parameters:

firstRow - The first row to start at.

Throws:

IOException - When restarting fails.

setTable

public void setTable(org.apache.hadoop.hbase.client.Table table)

Sets the HBase table.

Parameters:

table - The root Table to scan.

setScan

public void setScan(org.apache.hadoop.hbase.client.Scan scan)

Sets the scan defining the actual details like columns etc.

Parameters:

scan - The scan to set.

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit inputsplit,
                       org.apache.hadoop.mapreduce.TaskAttemptContext context)
                throws IOException,
                       InterruptedException

Not done in constructor to allow for extension.

Throws:

IOException - , InterruptedException

InterruptedException

close

public void close()

Closes the split.

getCurrentKey

public org.apache.hadoop.hbase.io.ImmutableBytesWritable getCurrentKey()
                                                                throws IOException,
                                                                       InterruptedException

Returns the current key.

Returns:

The current key.

Throws:

IOException

InterruptedException - When the job is aborted.

getCurrentValue

public GraphWritable getCurrentValue()
                              throws IOException,
                                     InterruptedException

Returns the current value.

Returns:

The current value.

Throws:

IOException - When the value is faulty.

InterruptedException - When the job is aborted.

nextKeyValue

public boolean nextKeyValue()
                     throws IOException,
                            InterruptedException

Positions the record reader to the next recognized record, scanning forward past any unrecognized records, not passing these records to Mapper clients.

Returns:

true if there was another record.

Throws:

IOException - When reading the record failed.

InterruptedException - When the job was aborted.

getProgress

public float getProgress()

The current progress of the record reader through its data.

Returns:

A number between 0.0 and 1.0, the fraction of the data read.