public class ParallelGraphSliceAssembler extends DistributedAssembler
Edges within a collection are "recognized" as members of a slice based on a binary expression syntax tree assembled from the path predicate describing the slice. While a path predicate may be quite complex resulting in any number of logical, relational or wildcard binary expressions, a single slice syntax tree is used to evaluate any number of edges within a collection. Where edge opposite graph nodes are found within the current row, an edge recognizer is used, but where edge opposite graph nodes are found "outside" the current row, a graph recognizer is used.
Parallel processing proceeds as a breadth-first traversal and subgraph tasks are dynamically added based on thread availability within a shared thread pool.
While the result graph may be of any arbitrary size or depth, because the traversal is breadth-first, many tasks are typically spawned at the "base" of the graph, exhausting the available pool threads. Each subgraph task can spawn further sub tasks based on thread availability, but typically this means each task will traverse and process a healthy segment of the total graph. Since the actual size or depth of the result graph is not known until discovered on traversal, a fixed number of parallel tasks cannot be initially created, but must be dynamically spawned during graph discovery.
The assembly is triggered by calling theParallelGraphSliceAssembler#assemble(Result resultRow) method which
recursively reads HBase keys and values re-constituting the data graph. The
assembly traversal is driven by HBase column values representing the original
edges or containment structure of the graph.
Since every column key in HBase must be unique, and a data graph may contain any number of nodes, a column key factory is used both to persist as well as re-constitute a graph. A minimal amount of "state" information is therefore stored with each graph which maps user readable sequence numbers (which are used in column keys) to UUID values. The nodes of the resulting data graph are re-created with the original UUID values.
LocalEdgeRecognizerSyntaxTreeAssembler,
LocalEdgeRecognizerContext,
GraphRecognizerSyntaxTreeAssembler,
GraphRecognizerContext,
GraphSliceSupport,
StatefullColumnKeyFactory,
ParallelSliceSubgraphTaskdistributedReadercharset, graph, keyFactories, root, rootTableReader, rootType, selection, snapshotDate| Constructor and Description |
|---|
ParallelGraphSliceAssembler(org.plasma.sdo.PlasmaType rootType,
org.plasma.query.collector.Selection selection,
DistributedReader distributedReader,
Timestamp snapshotDate,
ThreadPoolMappingProps config)
Constructor.
|
| Modifier and Type | Method and Description |
|---|---|
protected void |
assemble(org.plasma.sdo.PlasmaDataObject target,
long targetSequence,
EdgeReader sourceCollection,
org.plasma.sdo.PlasmaDataObject source,
org.plasma.sdo.PlasmaProperty sourceProperty,
RowReader rowReader,
int level)
Creates a single task and begins the traversal from root in the current
thread.
|
assemble, assembleEdge, assembleExternalEdge, clearassembleData, assembleUUID, createChild, createChild, createChild, createRoot, fetchGraph, fetchRootType, fetchRootUUID, fetchType, fetchUUID, findMetaDataColumnValue, findMetaDataColumnValue, findRootType, findRootUUID, getDataColumnValue, getDataColumnValue, getDataGraph, getKeyFactory, getMetaDataColumnValue, getMetaDataColumnValue, getProperties, getProperties, linkpublic ParallelGraphSliceAssembler(org.plasma.sdo.PlasmaType rootType,
org.plasma.query.collector.Selection selection,
DistributedReader distributedReader,
Timestamp snapshotDate,
ThreadPoolMappingProps config)
rootType - the SDO root type for the result data graphselection - selected SDO properties. Properties are mapped by selected types
required in the result graph.snapshotDate - the query snapshot date which is populated into every data object
in the result data graph.distributedReader - the distributed readerminPoolSize - the minimum or core size of the underlying thread pool used for
all tasks executed under this assemblermaxPoolSize - the maximum size of the underlying thread pool used for all tasks
executed under this assemblerprotected void assemble(org.plasma.sdo.PlasmaDataObject target,
long targetSequence,
EdgeReader sourceCollection,
org.plasma.sdo.PlasmaDataObject source,
org.plasma.sdo.PlasmaProperty sourceProperty,
RowReader rowReader,
int level)
throws IOException
assemble in class DistributedAssemblertargetSequence - the current data object sequencesource - the source or parent data objectsourceProperty - the source (reference) propertyrowReader - the current row readerlevel - the current graph levelIOException - if a remote or network exception occurs.Copyright © 2021. All Rights Reserved.