Package org.apache.iotdb.pipe.api

Interface PipeProcessor

All Superinterfaces:

AutoCloseable, PipePlugin

public interface PipeProcessor
extends PipePlugin

PipeProcessor

PipeProcessor is used to filter and transform the Event formed by the PipeExtractor.

The lifecycle of a PipeProcessor is as follows:

• When a collaboration task is created, the KV pairs of `WITH PROCESSOR` clause in SQL are parsed and the validation method validate(PipeParameterValidator) will be called to validate the PipeParameters.
• Before the collaboration task starts, the method customize(PipeParameters, PipeProcessorRuntimeConfiguration) will be called to config the runtime behavior of the PipeProcessor.
• While the collaboration task is in progress:
  • PipeExtractor captures the Events and wraps them into three types of Event instances.
  • PipeProcessor processes the Event and then passes them to the PipeConnector. The following 3 methods will be called: process(TabletInsertionEvent, EventCollector), process(TsFileInsertionEvent, EventCollector) and process(Event, EventCollector).
  • PipeConnector serializes the Events into binaries and send them to sinks.
• When the collaboration task is cancelled (the `DROP PIPE` command is executed), the AutoCloseable.close() method will be called.

Method Summary

Modifier and Type	Method	Description
void	customize(PipeParameters parameters, PipeProcessorRuntimeConfiguration configuration)	This method is mainly used to customize PipeProcessor.
void	process(TabletInsertionEvent tabletInsertionEvent, EventCollector eventCollector)	This method is called to process the TabletInsertionEvent.
default void	process(TsFileInsertionEvent tsFileInsertionEvent, EventCollector eventCollector)	This method is called to process the TsFileInsertionEvent.
void	process(Event event, EventCollector eventCollector)	This method is called to process the generic Event.
void	validate(PipeParameterValidator validator)	This method is mainly used to validate PipeParameters and it is executed before customize(PipeParameters, PipeProcessorRuntimeConfiguration) is called.

Methods inherited from interface java.lang.AutoCloseable

close

Method Details

validate

void validate(PipeParameterValidator validator)
       throws Exception

This method is mainly used to validate PipeParameters and it is executed before customize(PipeParameters, PipeProcessorRuntimeConfiguration) is called.

Parameters:

validator - the validator used to validate PipeParameters

Throws:

Exception - if any parameter is not valid

customize

void customize(PipeParameters parameters,
 PipeProcessorRuntimeConfiguration configuration)
        throws Exception

This method is mainly used to customize PipeProcessor. In this method, the user can do the following things:

• Use PipeParameters to parse key-value pair attributes entered by the user.
• Set the running configurations in PipeProcessorRuntimeConfiguration.

This method is called after the method validate(PipeParameterValidator) is called and before the beginning of the events processing.

Parameters:

parameters - used to parse the input PipeParameters entered by the user

configuration - used to set the required properties of the running PipeProcessor

Throws:

Exception - the user can throw errors if necessary

process

void process(TabletInsertionEvent tabletInsertionEvent,
 EventCollector eventCollector)
      throws Exception

This method is called to process the TabletInsertionEvent.

Parameters:

tabletInsertionEvent - TabletInsertionEvent to be processed

eventCollector - used to collect result events after processing

Throws:

Exception - the user can throw errors if necessary

process

default void process(TsFileInsertionEvent tsFileInsertionEvent,
 EventCollector eventCollector)
              throws Exception

This method is called to process the TsFileInsertionEvent.

Parameters:

tsFileInsertionEvent - TsFileInsertionEvent to be processed

eventCollector - used to collect result events after processing

Throws:

Exception - the user can throw errors if necessary

process

void process(Event event,
 EventCollector eventCollector)
      throws Exception

This method is called to process the generic Event.

Parameters:

event - Event to be processed

eventCollector - used to collect result events after processing

Throws:

Exception - the user can throw errors if necessary