org.apache.nifinifi-cdc-mysql-nar2.1.0org.apache.nifinifi-standard-shared-nar2.1.02.0.0rel/nifi-2.1.0main1org.apache.nifi.cdc.mysql.processors.CaptureChangeMySQLPROCESSORRetrieves Change Data Capture (CDC) events from a MySQL database. CDC Events include INSERT, UPDATE, DELETE operations. Events are output as either a group of a specified number of events (the default is 1 so each event becomes its own flow file) or grouped as a full transaction (BEGIN to COMMIT). All events are ordered by the time at which the operation occurred. NOTE: If the processor is stopped before the specified number of events have been written to a flow file, the partial flow file will be output in order to maintain the consistency of the event stream.sqljdbccdcmysqltransactioneventcapture-change-mysql-hostsMySQL NodesA list of hostname (and optional port) entries corresponding to nodes in a MySQL cluster. The entries should be comma separated using a colon (if the port is to be specified) such as host1:port,host2:port,.... For example mysql.myhost.com:3306. The port need not be specified, when omitted the default MySQL port value of 3306 will be used. This processor will attempt to connect to the hosts in the list in order. If one node goes down and failover is enabled for the cluster, then the processor will connect to the active node (assuming its node entry is specified in this property).truefalsetrueENVIRONMENTfalsefalsecapture-change-mysql-driver-classMySQL Driver Class NameThe class name of the MySQL database driver classcom.mysql.jdbc.DrivertruefalsetrueENVIRONMENTfalsefalsecapture-change-mysql-driver-locationsMySQL Driver Location(s)Comma-separated list of files/folders and/or URLs containing the MySQL driver JAR and its dependencies (if any). For example '/var/tmp/mysql-connector-java-5.1.38-bin.jar'falsefalsetrueENVIRONMENTtruefalseMULTIPLEFILEDIRECTORYURLcapture-change-mysql-usernameUsernameUsername to access the MySQL clusterfalsefalsetrueENVIRONMENTfalsefalsecapture-change-mysql-passwordPasswordPassword to access the MySQL clusterfalsetruetrueENVIRONMENTfalsefalseevents-per-flowfile-strategyEvent Processing StrategySpecifies the strategy to use when writing events to FlowFile(s), such as 'Max Events Per FlowFile'MAX_EVENTS_PER_FLOWFILEMax Events Per FlowFileMAX_EVENTS_PER_FLOWFILEThis strategy causes at most the number of events specified in the 'Number of Events Per FlowFile' property to be written per FlowFile. If the processor is stopped before the specified number of events has been written (or the event queue becomes empty), the fewer number of events will still be written as a FlowFile before stopping.One Transaction Per FlowFileONE_TRANSACTION_PER_FLOWFILEThis strategy causes each event from a transaction (from BEGIN to COMMIT) to be written to a FlowFiletruefalsefalseNONEfalsefalsenumber-of-events-per-flowfileEvents Per FlowFileSpecifies how many events should be written to a single FlowFile. If the processor is stopped before the specified number of events has been written,the events will still be written as a FlowFile before stopping.1truefalsetrueENVIRONMENTfalsefalseevents-per-flowfile-strategyEvent Processing StrategyMAX_EVENTS_PER_FLOWFILEcapture-change-mysql-server-idServer IDThe client connecting to the MySQL replication group is actually a simplified replica (server), and the Server ID value must be unique across the whole replication group (i.e. different from any other Server ID being used by any primary or replica). Thus, each instance of CaptureChangeMySQL must have a Server ID unique across the replication group. If the Server ID is not specified, it defaults to 65535.falsefalsetrueENVIRONMENTfalsefalsecapture-change-mysql-db-name-patternDatabase/Schema Name PatternA regular expression (regex) for matching databases (or schemas, depending on your RDBMS' terminology) against the list of CDC events. The regex must match the database name as it is stored in the RDBMS. If the property is not set, the database name will not be used to filter the CDC events. NOTE: DDL events, even if they affect different databases, are associated with the database used by the session to execute the DDL. This means if a connection is made to one database, but the DDL is issued against another, then the connected database will be the one matched against the specified pattern.falsefalsefalseNONEfalsefalsecapture-change-mysql-name-patternTable Name PatternA regular expression (regex) for matching CDC events affecting matching tables. The regex must match the table name as it is stored in the database. If the property is not set, no events will be filtered based on table name.falsefalsefalseNONEfalsefalsecapture-change-mysql-max-wait-timeMax Wait TimeThe maximum amount of time allowed for a connection to be established, zero means there is effectively no limit.30 secondstruefalsetrueENVIRONMENTfalsefalsecapture-change-mysql-dist-map-cache-clientDistributed Map Cache Client - unusedThis is a legacy property that is no longer used to store table information, the processor will handle the table information (column names, types, etc.)org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.1.0falsefalsefalseNONEfalsefalsecapture-change-mysql-retrieve-all-recordsRetrieve All RecordsSpecifies whether to get all available CDC events, regardless of the current binlog filename and/or position. If binlog filename and position values are present in the processor's State, this property's value is ignored. This allows for 4 different configurations: 1) If binlog data is available in processor State, that is used to determine the start location and the value of Retrieve All Records is ignored. 2) If no binlog data is in processor State, then Retrieve All Records set to true means start at the beginning of the binlog history. 3) If no binlog data is in processor State and Initial Binlog Filename/Position are not set, then Retrieve All Records set to false means start at the end of the binlog history. 4) If no binlog data is in processor State and Initial Binlog Filename/Position are set, then Retrieve All Records set to false means start at the specified initial binlog file/position. To reset the behavior, clear the processor state (refer to the State Management section of the processor's documentation).truetruetruefalsefalsetruefalsefalseNONEfalsefalsecapture-change-mysql-include-begin-commitInclude Begin/Commit EventsSpecifies whether to emit events corresponding to a BEGIN or COMMIT event in the binary log. Set to true if the BEGIN/COMMIT events are necessary in the downstream flow, otherwise set to false, which suppresses generation of these events and can increase flow performance.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsecapture-change-mysql-include-ddl-eventsInclude DDL EventsSpecifies whether to emit events corresponding to Data Definition Language (DDL) events such as ALTER TABLE, TRUNCATE TABLE, e.g. in the binary log. Set to true if the DDL events are desired/necessary in the downstream flow, otherwise set to false, which suppresses generation of these events and can increase flow performance.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsecapture-change-mysql-init-seq-idInitial Sequence IDSpecifies an initial sequence identifier to use if this processor's State does not have a current sequence identifier. If a sequence identifier is present in the processor's State, this property is ignored. Sequence identifiers are monotonically increasing integers that record the order of flow files generated by the processor. They can be used with the EnforceOrder processor to guarantee ordered delivery of CDC events.falsefalsetrueENVIRONMENTfalsefalsecapture-change-mysql-init-binlog-filenameInitial Binlog FilenameSpecifies an initial binlog filename to use if this processor's State does not have a current binlog filename. If a filename is present in the processor's State or "Use GTID" property is set to false, this property is ignored. This can be used along with Initial Binlog Position to "skip ahead" if previous events are not desired. Note that NiFi Expression Language is supported, but this property is evaluated when the processor is configured, so FlowFile attributes may not be used. Expression Language is supported to enable the use of the environment properties.falsefalsetrueENVIRONMENTfalsefalsecapture-change-mysql-init-binlog-positionInitial Binlog PositionSpecifies an initial offset into a binlog (specified by Initial Binlog Filename) to use if this processor's State does not have a current binlog filename. If a filename is present in the processor's State or "Use GTID" property is false, this property is ignored. This can be used along with Initial Binlog Filename to "skip ahead" if previous events are not desired. Note that NiFi Expression Language is supported, but this property is evaluated when the processor is configured, so FlowFile attributes may not be used. Expression Language is supported to enable the use of the environment properties.falsefalsetrueENVIRONMENTfalsefalsecapture-change-mysql-use-gtidUse Binlog GTIDSpecifies whether to use Global Transaction ID (GTID) for binlog tracking. If set to true, processor's state of binlog file name and position is ignored. The main benefit of using GTID is to have much reliable failover than using binlog filename/position.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsecapture-change-mysql-init-gtidInitial Binlog GTIDSpecifies an initial GTID to use if this processor's State does not have a current GTID. If a GTID is present in the processor's State or "Use GTID" property is set to false, this property is ignored. This can be used to "skip ahead" if previous events are not desired. Note that NiFi Expression Language is supported, but this property is evaluated when the processor is configured, so FlowFile attributes may not be used. Expression Language is supported to enable the use of the environment properties.falsefalsetrueENVIRONMENTfalsefalseSSL ModeSSL ModeSSL Mode used when SSL Context Service configured supporting certificate verification optionsDISABLEDDISABLEDDISABLEDConnect without TLSPREFERREDPREFERREDConnect with TLS when server support enabled, otherwise connect without TLSREQUIREDREQUIREDConnect with TLS or fail when server support not enabledVERIFY_IDENTITYVERIFY_IDENTITYConnect with TLS or fail when server support not enabled. Verify server hostname matches presented X.509 certificate names or fail when not matchedtruefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceSSL Context Service supporting encrypted socket communicationorg.apache.nifi.ssl.SSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.1.0falsefalsefalseNONEfalsefalseSSL ModeSSL ModePREFERREDVERIFY_IDENTITYREQUIREDsuccessSuccessfully created FlowFile from SQL query result set.falsecdc.sequence.idA sequence identifier (i.e. strictly increasing integer value) specifying the order of the CDC event flow file relative to the other event flow file(s).cdc.event.typeA string indicating the type of CDC event that occurred, including (but not limited to) 'begin', 'insert', 'update', 'delete', 'ddl' and 'commit'.mime.typeThe processor outputs flow file content in JSON format, and sets the mime.type attribute to application/jsontruetrueInformation such as a 'pointer' to the current CDC event in the database is stored by this processor, such that it can continue from the same location if restarted.CLUSTERreference remote resourcesDatabase Driver Location can reference resources over HTTPINPUT_FORBIDDEN