org.apache.nifinifi-elasticsearch-restapi-nar2.2.0org.apache.nifinifi-elasticsearch-client-service-api-nar2.2.02.0.0rel/nifi-2.2.0main1org.apache.nifi.processors.elasticsearch.ConsumeElasticsearchPROCESSORA processor that repeatedly runs a paginated query against a field using a Range query to consume new Documents from an Elasticsearch index/query. The processor will retrieve multiple pages of results until either no more results are available or the Pagination Keep Alive expiration is reached, after which the Range query will automatically update the field constraint based on the last retrieved Document value.elasticsearchelasticsearch5elasticsearch6elasticsearch7elasticsearch8queryscrollpagesearchjsones-rest-range-fieldRange Query FieldField to be tracked as part of an Elasticsearch Range query using a "gt" bound match. This field must exist within the Elasticsearch document for it to be retrieved.truefalsefalseNONEfalsefalsees-rest-sort-orderSort OrderThe order in which to sort the "Range Query Field". A "sort" clause for the "Range Query Field" field will be prepended to any provided "Sort" clauses. If a "sort" clause already exists for the "Range Query Field" field, it will not be updated.ascascascdescdesctruefalsefalseNONEfalsefalsees-rest-range-initial-valueInitial ValueThe initial value to use for the query if the processor has not run previously. If the processor has run previously and stored a value in its state, this property will be ignored. If no value is provided, and the processor has not previously run, no Range query bounds will be used, i.e. all documents will be retrieved in the specified "Sort Order".falsefalsefalseNONEfalsefalsees-rest-range-formatInitial Value Date FormatIf the "Range Query Field" is a Date field, convert the "Initial Value" to a date with this format. If not specified, Elasticsearch will use the date format provided by the "Range Query Field"'s mapping. For valid syntax, see https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.htmlfalsefalsefalseNONEfalsefalsees-rest-range-initial-valueInitial Valuees-rest-range-time-zoneInitial Value Date Time ZoneIf the "Range Query Field" is a Date field, convert the "Initial Value" to UTC with this time zone. Valid values are ISO 8601 UTC offsets, such as "+01:00" or "-08:00", and IANA time zone IDs, such as "Europe/London".falsefalsefalseNONEfalsefalsees-rest-range-initial-valueInitial Valuees-rest-additional-filtersAdditional FiltersOne or more query filters in JSON syntax, not Lucene syntax. Ex: [{"match":{"somefield":"somevalue"}}, {"match":{"anotherfield":"anothervalue"}}]. These filters wil be used as part of a Bool query's filter.falsefalsefalseNONEfalsefalsees-rest-sizeSizeThe maximum number of documents to retrieve in the query. If the query is paginated, this "size" applies to each page of the query, not the "size" of the entire result set.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsees-rest-query-sortSortSort results by one or more fields, in JSON syntax. Ex: [{"price" : {"order" : "asc", "mode" : "avg"}}, {"post_date" : {"format": "strict_date_optional_time_nanos"}}]falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsees-rest-query-aggsAggregationsOne or more query aggregations (or "aggs"), in JSON syntax. Ex: {"items": {"terms": {"field": "product", "size": 10}}}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsees-rest-query-fieldsFieldsFields of indexed documents to be retrieved, in JSON syntax. Ex: ["user.id", "http.response.*", {"field": "@timestamp", "format": "epoch_millis"}]falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsees-rest-query-script-fieldsScript FieldsFields to created using script evaluation at query runtime, in JSON syntax. Ex: {"test1": {"script": {"lang": "painless", "source": "doc['price'].value * 2"}}, "test2": {"script": {"lang": "painless", "source": "doc['price'].value * params.factor", "params": {"factor": 2.0}}}}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-query-attributeQuery AttributeIf set, the executed query will be set on each result flowfile in the specified attribute.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-fetch-indexIndexThe name of the index to use.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-typeTypeThe type of this document (used by Elasticsearch for indexing and searching).falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax JSON Field String LengthMax JSON Field String LengthThe maximum allowed length of a string value when parsing a JSON document or attribute.20 MBtruefalsefalseNONEfalsefalseel-rest-client-serviceClient ServiceAn Elasticsearch client service to use for running queries.org.apache.nifi.elasticsearch.ElasticSearchClientServiceorg.apache.nifinifi-elasticsearch-client-service-api-nar2.2.0truefalsefalseNONEfalsefalseel-rest-split-up-hitsSearch Results SplitOutput a flowfile containing all hits or one flowfile for each individual hit or one flowfile containing all hits from all paged responses.splitUp-noPER_HITsplitUp-yesFlowfile per hit.PER_RESPONSEsplitUp-noFlowfile per response.PER_QUERYsplitUp-queryCombine results from all query responses (one flowfile per entire paginated result set of hits). Note that aggregations cannot be paged, they are generated across the entire result set and returned as part of the first page. Results are output with one JSON object per line (allowing hits to be combined from multiple pages without loading all results into memory).truefalsefalseNONEfalsefalseel-rest-format-hitsSearch Results FormatFormat of Hits output.FULLFULLFULLContains full Elasticsearch Hit, including Document Source and Metadata.SOURCE_ONLYSOURCE_ONLYDocument Source only (where present).METADATA_ONLYMETADATA_ONLYHit Metadata only.truefalsefalseNONEfalsefalseel-rest-split-up-aggregationsAggregation Results SplitOutput a flowfile containing all aggregations or one flowfile for each individual aggregation.splitUp-noPER_HITsplitUp-yesFlowfile per hit.PER_RESPONSEsplitUp-noFlowfile per response.truefalsefalseNONEfalsefalseel-rest-format-aggregationsAggregation Results FormatFormat of Aggregation output.FULLFULLFULLContains full Elasticsearch Aggregation, including Buckets and Metadata.BUCKETS_ONLYBUCKETS_ONLYBucket Content only.METADATA_ONLYMETADATA_ONLYAggregation Metadata only.truefalsefalseNONEfalsefalseel-rest-output-no-hitsOutput No HitsOutput a "hits" flowfile even if no hits found for query. If true, an empty "hits" flowfile will be output even if "aggregations" are output.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseel-rest-pagination-typePagination TypePagination method to use. Not all types are available for all Elasticsearch versions, check the Elasticsearch docs to confirm which are applicable and recommended for your service.pagination-scrollSCROLLpagination-scrollUse Elasticsearch "_scroll" API to page results. Does not accept additional query parameters.SEARCH_AFTERpagination-search_afterUse Elasticsearch "search_after" _search API to page sorted results.POINT_IN_TIMEpagination-pitUse Elasticsearch (7.10+ with XPack) "point in time" _search API to page sorted results. Not available for use with AWS OpenSearch.truefalsefalseNONEfalsefalseel-rest-pagination-keep-alivePagination Keep AlivePagination "keep_alive" period. Period Elasticsearch will keep the scroll/pit cursor alive in between requests (this is not the time expected for all pages to be returned, but the maximum allowed time for requests between page retrievals).10 minstruefalsefalseNONEfalsefalseThe name of a URL query parameter to addThe value of the URL query parameterAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing. These parameters will override any matching parameters in the query request body. For SCROLL type queries, these parameters are only used in the initial (first page) query as the Elasticsearch Scroll API does not support the same query parameters for subsequent pages of data.ENVIRONMENThitsSearch hits are routed to this relationship.falseaggregationsAggregations are routed to this relationship.falsemime.typeapplication/jsonpage.numberThe number of the page (request), starting from 1, in which the results were returned that are in the output flowfilehit.countThe number of hits that are in the output flowfileelasticsearch.query.errorThe error message provided by Elasticsearch if there is an error querying the index.truetrueThe pagination state (scrollId, searchAfter, pitId, hitCount, pageCount, pageExpirationTimestamp, trackingRangeValue) is retained in between invocations of this processor until the Scroll/PiT has expired (when the current time is later than the last query execution plus the Pagination Keep Alive interval).CLUSTERINPUT_FORBIDDENMEMORYCare should be taken on the size of each page because each response from Elasticsearch will be loaded into memory all at once and converted into the resulting flowfiles.org.apache.nifi.processors.elasticsearch.SearchElasticsearchorg.apache.nifi.processors.elasticsearch.PaginatedJsonQueryElasticsearchTIMER_DRIVEN1 min1org.apache.nifi.processors.elasticsearch.DeleteByQueryElasticsearchPROCESSORDelete from an Elasticsearch index using a query. The query can be loaded from a flowfile body or from the Query parameter.elasticelasticsearchelasticsearch5elasticsearch6elasticsearch7elasticsearch8deletequeryel-rest-query-definition-styleQuery Definition StyleHow the JSON Query will be defined for use by the processor.fullFULL_QUERYfullProvide the full Query.BUILD_QUERYbuildBuild the Query from separate JSON objects.truefalsefalseNONEfalsefalseel-rest-queryQueryA query in JSON syntax, not Lucene syntax. Ex: {"query":{"match":{"somefield":"somevalue"}}}. If this parameter is not set, the query will be read from the flowfile content. If the query (property and flowfile content) is empty, a default empty JSON Object will be used, which will result in a "match_all" query in Elasticsearch.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylefullel-rest-query-clauseQuery ClauseA "query" clause in JSON syntax, not Lucene syntax. Ex: {"match":{"somefield":"somevalue"}}. If the query is empty, a default JSON Object will be used, which will result in a "match_all" query in Elasticsearch.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildel-query-attributeQuery AttributeIf set, the executed query will be set on each result flowfile in the specified attribute.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-fetch-indexIndexThe name of the index to use.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-typeTypeThe type of this document (used by Elasticsearch for indexing and searching).falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax JSON Field String LengthMax JSON Field String LengthThe maximum allowed length of a string value when parsing a JSON document or attribute.20 MBtruefalsefalseNONEfalsefalseel-rest-client-serviceClient ServiceAn Elasticsearch client service to use for running queries.org.apache.nifi.elasticsearch.ElasticSearchClientServiceorg.apache.nifinifi-elasticsearch-client-service-api-nar2.2.0truefalsefalseNONEfalsefalseThe name of a URL query parameter to addThe value of the URL query parameterAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing. These parameters will override any matching parameters in the query request bodyFLOWFILE_ATTRIBUTESfailureIf the "by query" operation fails, and a flowfile was read, it will be sent to this relationship.falsesuccessIf the "by query" operation succeeds, and a flowfile was read, it will be sent to this relationship.falseretryAll flowfiles that fail due to server/cluster availability go to this relationship.falseelasticsearch.delete.tookThe amount of time that it took to complete the delete operation in ms.elasticsearch.delete.errorThe error message provided by Elasticsearch if there is an error running the delete.INPUT_ALLOWEDorg.apache.nifi.processors.elasticsearch.GetElasticsearchPROCESSORElasticsearch get processor that uses the official Elastic REST client libraries to fetch a single document from Elasticsearch by _id. Note that the full body of the document will be read into memory before being written to a FlowFile for transfer.jsonelasticsearchelasticsearch5elasticsearch6elasticsearch7elasticsearch8putindexrecordget-es-idDocument IdThe _id of the document to retrieve.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-fetch-indexIndexThe name of the index to use.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-typeTypeThe type of this document (used by Elasticsearch for indexing and searching).falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseget-es-destinationDestinationIndicates whether the retrieved document is written to the FlowFile content or a FlowFile attribute.flowfile-contentFlowFile Contentflowfile-contentOutput the retrieved document as the FlowFile content.FlowFile Attributeflowfile-attributeOutput the retrieved document as a FlowFile attribute specified by the Attribute Name.truefalsefalseNONEfalsefalseget-es-attribute-nameAttribute NameThe name of the FlowFile attribute to use for the retrieved document output.elasticsearch.doctruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseget-es-destinationDestinationflowfile-attributeel-rest-client-serviceClient ServiceAn Elasticsearch client service to use for running queries.org.apache.nifi.elasticsearch.ElasticSearchClientServiceorg.apache.nifinifi-elasticsearch-client-service-api-nar2.2.0truefalsefalseNONEfalsefalseThe name of a URL query parameter to addThe value of the URL query parameterAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing.FLOWFILE_ATTRIBUTESretryAll flowfiles that fail due to server/cluster availability go to this relationship.falsenot_foundA FlowFile is routed to this relationship if the specified document does not exist in the Elasticsearch cluster.falsedocumentFetched documents are routed to this relationship.falsefailureAll flowfiles that fail for reasons unrelated to server availability go to this relationship.falsefilenameThe filename attribute is set to the document identifierelasticsearch.indexThe Elasticsearch index containing the documentelasticsearch.typeThe Elasticsearch document typeelasticsearch.get.errorThe error message provided by Elasticsearch if there is an error fetching the document.INPUT_ALLOWEDorg.apache.nifi.processors.elasticsearch.JsonQueryElasticsearchorg.apache.nifi.processors.elasticsearch.JsonQueryElasticsearchPROCESSORA processor that allows the user to run a query (with aggregations) written with the Elasticsearch JSON DSL. It does not automatically paginate queries for the user. If an incoming relationship is added to this processor, it will use the flowfile's content for the query. Care should be taken on the size of the query because the entire response from Elasticsearch will be loaded into memory all at once and converted into the resulting flowfiles.elasticsearchelasticsearch5elasticsearch6elasticsearch7elasticsearch8queryreadgetjsonel-rest-query-definition-styleQuery Definition StyleHow the JSON Query will be defined for use by the processor.fullFULL_QUERYfullProvide the full Query.BUILD_QUERYbuildBuild the Query from separate JSON objects.truefalsefalseNONEfalsefalseel-rest-queryQueryA query in JSON syntax, not Lucene syntax. Ex: {"query":{"match":{"somefield":"somevalue"}}}. If this parameter is not set, the query will be read from the flowfile content. If the query (property and flowfile content) is empty, a default empty JSON Object will be used, which will result in a "match_all" query in Elasticsearch.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylefullel-rest-query-clauseQuery ClauseA "query" clause in JSON syntax, not Lucene syntax. Ex: {"match":{"somefield":"somevalue"}}. If the query is empty, a default JSON Object will be used, which will result in a "match_all" query in Elasticsearch.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-sizeSizeThe maximum number of documents to retrieve in the query. If the query is paginated, this "size" applies to each page of the query, not the "size" of the entire result set.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-query-sortSortSort results by one or more fields, in JSON syntax. Ex: [{"price" : {"order" : "asc", "mode" : "avg"}}, {"post_date" : {"format": "strict_date_optional_time_nanos"}}]falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-query-aggsAggregationsOne or more query aggregations (or "aggs"), in JSON syntax. Ex: {"items": {"terms": {"field": "product", "size": 10}}}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-query-fieldsFieldsFields of indexed documents to be retrieved, in JSON syntax. Ex: ["user.id", "http.response.*", {"field": "@timestamp", "format": "epoch_millis"}]falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-query-script-fieldsScript FieldsFields to created using script evaluation at query runtime, in JSON syntax. Ex: {"test1": {"script": {"lang": "painless", "source": "doc['price'].value * 2"}}, "test2": {"script": {"lang": "painless", "source": "doc['price'].value * params.factor", "params": {"factor": 2.0}}}}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildel-query-attributeQuery AttributeIf set, the executed query will be set on each result flowfile in the specified attribute.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-fetch-indexIndexThe name of the index to use.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-typeTypeThe type of this document (used by Elasticsearch for indexing and searching).falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax JSON Field String LengthMax JSON Field String LengthThe maximum allowed length of a string value when parsing a JSON document or attribute.20 MBtruefalsefalseNONEfalsefalseel-rest-client-serviceClient ServiceAn Elasticsearch client service to use for running queries.org.apache.nifi.elasticsearch.ElasticSearchClientServiceorg.apache.nifinifi-elasticsearch-client-service-api-nar2.2.0truefalsefalseNONEfalsefalseel-rest-split-up-hitsSearch Results SplitOutput a flowfile containing all hits or one flowfile for each individual hit.splitUp-noPER_HITsplitUp-yesFlowfile per hit.PER_RESPONSEsplitUp-noFlowfile per response.truefalsefalseNONEfalsefalseel-rest-format-hitsSearch Results FormatFormat of Hits output.FULLFULLFULLContains full Elasticsearch Hit, including Document Source and Metadata.SOURCE_ONLYSOURCE_ONLYDocument Source only (where present).METADATA_ONLYMETADATA_ONLYHit Metadata only.truefalsefalseNONEfalsefalseel-rest-split-up-aggregationsAggregation Results SplitOutput a flowfile containing all aggregations or one flowfile for each individual aggregation.splitUp-noPER_HITsplitUp-yesFlowfile per hit.PER_RESPONSEsplitUp-noFlowfile per response.truefalsefalseNONEfalsefalseel-rest-format-aggregationsAggregation Results FormatFormat of Aggregation output.FULLFULLFULLContains full Elasticsearch Aggregation, including Buckets and Metadata.BUCKETS_ONLYBUCKETS_ONLYBucket Content only.METADATA_ONLYMETADATA_ONLYAggregation Metadata only.truefalsefalseNONEfalsefalseel-rest-output-no-hitsOutput No HitsOutput a "hits" flowfile even if no hits found for query. If true, an empty "hits" flowfile will be output even if "aggregations" are output.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseThe name of a URL query parameter to addThe value of the URL query parameterAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing. These parameters will override any matching parameters in the query request bodyFLOWFILE_ATTRIBUTEShitsSearch hits are routed to this relationship.falseaggregationsAggregations are routed to this relationship.falsefailureAll flowfiles that fail for reasons unrelated to server availability go to this relationship.falseoriginalAll original flowfiles that don't cause an error to occur go to this relationship.falsemime.typeapplication/jsonaggregation.nameThe name of the aggregation whose results are in the output flowfileaggregation.numberThe number of the aggregation whose results are in the output flowfilehit.countThe number of hits that are in the output flowfileelasticsearch.query.errorThe error message provided by Elasticsearch if there is an error querying the index.INPUT_ALLOWEDorg.apache.nifi.processors.elasticsearch.PaginatedJsonQueryElasticsearchorg.apache.nifi.processors.elasticsearch.PaginatedJsonQueryElasticsearchPROCESSORA processor that allows the user to run a paginated query (with aggregations) written with the Elasticsearch JSON DSL. It will use the flowfile's content for the query unless the QUERY attribute is populated. Search After/Point in Time queries must include a valid "sort" field.elasticsearchelasticsearch5elasticsearch6elasticsearch7elasticsearch8queryscrollpagereadjsonel-rest-query-definition-styleQuery Definition StyleHow the JSON Query will be defined for use by the processor.fullFULL_QUERYfullProvide the full Query.BUILD_QUERYbuildBuild the Query from separate JSON objects.truefalsefalseNONEfalsefalseel-rest-queryQueryA query in JSON syntax, not Lucene syntax. Ex: {"query":{"match":{"somefield":"somevalue"}}}. If this parameter is not set, the query will be read from the flowfile content. If the query (property and flowfile content) is empty, a default empty JSON Object will be used, which will result in a "match_all" query in Elasticsearch.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylefullel-rest-query-clauseQuery ClauseA "query" clause in JSON syntax, not Lucene syntax. Ex: {"match":{"somefield":"somevalue"}}. If the query is empty, a default JSON Object will be used, which will result in a "match_all" query in Elasticsearch.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-sizeSizeThe maximum number of documents to retrieve in the query. If the query is paginated, this "size" applies to each page of the query, not the "size" of the entire result set.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-query-sortSortSort results by one or more fields, in JSON syntax. Ex: [{"price" : {"order" : "asc", "mode" : "avg"}}, {"post_date" : {"format": "strict_date_optional_time_nanos"}}]falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-query-aggsAggregationsOne or more query aggregations (or "aggs"), in JSON syntax. Ex: {"items": {"terms": {"field": "product", "size": 10}}}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-query-fieldsFieldsFields of indexed documents to be retrieved, in JSON syntax. Ex: ["user.id", "http.response.*", {"field": "@timestamp", "format": "epoch_millis"}]falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-query-script-fieldsScript FieldsFields to created using script evaluation at query runtime, in JSON syntax. Ex: {"test1": {"script": {"lang": "painless", "source": "doc['price'].value * 2"}}, "test2": {"script": {"lang": "painless", "source": "doc['price'].value * params.factor", "params": {"factor": 2.0}}}}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildel-query-attributeQuery AttributeIf set, the executed query will be set on each result flowfile in the specified attribute.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-fetch-indexIndexThe name of the index to use.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-typeTypeThe type of this document (used by Elasticsearch for indexing and searching).falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax JSON Field String LengthMax JSON Field String LengthThe maximum allowed length of a string value when parsing a JSON document or attribute.20 MBtruefalsefalseNONEfalsefalseel-rest-client-serviceClient ServiceAn Elasticsearch client service to use for running queries.org.apache.nifi.elasticsearch.ElasticSearchClientServiceorg.apache.nifinifi-elasticsearch-client-service-api-nar2.2.0truefalsefalseNONEfalsefalseel-rest-split-up-hitsSearch Results SplitOutput a flowfile containing all hits or one flowfile for each individual hit or one flowfile containing all hits from all paged responses.splitUp-noPER_HITsplitUp-yesFlowfile per hit.PER_RESPONSEsplitUp-noFlowfile per response.PER_QUERYsplitUp-queryCombine results from all query responses (one flowfile per entire paginated result set of hits). Note that aggregations cannot be paged, they are generated across the entire result set and returned as part of the first page. Results are output with one JSON object per line (allowing hits to be combined from multiple pages without loading all results into memory).truefalsefalseNONEfalsefalseel-rest-format-hitsSearch Results FormatFormat of Hits output.FULLFULLFULLContains full Elasticsearch Hit, including Document Source and Metadata.SOURCE_ONLYSOURCE_ONLYDocument Source only (where present).METADATA_ONLYMETADATA_ONLYHit Metadata only.truefalsefalseNONEfalsefalseel-rest-split-up-aggregationsAggregation Results SplitOutput a flowfile containing all aggregations or one flowfile for each individual aggregation.splitUp-noPER_HITsplitUp-yesFlowfile per hit.PER_RESPONSEsplitUp-noFlowfile per response.truefalsefalseNONEfalsefalseel-rest-format-aggregationsAggregation Results FormatFormat of Aggregation output.FULLFULLFULLContains full Elasticsearch Aggregation, including Buckets and Metadata.BUCKETS_ONLYBUCKETS_ONLYBucket Content only.METADATA_ONLYMETADATA_ONLYAggregation Metadata only.truefalsefalseNONEfalsefalseel-rest-output-no-hitsOutput No HitsOutput a "hits" flowfile even if no hits found for query. If true, an empty "hits" flowfile will be output even if "aggregations" are output.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseel-rest-pagination-typePagination TypePagination method to use. Not all types are available for all Elasticsearch versions, check the Elasticsearch docs to confirm which are applicable and recommended for your service.pagination-scrollSCROLLpagination-scrollUse Elasticsearch "_scroll" API to page results. Does not accept additional query parameters.SEARCH_AFTERpagination-search_afterUse Elasticsearch "search_after" _search API to page sorted results.POINT_IN_TIMEpagination-pitUse Elasticsearch (7.10+ with XPack) "point in time" _search API to page sorted results. Not available for use with AWS OpenSearch.truefalsefalseNONEfalsefalseel-rest-pagination-keep-alivePagination Keep AlivePagination "keep_alive" period. Period Elasticsearch will keep the scroll/pit cursor alive in between requests (this is not the time expected for all pages to be returned, but the maximum allowed time for requests between page retrievals).10 minstruefalsefalseNONEfalsefalseThe name of a URL query parameter to addThe value of the URL query parameterAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing. These parameters will override any matching parameters in the query request body. For SCROLL type queries, these parameters are only used in the initial (first page) query as the Elasticsearch Scroll API does not support the same query parameters for subsequent pages of data.FLOWFILE_ATTRIBUTEShitsSearch hits are routed to this relationship.falseaggregationsAggregations are routed to this relationship.falsefailureAll flowfiles that fail for reasons unrelated to server availability go to this relationship.falseoriginalAll original flowfiles that don't cause an error to occur go to this relationship.falsemime.typeapplication/jsonaggregation.nameThe name of the aggregation whose results are in the output flowfileaggregation.numberThe number of the aggregation whose results are in the output flowfilepage.numberThe number of the page (request), starting from 1, in which the results were returned that are in the output flowfilehit.countThe number of hits that are in the output flowfileelasticsearch.query.errorThe error message provided by Elasticsearch if there is an error querying the index.INPUT_REQUIREDMEMORYCare should be taken on the size of each page because each response from Elasticsearch will be loaded into memory all at once and converted into the resulting flowfiles.org.apache.nifi.processors.elasticsearch.JsonQueryElasticsearchorg.apache.nifi.processors.elasticsearch.ConsumeElasticsearchorg.apache.nifi.processors.elasticsearch.SearchElasticsearchorg.apache.nifi.processors.elasticsearch.PutElasticsearchJsonPROCESSORAn Elasticsearch put processor that uses the official Elastic REST client libraries. Each FlowFile is treated as a document to be sent to the Elasticsearch _bulk API. Multiple FlowFiles can be batched together into each Request sent to Elasticsearch.jsonelasticsearchelasticsearch5elasticsearch6elasticsearch7elasticsearch8putindexput-es-json-id-attrIdentifier AttributeThe name of the FlowFile attribute containing the identifier for the document. If the Index Operation is "index", this property may be left empty or evaluate to an empty value, in which case the document's identifier will be auto-generated by Elasticsearch. For all other Index Operations, the attribute must evaluate to a non-empty value.falsefalsefalseNONEfalsefalseput-es-record-index-opIndex OperationThe type of the operation used to index (create, delete, index, update, upsert)indextruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-fetch-indexIndexThe name of the index to use.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-typeTypeThe type of this document (used by Elasticsearch for indexing and searching).falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-json-scriptScriptThe script for the document update/upsert. Only applies to Update/Upsert operations. Must be parsable as JSON Object. If left blank, the FlowFile content will be used for document update/upsertfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-json-scripted-upsertScripted UpsertWhether to add the scripted_upsert flag to the Upsert Operation. Forces Elasticsearch to execute the Script whether or not the document exists, defaults to false. If the Upsert Document provided (from FlowFile content) will be empty, but sure to set the Client Service controller service's Suppress Null/Empty Values to Never Suppress or no "upsert" doc will be, included in the request to Elasticsearch and the operation will not create a new document for the script to execute against, resulting in a "not_found" errorfalsetruetruefalsefalsefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-json-dynamic_templatesDynamic TemplatesThe dynamic_templates for the document. Must be parsable as a JSON Object. Requires Elasticsearch 7+falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-record-batch-sizeBatch SizeThe preferred number of FlowFiles to send over in a single batch100truefalsetrueENVIRONMENTfalsefalseput-es-json-charsetCharacter SetSpecifies the character set of the document data.UTF-8truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax JSON Field String LengthMax JSON Field String LengthThe maximum allowed length of a string value when parsing a JSON document or attribute.20 MBtruefalsefalseNONEfalsefalseel-rest-client-serviceClient ServiceAn Elasticsearch client service to use for running queries.org.apache.nifi.elasticsearch.ElasticSearchClientServiceorg.apache.nifinifi-elasticsearch-client-service-api-nar2.2.0truefalsefalseNONEfalsefalseput-es-record-log-error-responsesLog Error ResponsesIf this is enabled, errors will be logged to the NiFi logs at the error log level. Otherwise, they will only be logged if debug logging is enabled on NiFi as a whole. The purpose of this option is to give the user the ability to debug failed operations without having to turn on debug logging.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-es-output-error-responsesOutput Error ResponsesIf this is enabled, response messages from Elasticsearch marked as "error" will be output to the "error_responses" relationship.This does not impact the output of flowfiles to the "successful" or "errors" relationshipsfalsetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-es-not_found-is-errorTreat "Not Found" as SuccessIf true, "not_found" Elasticsearch Document associated Records will be routed to the "successful" relationship, otherwise to the "errors" relationship. If Output Error Responses is "true" then "not_found" responses from Elasticsearch will be sent to the error_responses relationship.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseThe name of the Bulk request headerThe value of the Bulk request headerPrefix: BULK: - adds the specified property name/value as a Bulk request header in the Elasticsearch Bulk API body used for processing. If the value is null or blank, the Bulk header will be omitted for the document operation. These parameters will override any matching parameters in the _bulk request body.FLOWFILE_ATTRIBUTESThe name of a URL query parameter to addThe value of the URL query parameterAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing. These parameters will override any matching parameters in the _bulk request body. If FlowFiles are batched, only the first FlowFile in the batch is used to evaluate property values.FLOWFILE_ATTRIBUTESretryAll flowfiles that fail due to server/cluster availability go to this relationship.falseerrorsRecord(s)/Flowfile(s) corresponding to Elasticsearch document(s) that resulted in an "error" (within Elasticsearch) will be routed here.falsesuccessfulRecord(s)/Flowfile(s) corresponding to Elasticsearch document(s) that did not result in an "error" (within Elasticsearch) will be routed here.falseoriginalAll flowfiles that are sent to Elasticsearch without request failures go to this relationship.falsefailureAll flowfiles that fail for reasons unrelated to server availability go to this relationship.falseelasticsearch.put.errorThe error message if there is an issue parsing the FlowFile, sending the parsed document to Elasticsearch or parsing the Elasticsearch responseelasticsearch.bulk.errorThe _bulk response if there was an error during processing the document within Elasticsearch.INPUT_REQUIREDMEMORYThe Batch of FlowFiles will be stored in memory until the bulk operation is performed.org.apache.nifi.processors.elasticsearch.PutElasticsearchRecordorg.apache.nifi.processors.elasticsearch.PutElasticsearchRecordPROCESSORA record-aware Elasticsearch put processor that uses the official Elastic REST client libraries. Each Record within the FlowFile is converted into a document to be sent to the Elasticsearch _bulk APi. Multiple documents can be batched into each Request sent to Elasticsearch. Each document's Bulk operation can be configured using Record Path expressions.jsonelasticsearchelasticsearch5elasticsearch6elasticsearch7elasticsearch8putindexrecordput-es-record-index-opIndex OperationThe type of the operation used to index (create, delete, index, update, upsert)indextruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-fetch-indexIndexThe name of the index to use.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-typeTypeThe type of this document (used by Elasticsearch for indexing and searching).falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-record-at-timestamp@timestamp ValueThe value to use as the @timestamp field (required for Elasticsearch Data Streams)falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax JSON Field String LengthMax JSON Field String LengthThe maximum allowed length of a string value when parsing a JSON document or attribute.20 MBtruefalsefalseNONEfalsefalseel-rest-client-serviceClient ServiceAn Elasticsearch client service to use for running queries.org.apache.nifi.elasticsearch.ElasticSearchClientServiceorg.apache.nifinifi-elasticsearch-client-service-api-nar2.2.0truefalsefalseNONEfalsefalseput-es-record-readerRecord ReaderThe record reader to use for reading incoming records from flowfiles.org.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.2.0truefalsefalseNONEfalsefalseput-es-record-batch-sizeBatch SizeThe number of records to send over in a single batch.100truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-record-id-pathID Record PathA record path expression to retrieve the ID field for use with Elasticsearch. If left blank the ID will be automatically generated by Elasticsearch.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-record-retain-id-fieldRetain ID (Record Path)Whether to retain the existing field used as the ID Record Path.falsetruetruefalsefalsefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-record-id-pathID Record Pathput-es-record-index-op-pathIndex Operation Record PathA record path expression to retrieve the Index Operation field for use with Elasticsearch. If left blank the Index Operation will be determined using the main Index Operation property.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-record-index-record-pathIndex Record PathA record path expression to retrieve the index field for use with Elasticsearch. If left blank the index will be determined using the main index property.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-record-type-record-pathType Record PathA record path expression to retrieve the type field for use with Elasticsearch. If left blank the type will be determined using the main type property.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-record-at-timestamp-path@timestamp Record PathA RecordPath pointing to a field in the record(s) that contains the @timestamp for the document. If left blank the @timestamp will be determined using the main @timestamp propertyfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-record-retain-at-timestamp-fieldRetain @timestamp (Record Path)Whether to retain the existing field used as the @timestamp Record Path.falsetruetruefalsefalsefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-record-at-timestamp-path@timestamp Record Pathput-es-record-script-pathScript Record PathA RecordPath pointing to a field in the record(s) that contains the script for the document update/upsert. Only applies to Update/Upsert operations. Field must be Map-type compatible (e.g. a Map or a Record) or a String parsable into a JSON ObjectfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-record-scripted-upsert-pathScripted Upsert Record PathA RecordPath pointing to a field in the record(s) that contains the scripted_upsert boolean flag. Whether to add the scripted_upsert flag to the Upsert Operation. Forces Elasticsearch to execute the Script whether or not the document exists, defaults to false. If the Upsert Document provided (from FlowFile content) will be empty, but sure to set the Client Service controller service's Suppress Null/Empty Values to Never Suppress or no "upsert" doc will be, included in the request to Elasticsearch and the operation will not create a new document for the script to execute against, resulting in a "not_found" errorfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-record-dynamic-templates-pathDynamic Templates Record PathA RecordPath pointing to a field in the record(s) that contains the dynamic_templates for the document. Field must be Map-type compatible (e.g. a Map or Record) or a String parsable into a JSON Object. Requires Elasticsearch 7+falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-es-record-at-timestamp-date-formatDate FormatSpecifies the format to use when writing Date fields. If not specified, the default format 'yyyy-MM-dd' is used. If specified, the value must match the Java Simple Date Format (for example, MM/dd/yyyy for a two-digit month, followed by a two-digit day, followed by a four-digit year, all separated by '/' characters, as in 01/25/2017).falsefalsetrueENVIRONMENTfalsefalseput-es-record-at-timestamp-time-formatTime FormatSpecifies the format to use when writing Time fields. If not specified, the default format 'HH:mm:ss' is used. If specified, the value must match the Java Simple Date Format (for example, HH:mm:ss for a two-digit hour in 24-hour format, followed by a two-digit minute, followed by a two-digit second, all separated by ':' characters, as in 18:04:15).falsefalsetrueENVIRONMENTfalsefalseput-es-record-at-timestamp-timestamp-formatTimestamp FormatSpecifies the format to use when writing Timestamp fields. If not specified, the default format 'yyyy-MM-dd HH:mm:ss' is used. If specified, the value must match the Java Simple Date Format (for example, MM/dd/yyyy HH:mm:ss for a two-digit month, followed by a two-digit day, followed by a four-digit year, all separated by '/' characters; and then followed by a two-digit hour in 24-hour format, followed by a two-digit minute, followed by a two-digit second, all separated by ':' characters, as in 01/25/2017 18:04:15).falsefalsetrueENVIRONMENTfalsefalseput-es-record-log-error-responsesLog Error ResponsesIf this is enabled, errors will be logged to the NiFi logs at the error log level. Otherwise, they will only be logged if debug logging is enabled on NiFi as a whole. The purpose of this option is to give the user the ability to debug failed operations without having to turn on debug logging.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-es-output-error-responsesOutput Error ResponsesIf this is enabled, response messages from Elasticsearch marked as "error" will be output to the "error_responses" relationship.This does not impact the output of flowfiles to the "successful" or "errors" relationshipsfalsetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-es-record-error-writerResult Record WriterThe response from Elasticsearch will be examined for failed records and the failed records will be written to a record set with this record writer service and sent to the "errors" relationship. Successful records will be written to a record set with this record writer service and sent to the "successful" relationship.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.2.0truefalsefalseNONEfalsefalseput-es-not_found-is-errorTreat "Not Found" as SuccessIf true, "not_found" Elasticsearch Document associated Records will be routed to the "successful" relationship, otherwise to the "errors" relationship. If Output Error Responses is "true" then "not_found" responses from Elasticsearch will be sent to the error_responses relationship.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-es-record-bulk-error-groupsGroup Results by Bulk Error TypeThe errored records written to the "errors" relationship will be grouped by error type and the error related to the first record within the FlowFile added to the FlowFile as "elasticsearch.bulk.error". If "Treat "Not Found" as Success" is "false" then records associated with "not_found" Elasticsearch document responses will also be send to the "errors" relationship.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-es-record-error-writerResult Record WriterThe name of the Bulk request headerA Record Path expression to retrieve the Bulk request header valuePrefix: BULK: - adds the specified property name/value as a Bulk request header in the Elasticsearch Bulk API body used for processing. If the Record Path expression results in a null or blank value, the Bulk header will be omitted for the document operation. These parameters will override any matching parameters in the _bulk request body.FLOWFILE_ATTRIBUTESThe name of a URL query parameter to addThe value of the URL query parameterAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing. These parameters will override any matching parameters in the _bulk request bodyFLOWFILE_ATTRIBUTESretryAll flowfiles that fail due to server/cluster availability go to this relationship.falsesuccessfulRecord(s)/Flowfile(s) corresponding to Elasticsearch document(s) that did not result in an "error" (within Elasticsearch) will be routed here.falseerrorsRecord(s)/Flowfile(s) corresponding to Elasticsearch document(s) that resulted in an "error" (within Elasticsearch) will be routed here.falseoriginalAll flowfiles that are sent to Elasticsearch without request failures go to this relationship.falsefailureAll flowfiles that fail for reasons unrelated to server availability go to this relationship.falseelasticsearch.put.errorThe error message if there is an issue parsing the FlowFile records, sending the parsed documents to Elasticsearch or parsing the Elasticsearch response.elasticsearch.put.error.countThe number of records that generated errors in the Elasticsearch _bulk API.elasticsearch.put.success.countThe number of records that were successfully processed by the Elasticsearch _bulk API.elasticsearch.bulk.errorThe _bulk response if there was an error during processing the record within Elasticsearch.INPUT_REQUIREDMEMORYThe Batch of Records will be stored in memory until the bulk operation is performed.org.apache.nifi.processors.elasticsearch.PutElasticsearchJsonorg.apache.nifi.processors.elasticsearch.SearchElasticsearchPROCESSORA processor that allows the user to repeatedly run a paginated query (with aggregations) written with the Elasticsearch JSON DSL. Search After/Point in Time queries must include a valid "sort" field. The processor will retrieve multiple pages of results until either no more results are available or the Pagination Keep Alive expiration is reached, after which the query will restart with the first page of results being retrieved.elasticsearchelasticsearch5elasticsearch6elasticsearch7elasticsearch8queryscrollpagesearchjsonel-rest-query-definition-styleQuery Definition StyleHow the JSON Query will be defined for use by the processor.fullFULL_QUERYfullProvide the full Query.BUILD_QUERYbuildBuild the Query from separate JSON objects.truefalsefalseNONEfalsefalseel-rest-queryQueryA query in JSON syntax, not Lucene syntax. Ex: {"query":{"match":{"somefield":"somevalue"}}}. If the query is empty, a default JSON Object will be used, which will result in a "match_all" query in Elasticsearch.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylefullel-rest-query-clauseQuery ClauseA "query" clause in JSON syntax, not Lucene syntax. Ex: {"match":{"somefield":"somevalue"}}. If the query is empty, a default JSON Object will be used, which will result in a "match_all" query in Elasticsearch.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-sizeSizeThe maximum number of documents to retrieve in the query. If the query is paginated, this "size" applies to each page of the query, not the "size" of the entire result set.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-query-sortSortSort results by one or more fields, in JSON syntax. Ex: [{"price" : {"order" : "asc", "mode" : "avg"}}, {"post_date" : {"format": "strict_date_optional_time_nanos"}}]falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-query-aggsAggregationsOne or more query aggregations (or "aggs"), in JSON syntax. Ex: {"items": {"terms": {"field": "product", "size": 10}}}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-query-fieldsFieldsFields of indexed documents to be retrieved, in JSON syntax. Ex: ["user.id", "http.response.*", {"field": "@timestamp", "format": "epoch_millis"}]falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildes-rest-query-script-fieldsScript FieldsFields to created using script evaluation at query runtime, in JSON syntax. Ex: {"test1": {"script": {"lang": "painless", "source": "doc['price'].value * 2"}}, "test2": {"script": {"lang": "painless", "source": "doc['price'].value * params.factor", "params": {"factor": 2.0}}}}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildel-query-attributeQuery AttributeIf set, the executed query will be set on each result flowfile in the specified attribute.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-fetch-indexIndexThe name of the index to use.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-typeTypeThe type of this document (used by Elasticsearch for indexing and searching).falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax JSON Field String LengthMax JSON Field String LengthThe maximum allowed length of a string value when parsing a JSON document or attribute.20 MBtruefalsefalseNONEfalsefalseel-rest-client-serviceClient ServiceAn Elasticsearch client service to use for running queries.org.apache.nifi.elasticsearch.ElasticSearchClientServiceorg.apache.nifinifi-elasticsearch-client-service-api-nar2.2.0truefalsefalseNONEfalsefalseel-rest-split-up-hitsSearch Results SplitOutput a flowfile containing all hits or one flowfile for each individual hit or one flowfile containing all hits from all paged responses.splitUp-noPER_HITsplitUp-yesFlowfile per hit.PER_RESPONSEsplitUp-noFlowfile per response.PER_QUERYsplitUp-queryCombine results from all query responses (one flowfile per entire paginated result set of hits). Note that aggregations cannot be paged, they are generated across the entire result set and returned as part of the first page. Results are output with one JSON object per line (allowing hits to be combined from multiple pages without loading all results into memory).truefalsefalseNONEfalsefalseel-rest-format-hitsSearch Results FormatFormat of Hits output.FULLFULLFULLContains full Elasticsearch Hit, including Document Source and Metadata.SOURCE_ONLYSOURCE_ONLYDocument Source only (where present).METADATA_ONLYMETADATA_ONLYHit Metadata only.truefalsefalseNONEfalsefalseel-rest-split-up-aggregationsAggregation Results SplitOutput a flowfile containing all aggregations or one flowfile for each individual aggregation.splitUp-noPER_HITsplitUp-yesFlowfile per hit.PER_RESPONSEsplitUp-noFlowfile per response.truefalsefalseNONEfalsefalseel-rest-format-aggregationsAggregation Results FormatFormat of Aggregation output.FULLFULLFULLContains full Elasticsearch Aggregation, including Buckets and Metadata.BUCKETS_ONLYBUCKETS_ONLYBucket Content only.METADATA_ONLYMETADATA_ONLYAggregation Metadata only.truefalsefalseNONEfalsefalseel-rest-output-no-hitsOutput No HitsOutput a "hits" flowfile even if no hits found for query. If true, an empty "hits" flowfile will be output even if "aggregations" are output.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseel-rest-pagination-typePagination TypePagination method to use. Not all types are available for all Elasticsearch versions, check the Elasticsearch docs to confirm which are applicable and recommended for your service.pagination-scrollSCROLLpagination-scrollUse Elasticsearch "_scroll" API to page results. Does not accept additional query parameters.SEARCH_AFTERpagination-search_afterUse Elasticsearch "search_after" _search API to page sorted results.POINT_IN_TIMEpagination-pitUse Elasticsearch (7.10+ with XPack) "point in time" _search API to page sorted results. Not available for use with AWS OpenSearch.truefalsefalseNONEfalsefalseel-rest-pagination-keep-alivePagination Keep AlivePagination "keep_alive" period. Period Elasticsearch will keep the scroll/pit cursor alive in between requests (this is not the time expected for all pages to be returned, but the maximum allowed time for requests between page retrievals).10 minstruefalsefalseNONEfalsefalseThe name of a URL query parameter to addThe value of the URL query parameterAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing. These parameters will override any matching parameters in the query request body. For SCROLL type queries, these parameters are only used in the initial (first page) query as the Elasticsearch Scroll API does not support the same query parameters for subsequent pages of data.ENVIRONMENThitsSearch hits are routed to this relationship.falseaggregationsAggregations are routed to this relationship.falsemime.typeapplication/jsonaggregation.nameThe name of the aggregation whose results are in the output flowfileaggregation.numberThe number of the aggregation whose results are in the output flowfilepage.numberThe number of the page (request), starting from 1, in which the results were returned that are in the output flowfilehit.countThe number of hits that are in the output flowfileelasticsearch.query.errorThe error message provided by Elasticsearch if there is an error querying the index.truetrueThe pagination state (scrollId, searchAfter, pitId, hitCount, pageCount, pageExpirationTimestamp) is retained in between invocations of this processor until the Scroll/PiT has expired (when the current time is later than the last query execution plus the Pagination Keep Alive interval).LOCALINPUT_FORBIDDENMEMORYCare should be taken on the size of each page because each response from Elasticsearch will be loaded into memory all at once and converted into the resulting flowfiles.org.apache.nifi.processors.elasticsearch.PaginatedJsonQueryElasticsearchorg.apache.nifi.processors.elasticsearch.ConsumeElasticsearchTIMER_DRIVEN1 min1org.apache.nifi.processors.elasticsearch.UpdateByQueryElasticsearchPROCESSORUpdate documents in an Elasticsearch index using a query. The query can be loaded from a flowfile body or from the Query parameter. The loaded Query can contain any JSON accepted by Elasticsearch's _update_by_query API, for example a "query" object to identify what documents are to be updated, plus a "script" to define the updates to perform.elasticelasticsearchelasticsearch5elasticsearch6elasticsearch7elasticsearch8updatequeryel-rest-query-definition-styleQuery Definition StyleHow the JSON Query will be defined for use by the processor.fullFULL_QUERYfullProvide the full Query.BUILD_QUERYbuildBuild the Query from separate JSON objects.truefalsefalseNONEfalsefalseel-rest-queryQueryA query in JSON syntax, not Lucene syntax. Ex: {"query":{"match":{"somefield":"somevalue"}}}. If this parameter is not set, the query will be read from the flowfile content. If the query (property and flowfile content) is empty, a default empty JSON Object will be used, which will result in a "match_all" query in Elasticsearch.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylefullel-rest-query-clauseQuery ClauseA "query" clause in JSON syntax, not Lucene syntax. Ex: {"match":{"somefield":"somevalue"}}. If the query is empty, a default JSON Object will be used, which will result in a "match_all" query in Elasticsearch.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildel-rest-scriptScriptA "script" to execute during the operation, in JSON syntax. Ex: {"source": "ctx._source.count++", "lang": "painless"}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-query-definition-styleQuery Definition Stylebuildel-query-attributeQuery AttributeIf set, the executed query will be set on each result flowfile in the specified attribute.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-fetch-indexIndexThe name of the index to use.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseel-rest-typeTypeThe type of this document (used by Elasticsearch for indexing and searching).falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax JSON Field String LengthMax JSON Field String LengthThe maximum allowed length of a string value when parsing a JSON document or attribute.20 MBtruefalsefalseNONEfalsefalseel-rest-client-serviceClient ServiceAn Elasticsearch client service to use for running queries.org.apache.nifi.elasticsearch.ElasticSearchClientServiceorg.apache.nifinifi-elasticsearch-client-service-api-nar2.2.0truefalsefalseNONEfalsefalseThe name of a URL query parameter to addThe value of the URL query parameterAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing. These parameters will override any matching parameters in the query request bodyFLOWFILE_ATTRIBUTESfailureIf the "by query" operation fails, and a flowfile was read, it will be sent to this relationship.falsesuccessIf the "by query" operation succeeds, and a flowfile was read, it will be sent to this relationship.falseretryAll flowfiles that fail due to server/cluster availability go to this relationship.falseelasticsearch.update.tookThe amount of time that it took to complete the update operation in ms.elasticsearch.update.errorThe error message provided by Elasticsearch if there is an error running the update.INPUT_ALLOWED