org.apache.nifinifi-azure-nar2.2.0org.apache.nifinifi-azure-services-api-nar2.2.02.0.0rel/nifi-2.2.0main1org.apache.nifi.processors.azure.cosmos.document.PutAzureCosmosDBRecordPROCESSORThis processor is a record-aware processor for inserting data into Cosmos DB with Core SQL API. It uses a configured record reader and schema to read an incoming record set from the body of a Flowfile and then inserts those records into a configured Cosmos DB Container.azurecosmosinsertrecordputazure-cosmos-db-connection-serviceCosmos DB Connection ServiceIf configured, the controller service used to obtain the connection string and access keyorg.apache.nifi.services.azure.cosmos.AzureCosmosDBConnectionServiceorg.apache.nifinifi-azure-services-api-nar2.2.0falsefalsefalseNONEfalsefalseazure-cosmos-db-uriCosmos DB URICosmos DB URI, typically in the form of https://{databaseaccount}.documents.azure.com:443/ Note this host URL is for Cosmos DB with Core SQL API from Azure Portal (Overview->URI)falsetruefalseNONEfalsefalseazure-cosmos-db-keyCosmos DB Access KeyCosmos DB Access Key from Azure Portal (Settings->Keys). Choose a read-write key to enable database or container creation at run timefalsetruefalseNONEfalsefalseazure-cosmos-db-consistency-levelCosmos DB Consistency LevelChoose from five consistency levels on the consistency spectrum. Refer to Cosmos DB documentation for their differencesSESSIONSTRONGSTRONGBOUNDED_STALENESSBOUNDED_STALENESSSESSIONSESSIONCONSISTENT_PREFIXCONSISTENT_PREFIXEVENTUALEVENTUALfalsefalsefalseNONEfalsefalseazure-cosmos-db-nameCosmos DB NameThe database name or id. This is used as the namespace for document collections or containerstruefalsefalseNONEfalsefalseazure-cosmos-db-container-idCosmos DB Container IDThe unique identifier for the containertruefalsefalseNONEfalsefalseazure-cosmos-db-partition-keyCosmos DB Partition KeyThe partition key used to distribute data among serverstruefalsefalseNONEfalsefalserecord-readerRecord ReaderSpecifies the Controller Service to use for parsing incoming data and determining the data's schemaorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.2.0truefalsefalseNONEfalsefalseinsert-batch-sizeInsert Batch SizeThe number of records to group together for one single insert operation against Cosmos DB20falsefalsefalseNONEfalsefalseazure-cosmos-db-conflict-handling-strategyCosmos DB Conflict Handling StrategyChoose whether to ignore or upsert when conflict error occurs during insertionIGNOREIgnoreIGNOREConflicting records will not be inserted, and FlowFile will not be routed to failureUpsertUPSERTConflicting records will be upserted, and FlowFile will not be routed to failurefalsefalsefalseNONEfalsefalsesuccessAll FlowFiles that are written to Cosmos DB are routed to this relationshipfalsefailureAll FlowFiles that cannot be written to Cosmos DB are routed to this relationshipfalseINPUT_REQUIREDMEMORYAn instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.org.apache.nifi.processors.azure.data.explorer.PutAzureDataExplorerPROCESSORActs as an Azure Data Explorer sink which sends FlowFiles to the provided endpoint. Data can be sent through queued ingestion or streaming ingestion to the Azure Data Explorer cluster.AzureKustoADXExplorerDataKusto Ingest ServiceKusto Ingest ServiceAzure Data Explorer Kusto Ingest Serviceorg.apache.nifi.services.azure.data.explorer.KustoIngestServiceorg.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalseDatabase NameDatabase NameAzure Data Explorer Database Name for ingesting datatruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseTable NameTable NameAzure Data Explorer Table Name for ingesting datatruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseIngest Mapping NameIngest Mapping NameThe name of the mapping responsible for storing the data in the appropriate columns.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseData FormatData FormatThe format of the data that is sent to Azure Data Explorer. Supported formats include: avro, csv, jsonavroavroAn Avro format with support for logical types and for the snappy compression codecapacheavroapacheavroAn Avro format with support for logical types and for the snappy compression codec.csvcsvA text file with comma-separated values (,). For more information, see RFC 4180: Common Format and MIME Type for Comma-Separated Values (CSV) Files.jsonjsonA text file containing JSON objects separated by \n or \r\n. For more information, see JSON Lines (JSONL).multijsonmultijsonA text file containing a JSON array of property containers (each representing a record) or any number of property containers separated by spaces, \n or \r\n. Each property container may be spread across multiple lines. This format is preferable to JSON unless the data is not property containers.orcorcAn ORC file.parquetparquetA parquet file.psvpsvA text file with values separated by vertical bars (|).scsvscsvA text file with values separated by semicolons (;).sohsvsohsvA text file with SOH-separated values. (SOH is the ASCII code point 1. This format is used by Hive in HDInsight).tsvtsvA text file with tab delimited values (\t).tsvetsveA text file with tab-delimited values (\t). A backslash (\) is used as escape character.txttxtA text file with lines separated by \n. Empty lines are skippedtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePartially Succeeded Routing StrategyPartially Succeeded Routing StrategyDefines where to route FlowFiles that resulted in a partially succeeded status.failuresuccesssuccessfailurefailuretruefalsefalseNONEfalsefalseStreaming EnabledStreaming EnabledWhether to stream data to Azure Data Explorer.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseIngestion Ignore First RecordIngestion Ignore First RecordDefines whether ignore first record while ingestion.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsePoll for Ingest StatusPoll for Ingest StatusDetermines whether to poll on ingestion status after an ingestion to Azure Data Explorer is completedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseIngest Status Polling TimeoutIngest Status Polling TimeoutDefines the total amount time to poll for ingestion status5 mtruefalsefalseNONEfalsefalsePoll for Ingest StatusPoll for Ingest StatustrueIngest Status Polling IntervalIngest Status Polling IntervalDefines the value of interval of time to poll for ingestion status5 struefalsefalseNONEfalsefalsePoll for Ingest StatusPoll for Ingest StatustruesuccessIngest processing succeededfalsefailureIngest processing failedfalseINPUT_REQUIREDorg.apache.nifi.processors.azure.data.explorer.QueryAzureDataExplorerPROCESSORQuery Azure Data Explorer and stream JSON results to output FlowFilesAzureDataExplorerADXKustoKusto Query ServiceKusto Query ServiceAzure Data Explorer Kusto Query Serviceorg.apache.nifi.services.azure.data.explorer.KustoQueryServiceorg.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalseDatabase NameDatabase NameAzure Data Explorer Database Name for queryingtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseQueryQueryQuery to be run against Azure Data ExplorertruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessFlowFiles containing results of a successful QueryfalsefailureFlowFiles containing original input associated with a failed Queryfalsequery.error.messageAzure Data Explorer query error message on failuresquery.executedAzure Data Explorer query executedmime.typeContent Type set to application/jsonINPUT_REQUIREDorg.apache.nifi.processors.azure.eventhub.ConsumeAzureEventHubPROCESSORReceives messages from Microsoft Azure Event Hubs with checkpointing to ensure consistent event processing. Checkpoint tracking avoids consuming a message multiple times and enables reliable resumption of processing in the event of intermittent network failures. Checkpoint tracking requires external storage and provides the preferred approach to consuming messages from Azure Event Hubs. In clustered environment, ConsumeAzureEventHub processor instances form a consumer group and the messages are distributed among the cluster nodes (each message is processed on one cluster node only).azuremicrosoftcloudeventhubeventsstreamingstreamsevent-hub-namespaceEvent Hub NamespaceThe namespace that the Azure Event Hubs is assigned to. This is generally equal to <Event Hub Names>-ns.truefalsetrueENVIRONMENTfalsefalseevent-hub-nameEvent Hub NameThe name of the event hub to pull messages from.truefalsetrueENVIRONMENTfalsefalseService Bus EndpointService Bus EndpointTo support namespaces not in the default windows.net domain..servicebus.windows.netAzure.servicebus.windows.netServicebus endpoint for general useAzure China.servicebus.chinacloudapi.cnServicebus endpoint for ChinaAzure Germany.servicebus.cloudapi.deServicebus endpoint for GermanyAzure US Government.servicebus.usgovcloudapi.netServicebus endpoint for US GovernmenttruefalsefalseNONEfalsefalseTransport TypeTransport TypeAdvanced Message Queuing Protocol Transport Type for communication with Azure Event HubsAmqpAMQPAmqpAMQP over TCP on ports 5671 and 5672AMQP_WEB_SOCKETSAmqpWebSocketsAMQP over HTTPS with WebSockets on port 443truefalsefalseNONEfalsefalseevent-hub-shared-access-policy-nameShared Access Policy NameThe name of the shared access policy. This policy must have Listen claims.falsefalsetrueENVIRONMENTfalsefalseevent-hub-shared-access-policy-primary-keyShared Access Policy KeyThe key of the shared access policy. Either the primary or the secondary key can be used.falsetruefalseNONEfalsefalseuse-managed-identityUse Azure Managed IdentityChoose whether or not to use the managed identity of Azure VM/VMSSfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseevent-hub-consumer-groupConsumer GroupThe name of the consumer group to use.$DefaulttruefalsetrueENVIRONMENTfalsefalserecord-readerRecord ReaderThe Record Reader to use for reading received messages. The event hub name can be referred by Expression Language '${eventhub.name}' to access a schema.org.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalserecord-writerRecord WriterThe Record Writer to use for serializing Records to an output FlowFile. The event hub name can be referred by Expression Language '${eventhub.name}' to access a schema. If not specified, each message will create a FlowFile.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalseevent-hub-initial-offsetInitial OffsetSpecify where to start receiving messages if offset is not yet stored in the checkpoint store.end-of-streamStart of streamstart-of-streamRead from the oldest message retained in the stream.End of streamend-of-streamIgnore old retained messages even if exist, start reading new ones from now.truefalsefalseNONEfalsefalseevent-hub-prefetch-countPrefetch Count300truefalsetrueENVIRONMENTfalsefalseevent-hub-batch-sizeBatch SizeThe number of messages to process within a NiFi session. This parameter affects throughput and consistency. NiFi commits its session and Event Hubs checkpoints after processing this number of messages. If NiFi session is committed, but fails to create an Event Hubs checkpoint, then it is possible that the same messages will be received again. The higher number, the higher throughput, but possibly less consistent.10truefalsetrueENVIRONMENTfalsefalseevent-hub-message-receive-timeoutMessage Receive TimeoutThe amount of time this consumer should wait to receive the Batch Size before returning.1 mintruefalsetrueENVIRONMENTfalsefalsecheckpoint-strategyCheckpoint StrategySpecifies which strategy to use for storing and retrieving partition ownership and checkpoint information for each partition.AZURE_BLOB_STORAGEAzure Blob StorageAZURE_BLOB_STORAGEUse Azure Blob Storage to store partition ownership and checkpoint informationComponent StateCOMPONENT_STATEUse component state to store partition ownership and checkpoint informationtruefalsefalseNONEfalsefalsestorage-account-nameStorage Account NameName of the Azure Storage account to store event hub consumer group state.truefalsetrueENVIRONMENTfalsefalsecheckpoint-strategyCheckpoint StrategyAZURE_BLOB_STORAGEstorage-account-keyStorage Account KeyThe Azure Storage account key to store event hub consumer group state.falsetruetrueENVIRONMENTfalsefalsecheckpoint-strategyCheckpoint StrategyAZURE_BLOB_STORAGEstorage-sas-tokenStorage SAS TokenThe Azure Storage SAS token to store Event Hub consumer group state. Always starts with a ? character.falsetruetrueENVIRONMENTfalsefalsecheckpoint-strategyCheckpoint StrategyAZURE_BLOB_STORAGEstorage-container-nameStorage Container NameName of the Azure Storage container to store the event hub consumer group state. If not specified, event hub name is used.falsefalsetrueENVIRONMENTfalsefalsecheckpoint-strategyCheckpoint StrategyAZURE_BLOB_STORAGEproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalseTransport TypeTransport TypeAmqpWebSocketssuccessFlowFiles received from Event Hub.falseeventhub.enqueued.timestampThe time (in milliseconds since epoch, UTC) at which the message was enqueued in the event hubeventhub.offsetThe offset into the partition at which the message was storedeventhub.sequenceThe sequence number associated with the messageeventhub.nameThe name of the event hub from which the message was pulledeventhub.partitionThe name of the partition from which the message was pulledeventhub.property.*The application properties of this message. IE: 'application' would be 'eventhub.property.application'trueLocal state is used to store the client id. Cluster state is used to store partition ownership and checkpoint information when component state is configured as the checkpointing strategy.LOCALCLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.azure.eventhub.GetAzureEventHubPROCESSORReceives messages from Microsoft Azure Event Hubs without reliable checkpoint tracking. In clustered environment, GetAzureEventHub processor instances work independently and all cluster nodes process all messages (unless running the processor in Primary Only mode). ConsumeAzureEventHub offers the recommended approach to receiving messages from Azure Event Hubs. This processor creates a thread pool for connections to Azure Event Hubs.azuremicrosoftcloudeventhubeventsstreamingstreamsEvent Hub NamespaceEvent Hub NamespaceNamespace of Azure Event Hubs prefixed to Service Bus Endpoint domaintruefalsefalseNONEfalsefalseEvent Hub NameEvent Hub NameName of Azure Event Hubs sourcetruefalsefalseNONEfalsefalseService Bus EndpointService Bus EndpointTo support namespaces not in the default windows.net domain..servicebus.windows.netAzure.servicebus.windows.netServicebus endpoint for general useAzure China.servicebus.chinacloudapi.cnServicebus endpoint for ChinaAzure Germany.servicebus.cloudapi.deServicebus endpoint for GermanyAzure US Government.servicebus.usgovcloudapi.netServicebus endpoint for US GovernmenttruefalsefalseNONEfalsefalseTransport TypeTransport TypeAdvanced Message Queuing Protocol Transport Type for communication with Azure Event HubsAmqpAMQPAmqpAMQP over TCP on ports 5671 and 5672AMQP_WEB_SOCKETSAmqpWebSocketsAMQP over HTTPS with WebSockets on port 443truefalsefalseNONEfalsefalseShared Access Policy NameShared Access Policy NameThe name of the shared access policy. This policy must have Listen claims.falsefalsefalseNONEfalsefalseShared Access Policy Primary KeyShared Access Policy KeyThe key of the shared access policy. Either the primary or the secondary key can be used.falsetruefalseNONEfalsefalseuse-managed-identityUse Azure Managed IdentityChoose whether or not to use the managed identity of Azure VM/VMSSfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseEvent Hub Consumer GroupConsumer GroupThe name of the consumer group to use when pulling events$DefaulttruefalsefalseNONEfalsefalseEvent Hub Message Enqueue TimeMessage Enqueue TimeA timestamp (ISO-8601 Instant) formatted as YYYY-MM-DDThhmmss.sssZ (2016-01-01T01:01:01.000Z) from which messages should have been enqueued in the Event Hub to start reading fromfalsefalsefalseNONEfalsefalsePartition Recivier Fetch SizePartition Receiver Fetch SizeThe number of events that a receiver should fetch from an Event Hubs partition before returning. The default is 100falsefalsefalseNONEfalsefalsePartition Receiver Timeout (millseconds)Partition Receiver TimeoutThe amount of time in milliseconds a Partition Receiver should wait to receive the Fetch Size before returning. The default is 60000falsefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalseTransport TypeTransport TypeAmqpWebSocketssuccessAny FlowFile that is successfully received from the event hub will be transferred to this Relationship.falseeventhub.enqueued.timestampThe time (in milliseconds since epoch, UTC) at which the message was enqueued in the event hubeventhub.offsetThe offset into the partition at which the message was storedeventhub.sequenceThe Azure sequence number associated with the messageeventhub.nameThe name of the event hub from which the message was pulledeventhub.partitionThe name of the event hub partition from which the message was pulledeventhub.property.*The application properties of this message. IE: 'application' would be 'eventhub.property.application'INPUT_FORBIDDENorg.apache.nifi.processors.azure.eventhub.ConsumeAzureEventHuborg.apache.nifi.processors.azure.eventhub.PutAzureEventHubPROCESSORSend FlowFile contents to Azure Event HubsmicrosoftazurecloudeventhubeventsstreamsstreamingEvent Hub NamespaceEvent Hub NamespaceNamespace of Azure Event Hubs prefixed to Service Bus Endpoint domaintruefalsefalseNONEfalsefalseEvent Hub NameEvent Hub NameName of Azure Event Hubs destinationtruefalsefalseNONEfalsefalseService Bus EndpointService Bus EndpointTo support namespaces not in the default windows.net domain..servicebus.windows.netAzure.servicebus.windows.netServicebus endpoint for general useAzure China.servicebus.chinacloudapi.cnServicebus endpoint for ChinaAzure Germany.servicebus.cloudapi.deServicebus endpoint for GermanyAzure US Government.servicebus.usgovcloudapi.netServicebus endpoint for US GovernmenttruefalsefalseNONEfalsefalseTransport TypeTransport TypeAdvanced Message Queuing Protocol Transport Type for communication with Azure Event HubsAmqpAMQPAmqpAMQP over TCP on ports 5671 and 5672AMQP_WEB_SOCKETSAmqpWebSocketsAMQP over HTTPS with WebSockets on port 443truefalsefalseNONEfalsefalseShared Access Policy NameShared Access Policy NameThe name of the shared access policy. This policy must have Send claims.falsefalsefalseNONEfalsefalseShared Access Policy Primary KeyShared Access Policy KeyThe key of the shared access policy. Either the primary or the secondary key can be used.falsetruefalseNONEfalsefalseuse-managed-identityUse Azure Managed IdentityChoose whether or not to use the managed identity of Azure VM/VMSSfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsepartitioning-key-attribute-namePartitioning Key Attribute NameIf specified, the value from argument named by this field will be used as a partitioning key to be used by event hub.falsefalsefalseNONEfalsefalsemax-batch-sizeMaximum Batch SizeMaximum number of FlowFiles processed for each Processor invocation100truefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalseTransport TypeTransport TypeAmqpWebSocketssuccessAny FlowFile that is successfully sent to the event hubs will be transferred to this Relationship.falsefailureAny FlowFile that could not be sent to the event hub will be transferred to this Relationship.falsetrueINPUT_REQUIREDMEMORYThe Processor buffers FlowFile contents in memory before sendingorg.apache.nifi.processors.azure.storage.CopyAzureBlobStorage_v12PROCESSORCopies a blob in Azure Blob Storage from one account/container to another. The processor uses Azure Blob Storage client library v12.azuremicrosoftcloudstorageblobSource Storage CredentialsSource Storage CredentialsCredentials Service used to obtain Azure Blob Storage Credentials to read Source Blob informationorg.apache.nifi.services.azure.storage.AzureStorageCredentialsService_v12org.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalseSource Container NameSource Container NameName of the Azure storage container that will be copiedtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSource Blob NameSource Blob NameThe full name of the source blobtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsestorage-credentials-serviceDestination Storage CredentialsController Service used to obtain Azure Blob Storage Credentials.org.apache.nifi.services.azure.storage.AzureStorageCredentialsService_v12org.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalsecontainer-nameDestination Container NameName of the Azure storage container destination defaults to the Source Container Name when not specifiedfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseblob-nameDestination Blob NameThe full name of the destination blob defaults to the Source Blob Name when not specifiedfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseconflict-resolution-strategyConflict Resolution StrategySpecifies whether an existing blob will have its contents replaced upon conflict.FAIL_RESOLUTIONfailFAIL_RESOLUTIONFail if the blob already existsignoreIGNORE_RESOLUTIONIgnore if the blob already exists; the 'azure.error' attribute will be set to the value 'BLOB_ALREADY_EXISTS'replaceREPLACE_RESOLUTIONReplace blob contents if the blob already existtruefalsefalseNONEfalsefalsecreate-containerCreate ContainerSpecifies whether to check if the container exists and to automatically create it if it does not. Permission to list containers is required. If false, this check is not made, but the Put operation will fail if the container does not exist.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalsesuccessAll successfully processed FlowFiles are routed to this relationshipfalsefailureUnsuccessful operations will be transferred to the failure relationship.falseazure.containerThe name of the Azure Blob Storage containerazure.blobnameThe name of the blob on Azure Blob Storageazure.primaryUriPrimary location of the blobazure.etagETag of the blobazure.blobtypeType of the blob (either BlockBlob, PageBlob or AppendBlob)mime.typeMIME Type of the contentlangLanguage code for the contentazure.timestampTimestamp of the blobazure.lengthLength of the blobazure.error.codeError code reported during blob operationazure.ignoredWhen Conflict Resolution Strategy is 'ignore', this property will be true/false depending on whether the blob was ignored.INPUT_REQUIREDorg.apache.nifi.processors.azure.storage.ListAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.FetchAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.DeleteAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.PutAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.DeleteAzureBlobStorage_v12PROCESSORDeletes the specified blob from Azure Blob Storage. The processor uses Azure Blob Storage client library v12.azuremicrosoftcloudstorageblobstorage-credentials-serviceStorage CredentialsController Service used to obtain Azure Blob Storage Credentials.org.apache.nifi.services.azure.storage.AzureStorageCredentialsService_v12org.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalsecontainer-nameContainer NameName of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.${azure.container}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseblob-nameBlob NameThe full name of the blob${azure.blobname}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedelete-snapshots-optionDelete Snapshots OptionSpecifies the snapshot deletion options to be used when deleting a blob.NONENoneNONEDelete the blob only.Include SnapshotsINCLUDEDelete the blob and its snapshots.Delete Snapshots OnlyONLYDelete only the blob's snapshots.truefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalsesuccessAll successfully processed FlowFiles are routed to this relationshipfalsefailureUnsuccessful operations will be transferred to the failure relationship.falseINPUT_REQUIREDorg.apache.nifi.processors.azure.storage.ListAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.FetchAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.PutAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.CopyAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.DeleteAzureDataLakeStoragePROCESSORDeletes the provided file from Azure Data Lake Storageazuremicrosoftcloudstorageadlsgen2datalakeadls-credentials-serviceADLS CredentialsController Service used to obtain Azure Credentials.org.apache.nifi.services.azure.storage.ADLSCredentialsServiceorg.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalsefilesystem-nameFilesystem NameName of the Azure Storage File System (also called Container). It is assumed to be already existing.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsefilesystem-object-typeFilesystem Object TypeThey type of the file system object to be deleted. It can be either folder or file.fileFilefileThe object to be deleted is a file.DirectorydirectoryThe object to be deleted is a directory.truefalsefalseNONEfalsefalsedirectory-nameDirectory NameName of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. The root directory can be designated by the empty string value. In case of the PutAzureDataLakeStorage processor, the directory will be created if not already existing.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsefile-nameFile NameThe filename${azure.filename}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsefilesystem-object-typeFilesystem Object Typefileproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalsesuccessFiles that have been successfully written to Azure storage are transferred to this relationshipfalsefailureFiles that could not be written to Azure storage for some reason are transferred to this relationshipfalseINPUT_REQUIREDorg.apache.nifi.processors.azure.storage.PutAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.FetchAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.ListAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.FetchAzureBlobStorage_v12PROCESSORRetrieves the specified blob from Azure Blob Storage and writes its content to the content of the FlowFile. The processor uses Azure Blob Storage client library v12.azuremicrosoftcloudstorageblobstorage-credentials-serviceStorage CredentialsController Service used to obtain Azure Blob Storage Credentials.org.apache.nifi.services.azure.storage.AzureStorageCredentialsService_v12org.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalsecontainer-nameContainer NameName of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.${azure.container}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseblob-nameBlob NameThe full name of the blob${azure.blobname}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalserange-startRange StartThe byte position at which to start reading from the blob. An empty value or a value of zero will start reading at the beginning of the blob.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalserange-lengthRange LengthThe number of bytes to download from the blob, starting from the Range Start. An empty value or a value that extends beyond the end of the blob will read to the end of the blob.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalseClient-Side Encryption Key TypeClient-Side Encryption Key TypeSpecifies the key type to use for client-side encryption.NONENONENONEClient-Side Encryption disabledLOCALLOCALClient-Side Encryption enabled using local keytruefalsefalseNONEfalsefalseClient-Side Encryption Key IDClient-Side Encryption Key IDSpecifies the ID of the key to use for client-side encryption.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseClient-Side Encryption Key TypeClient-Side Encryption Key TypeLOCALClient-Side Encryption Local KeyClient-Side Encryption Local KeyWhen using local client-side encryption, this is the raw key, encoded in hexadecimaltruetruefalseNONEfalsefalseClient-Side Encryption Key TypeClient-Side Encryption Key TypeLOCALsuccessAll successfully processed FlowFiles are routed to this relationshipfalsefailureUnsuccessful operations will be transferred to the failure relationship.falseazure.containerThe name of the Azure Blob Storage containerazure.blobnameThe name of the blob on Azure Blob Storageazure.primaryUriPrimary location of the blobazure.etagETag of the blobazure.blobtypeType of the blob (either BlockBlob, PageBlob or AppendBlob)mime.typeMIME Type of the contentlangLanguage code for the contentazure.timestampTimestamp of the blobazure.lengthLength of the blobINPUT_REQUIREDRetrieve all files in an Azure Blob Storage containerazureblobstoragestateretrievefetchallstreamorg.apache.nifi.processors.azure.storage.ListAzureBlobStorage_v12The "Container Name" property should be set to the name of the Blob Storage Container that files reside in. If the flow being built is to be reused elsewhere, it's a good idea to parameterize this property by setting it to something like `#{AZURE_CONTAINER}`. The "Storage Credentials" property should specify an instance of the AzureStorageCredentialsService_v12 in order to provide credentials for accessing the storage container. The 'success' Relationship of this Processor is then connected to FetchAzureBlobStorage_v12. org.apache.nifi.processors.azure.storage.FetchAzureBlobStorage_v12"Container Name" = "${azure.container}" "Blob Name" = "${azure.blobname}" The "Storage Credentials" property should specify an instance of the AzureStorageCredentialsService_v12 in order to provide credentials for accessing the storage container. org.apache.nifi.processors.azure.storage.ListAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.PutAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.DeleteAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.FetchAzureDataLakeStoragePROCESSORFetch the specified file from Azure Data Lake Storageazuremicrosoftcloudstorageadlsgen2datalakeadls-credentials-serviceADLS CredentialsController Service used to obtain Azure Credentials.org.apache.nifi.services.azure.storage.ADLSCredentialsServiceorg.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalsefilesystem-nameFilesystem NameName of the Azure Storage File System (also called Container). It is assumed to be already existing.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedirectory-nameDirectory NameName of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. The root directory can be designated by the empty string value. In case of the PutAzureDataLakeStorage processor, the directory will be created if not already existing.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsefile-nameFile NameThe filename${azure.filename}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalserange-startRange StartThe byte position at which to start reading from the object. An empty value or a value of zero will start reading at the beginning of the object.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalserange-lengthRange LengthThe number of bytes to download from the object, starting from the Range Start. An empty value or a value that extends beyond the end of the object will read to the end of the object.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsenumber-of-retriesNumber of RetriesThe number of automatic retries to perform if the download fails.0falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalsesuccessFiles that have been successfully written to Azure storage are transferred to this relationshipfalsefailureFiles that could not be written to Azure storage for some reason are transferred to this relationshipfalseazure.datalake.storage.statusCodeThe HTTP error code (if available) from the failed operationazure.datalake.storage.errorCodeThe Azure Data Lake Storage moniker of the failed operationazure.datalake.storage.errorMessageThe Azure Data Lake Storage error message from the failed operationINPUT_REQUIREDRetrieve all files in an Azure DataLake Storage directoryazuredatalakeadlsstateretrievefetchallstreamorg.apache.nifi.processors.azure.storage.ListAzureDataLakeStorageThe "Filesystem Name" property should be set to the name of the Azure Filesystem (also known as a Container) that files reside in. If the flow being built is to be reused elsewhere, it's a good idea to parameterize this property by setting it to something like `#{AZURE_FILESYSTEM}`. Configure the "Directory Name" property to specify the name of the directory in the file system. If the flow being built is to be reused elsewhere, it's a good idea to parameterize this property by setting it to something like `#{AZURE_DIRECTORY}`. The "ADLS Credentials" property should specify an instance of the ADLSCredentialsService in order to provide credentials for accessing the filesystem. The 'success' Relationship of this Processor is then connected to FetchAzureDataLakeStorage. org.apache.nifi.processors.azure.storage.FetchAzureDataLakeStorage"Filesystem Name" = "${azure.filesystem}" "Directory Name" = "${azure.directory}" "File Name" = "${azure.filename}" The "ADLS Credentials" property should specify an instance of the ADLSCredentialsService in order to provide credentials for accessing the filesystem. org.apache.nifi.processors.azure.storage.PutAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.DeleteAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.ListAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.ListAzureBlobStorage_v12PROCESSORLists blobs in an Azure Blob Storage container. Listing details are attached to an empty FlowFile for use with FetchAzureBlobStorage. This Processor is designed to run on Primary Node only in a cluster. If the primary node changes, the new Primary Node will pick up where the previous node left off without duplicating all of the data. The processor uses Azure Blob Storage client library v12.azuremicrosoftcloudstorageblobstorage-credentials-serviceStorage CredentialsController Service used to obtain Azure Blob Storage Credentials.org.apache.nifi.services.azure.storage.AzureStorageCredentialsService_v12org.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalsecontainer-nameContainer NameName of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.truefalsetrueENVIRONMENTfalsefalseblob-name-prefixBlob Name PrefixSearch prefix for listingfalsefalsetrueENVIRONMENTfalsefalserecord-writerRecord WriterSpecifies the Record Writer to use for creating the listing. If not specified, one FlowFile will be created for each entity that is listed. If the Record Writer is specified, all entities will be written to a single FlowFile instead of adding attributes to individual FlowFiles.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalselisting-strategyListing StrategySpecify how to determine new/updated entities. See each strategy descriptions for detail.timestampsTracking TimestampstimestampsThis strategy tracks the latest timestamp of listed entity to determine new/updated entities. Since it only tracks few timestamps, it can manage listing state efficiently. However, any newly added, or updated entity having timestamp older than the tracked latest timestamp can not be picked by this strategy. For example, such situation can happen in a file system if a file with old timestamp is copied or moved into the target directory without its last modified timestamp being updated. Also may miss files when multiple subdirectories are being written at the same time while listing is running.Tracking EntitiesentitiesThis strategy tracks information of all the listed entities within the latest 'Entity Tracking Time Window' to determine new/updated entities. This strategy can pick entities having old timestamp that can be missed with 'Tracking Timestamps'. Works even when multiple subdirectories are being written at the same time while listing is running. However additional DistributedMapCache controller service is required and more JVM heap memory is used. See the description of 'Entity Tracking Time Window' property for further details on how it works.No TrackingnoneThis strategy lists an entity without any tracking. The same entity will be listed each time on executing this processor. It is recommended to change the default run schedule value. Any property that related to the persisting state will be disregarded.truefalsefalseNONEfalsefalseet-state-cacheEntity Tracking State CacheListed entities are stored in the specified cache storage so that this processor can resume listing across NiFi restart or in case of primary node change. 'Tracking Entities' strategy require tracking information of all listed entities within the last 'Tracking Time Window'. To support large number of entities, the strategy uses DistributedMapCache instead of managed state. Cache key format is 'ListedEntities::{processorId}(::{nodeId})'. If it tracks per node listed entities, then the optional '::{nodeId}' part is added to manage state separately. E.g. cluster wide cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b', per node cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b::nifi-node3' The stored cache content is Gzipped JSON string. The cache key will be deleted when target listing configuration is changed. Used by 'Tracking Entities' strategy.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalselisting-strategyListing Strategyentitieset-time-windowEntity Tracking Time WindowSpecify how long this processor should track already-listed entities. 'Tracking Entities' strategy can pick any entity whose timestamp is inside the specified time window. For example, if set to '30 minutes', any entity having timestamp in recent 30 minutes will be the listing target when this processor runs. A listed entity is considered 'new/updated' and a FlowFile is emitted if one of following condition meets: 1. does not exist in the already-listed entities, 2. has newer timestamp than the cached entity, 3. has different size than the cached entity. If a cached entity's timestamp becomes older than specified time window, that entity will be removed from the cached already-listed entities. Used by 'Tracking Entities' strategy.3 hoursfalsefalsetrueENVIRONMENTfalsefalselisting-strategyListing Strategyentitieset-initial-listing-targetEntity Tracking Initial Listing TargetSpecify how initial listing should be handled. Used by 'Tracking Entities' strategy.allTracking Time WindowwindowIgnore entities having timestamp older than the specified 'Tracking Time Window' at the initial listing activity.All AvailableallRegardless of entities timestamp, all existing entities will be listed at the initial listing activity.falsefalsefalseNONEfalsefalselisting-strategyListing StrategyentitiesMinimum File AgeMinimum File AgeThe minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored0 sectruefalsefalseNONEfalsefalseMaximum File AgeMaximum File AgeThe maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignoredfalsefalsefalseNONEfalsefalseMinimum File SizeMinimum File SizeThe minimum size that a file must be in order to be pulled0 BtruefalsefalseNONEfalsefalseMaximum File SizeMaximum File SizeThe maximum size that a file can be in order to be pulledfalsefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalsesuccessAll FlowFiles that are received are routed to successfalseazure.containerThe name of the Azure Blob Storage containerazure.blobnameThe name of the blob on Azure Blob Storageazure.primaryUriPrimary location of the blobazure.etagETag of the blobazure.blobtypeType of the blob (either BlockBlob, PageBlob or AppendBlob)mime.typeMIME Type of the contentlangLanguage code for the contentazure.timestampTimestamp of the blobazure.lengthLength of the blobtruetrueAfter performing a listing of blobs, the timestamp of the newest blob is stored if 'Tracking Timestamps' Listing Strategy is in use (by default). This allows the Processor to list only blobs that have been added or modified after this date the next time that the Processor is run. State is stored across the cluster so that this Processor can be run on Primary Node only and if a new Primary Node is selected, the new node can pick up where the previous node left off, without duplicating the data.CLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.azure.storage.FetchAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.PutAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.DeleteAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.CopyAzureBlobStorage_v12TIMER_DRIVEN1 min1org.apache.nifi.processors.azure.storage.ListAzureDataLakeStoragePROCESSORLists directory in an Azure Data Lake Storage Gen 2 filesystemazuremicrosoftcloudstorageadlsgen2datalakeadls-credentials-serviceADLS CredentialsController Service used to obtain Azure Credentials.org.apache.nifi.services.azure.storage.ADLSCredentialsServiceorg.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalsefilesystem-nameFilesystem NameName of the Azure Storage File System (also called Container). It is assumed to be already existing.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedirectory-nameDirectory NameName of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. The root directory can be designated by the empty string value. In case of the PutAzureDataLakeStorage processor, the directory will be created if not already existing.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalserecurse-subdirectoriesRecurse SubdirectoriesIndicates whether to list files from subdirectories of the directorytruetruetruefalsefalsetruefalsefalseNONEfalsefalsefile-filterFile FilterOnly files whose names match the given regular expression will be listedfalsefalsetrueENVIRONMENTfalsefalsepath-filterPath FilterWhen 'Recurse Subdirectories' is true, then only subdirectories whose paths match the given regular expression will be scannedfalsefalsetrueENVIRONMENTfalsefalseinclude-temporary-filesInclude Temporary FilesWhether to include temporary files when listing the contents of configured directory paths.falsetruetruefalsefalsetruefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Record Writer to use for creating the listing. If not specified, one FlowFile will be created for each entity that is listed. If the Record Writer is specified, all entities will be written to a single FlowFile instead of adding attributes to individual FlowFiles.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalselisting-strategyListing StrategySpecify how to determine new/updated entities. See each strategy descriptions for detail.timestampsTracking TimestampstimestampsThis strategy tracks the latest timestamp of listed entity to determine new/updated entities. Since it only tracks few timestamps, it can manage listing state efficiently. However, any newly added, or updated entity having timestamp older than the tracked latest timestamp can not be picked by this strategy. For example, such situation can happen in a file system if a file with old timestamp is copied or moved into the target directory without its last modified timestamp being updated. Also may miss files when multiple subdirectories are being written at the same time while listing is running.Tracking EntitiesentitiesThis strategy tracks information of all the listed entities within the latest 'Entity Tracking Time Window' to determine new/updated entities. This strategy can pick entities having old timestamp that can be missed with 'Tracking Timestamps'. Works even when multiple subdirectories are being written at the same time while listing is running. However additional DistributedMapCache controller service is required and more JVM heap memory is used. See the description of 'Entity Tracking Time Window' property for further details on how it works.No TrackingnoneThis strategy lists an entity without any tracking. The same entity will be listed each time on executing this processor. It is recommended to change the default run schedule value. Any property that related to the persisting state will be disregarded.truefalsefalseNONEfalsefalseet-state-cacheEntity Tracking State CacheListed entities are stored in the specified cache storage so that this processor can resume listing across NiFi restart or in case of primary node change. 'Tracking Entities' strategy require tracking information of all listed entities within the last 'Tracking Time Window'. To support large number of entities, the strategy uses DistributedMapCache instead of managed state. Cache key format is 'ListedEntities::{processorId}(::{nodeId})'. If it tracks per node listed entities, then the optional '::{nodeId}' part is added to manage state separately. E.g. cluster wide cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b', per node cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b::nifi-node3' The stored cache content is Gzipped JSON string. The cache key will be deleted when target listing configuration is changed. Used by 'Tracking Entities' strategy.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalseet-time-windowEntity Tracking Time WindowSpecify how long this processor should track already-listed entities. 'Tracking Entities' strategy can pick any entity whose timestamp is inside the specified time window. For example, if set to '30 minutes', any entity having timestamp in recent 30 minutes will be the listing target when this processor runs. A listed entity is considered 'new/updated' and a FlowFile is emitted if one of following condition meets: 1. does not exist in the already-listed entities, 2. has newer timestamp than the cached entity, 3. has different size than the cached entity. If a cached entity's timestamp becomes older than specified time window, that entity will be removed from the cached already-listed entities. Used by 'Tracking Entities' strategy.3 hoursfalsefalsetrueENVIRONMENTfalsefalseet-initial-listing-targetEntity Tracking Initial Listing TargetSpecify how initial listing should be handled. Used by 'Tracking Entities' strategy.allTracking Time WindowwindowIgnore entities having timestamp older than the specified 'Tracking Time Window' at the initial listing activity.All AvailableallRegardless of entities timestamp, all existing entities will be listed at the initial listing activity.falsefalsefalseNONEfalsefalseMinimum File AgeMinimum File AgeThe minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored0 sectruefalsefalseNONEfalsefalseMaximum File AgeMaximum File AgeThe maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignoredfalsefalsefalseNONEfalsefalseMinimum File SizeMinimum File SizeThe minimum size that a file must be in order to be pulled0 BtruefalsefalseNONEfalsefalseMaximum File SizeMaximum File SizeThe maximum size that a file can be in order to be pulledfalsefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalsesuccessAll FlowFiles that are received are routed to successfalseazure.filesystemThe name of the Azure File Systemazure.filePathThe full path of the Azure Fileazure.directoryThe name of the Azure Directoryazure.filenameThe name of the Azure Fileazure.lengthThe length of the Azure Fileazure.lastModifiedThe last modification time of the Azure Fileazure.etagThe ETag of the Azure FiletruetrueAfter performing a listing of files, the timestamp of the newest file is stored. This allows the Processor to list only files that have been added or modified after this date the next time that the Processor is run. State is stored across the cluster so that this Processor can be run on Primary Node only and if a new Primary Node is selected, the new node can pick up where the previous node left off, without duplicating the data.CLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.azure.storage.PutAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.DeleteAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.FetchAzureDataLakeStorageTIMER_DRIVEN1 min1org.apache.nifi.processors.azure.storage.MoveAzureDataLakeStoragePROCESSORMoves content within an Azure Data Lake Storage Gen 2. After the move, files will be no longer available on source location.azuremicrosoftcloudstorageadlsgen2datalakeadls-credentials-serviceADLS CredentialsController Service used to obtain Azure Credentials.org.apache.nifi.services.azure.storage.ADLSCredentialsServiceorg.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalsesource-filesystem-nameSource FilesystemName of the Azure Storage File System from where the move should happen.${azure.filesystem}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesource-directory-nameSource DirectoryName of the Azure Storage Directory from where the move should happen. The Directory Name cannot contain a leading '/'. The root directory can be designated by the empty string value.${azure.directory}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsefilesystem-nameDestination FilesystemName of the Azure Storage File System where the files will be moved.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedirectory-nameDestination DirectoryName of the Azure Storage Directory where the files will be moved. The Directory Name cannot contain a leading '/'. The root directory can be designated by the empty string value. Non-existing directories will be created. If the original directory structure should be kept, the full directory path needs to be provided after the destination directory. e.g.: destdir/${azure.directory}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsefile-nameFile NameThe filename${azure.filename}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseconflict-resolution-strategyConflict Resolution StrategyIndicates what should happen when a file with the same name already exists in the output directoryfailfailfailreplacereplaceignoreignoretruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalsesuccessFiles that have been successfully written to Azure storage are transferred to this relationshipfalsefailureFiles that could not be written to Azure storage for some reason are transferred to this relationshipfalseazure.source.filesystemThe name of the source Azure File Systemazure.source.directoryThe name of the source Azure Directoryazure.filesystemThe name of the Azure File Systemazure.directoryThe name of the Azure Directoryazure.filenameThe name of the Azure Fileazure.primaryUriPrimary location for file contentazure.lengthThe length of the Azure FileINPUT_REQUIREDorg.apache.nifi.processors.azure.storage.DeleteAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.FetchAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.ListAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.PutAzureBlobStorage_v12PROCESSORPuts content into a blob on Azure Blob Storage. The processor uses Azure Blob Storage client library v12.azuremicrosoftcloudstorageblobstorage-credentials-serviceStorage CredentialsController Service used to obtain Azure Blob Storage Credentials.org.apache.nifi.services.azure.storage.AzureStorageCredentialsService_v12org.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalsecontainer-nameContainer NameName of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsecreate-containerCreate ContainerSpecifies whether to check if the container exists and to automatically create it if it does not. Permission to list containers is required. If false, this check is not made, but the Put operation will fail if the container does not exist.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseconflict-resolution-strategyConflict Resolution StrategySpecifies whether an existing blob will have its contents replaced upon conflict.FAIL_RESOLUTIONfailFAIL_RESOLUTIONFail if the blob already existsignoreIGNORE_RESOLUTIONIgnore if the blob already exists; the 'azure.error' attribute will be set to the value 'BLOB_ALREADY_EXISTS'replaceREPLACE_RESOLUTIONReplace blob contents if the blob already existtruefalsefalseNONEfalsefalseblob-nameBlob NameThe full name of the blobtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseResource Transfer SourceResource Transfer SourceThe source of the content to be transferredFLOWFILE_CONTENTFlowFile ContentFLOWFILE_CONTENTThe content of the incoming FlowFile provides the source for transferFile Resource ServiceFILE_RESOURCE_SERVICEThe File Resource Service provides the source for transfertruefalsefalseNONEfalsefalseFile Resource ServiceFile Resource ServiceFile Resource Service providing access to the local resource to be transferredorg.apache.nifi.fileresource.service.api.FileResourceServiceorg.apache.nifinifi-standard-services-api-nar2.2.0truefalsefalseNONEfalsefalseResource Transfer SourceResource Transfer SourceFILE_RESOURCE_SERVICEproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalseClient-Side Encryption Key TypeClient-Side Encryption Key TypeSpecifies the key type to use for client-side encryption.NONENONENONEClient-Side Encryption disabledLOCALLOCALClient-Side Encryption enabled using local keytruefalsefalseNONEfalsefalseClient-Side Encryption Key IDClient-Side Encryption Key IDSpecifies the ID of the key to use for client-side encryption.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseClient-Side Encryption Key TypeClient-Side Encryption Key TypeLOCALClient-Side Encryption Local KeyClient-Side Encryption Local KeyWhen using local client-side encryption, this is the raw key, encoded in hexadecimaltruetruefalseNONEfalsefalseClient-Side Encryption Key TypeClient-Side Encryption Key TypeLOCALsuccessAll successfully processed FlowFiles are routed to this relationshipfalsefailureUnsuccessful operations will be transferred to the failure relationship.falseazure.containerThe name of the Azure Blob Storage containerazure.blobnameThe name of the blob on Azure Blob Storageazure.primaryUriPrimary location of the blobazure.etagETag of the blobazure.blobtypeType of the blob (either BlockBlob, PageBlob or AppendBlob)mime.typeMIME Type of the contentlangLanguage code for the contentazure.timestampTimestamp of the blobazure.lengthLength of the blobazure.error.codeError code reported during blob operationazure.ignoredWhen Conflict Resolution Strategy is 'ignore', this property will be true/false depending on whether the blob was ignored.INPUT_REQUIREDorg.apache.nifi.processors.azure.storage.ListAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.FetchAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.DeleteAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.CopyAzureBlobStorage_v12org.apache.nifi.processors.azure.storage.PutAzureDataLakeStoragePROCESSORWrites the contents of a FlowFile as a file on Azure Data Lake Storage Gen 2azuremicrosoftcloudstorageadlsgen2datalakeadls-credentials-serviceADLS CredentialsController Service used to obtain Azure Credentials.org.apache.nifi.services.azure.storage.ADLSCredentialsServiceorg.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalsefilesystem-nameFilesystem NameName of the Azure Storage File System (also called Container). It is assumed to be already existing.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedirectory-nameDirectory NameName of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. The root directory can be designated by the empty string value. In case of the PutAzureDataLakeStorage processor, the directory will be created if not already existing.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsefile-nameFile NameThe filename${azure.filename}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsewriting-strategyWriting StrategyDefines the approach for writing the Azure file.WRITE_AND_RENAMEWrite and RenameWRITE_AND_RENAMEThe processor writes the Azure file into a temporary directory and then renames/moves it to the final destination. This prevents other processes from reading partially written files.Simple WriteSIMPLE_WRITEThe processor writes the Azure file directly to the destination. This might result in the reading of partially written files.truefalsefalseNONEfalsefalsebase-temporary-pathBase Temporary PathThe Path where the temporary directory will be created. The Path name cannot contain a leading '/'. The root directory can be designated by the empty string value. Non-existing directories will be created.The Temporary File Directory name is _nifitempdirectoryfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsewriting-strategyWriting StrategyWRITE_AND_RENAMEconflict-resolution-strategyConflict Resolution StrategyIndicates what should happen when a file with the same name already exists in the output directoryfailfailfailreplacereplaceignoreignoretruefalsefalseNONEfalsefalseResource Transfer SourceResource Transfer SourceThe source of the content to be transferredFLOWFILE_CONTENTFlowFile ContentFLOWFILE_CONTENTThe content of the incoming FlowFile provides the source for transferFile Resource ServiceFILE_RESOURCE_SERVICEThe File Resource Service provides the source for transfertruefalsefalseNONEfalsefalseFile Resource ServiceFile Resource ServiceFile Resource Service providing access to the local resource to be transferredorg.apache.nifi.fileresource.service.api.FileResourceServiceorg.apache.nifinifi-standard-services-api-nar2.2.0truefalsefalseNONEfalsefalseResource Transfer SourceResource Transfer SourceFILE_RESOURCE_SERVICEproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalsesuccessFiles that have been successfully written to Azure storage are transferred to this relationshipfalsefailureFiles that could not be written to Azure storage for some reason are transferred to this relationshipfalseazure.filesystemThe name of the Azure File Systemazure.directoryThe name of the Azure Directoryazure.filenameThe name of the Azure Fileazure.primaryUriPrimary location for file contentazure.lengthThe length of the Azure FileINPUT_REQUIREDorg.apache.nifi.processors.azure.storage.DeleteAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.FetchAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.ListAzureDataLakeStorageorg.apache.nifi.processors.azure.storage.queue.GetAzureQueueStorage_v12PROCESSORRetrieves the messages from an Azure Queue Storage. The retrieved messages will be deleted from the queue by default. If the requirement is to consume messages without deleting them, set 'Auto Delete Messages' to 'false'. Note: There might be chances of receiving duplicates in situations like when a message is received but was unable to be deleted from the queue due to some unexpected situations.azurequeuemicrosoftstoragedequeuecloudQueue NameQueue NameName of the Azure Storage QueuetruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsestorage-endpoint-suffixEndpoint SuffixStorage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions).queue.core.windows.nettruefalsefalseNONEfalsefalseCredentials ServiceCredentials ServiceController Service used to obtain Azure Storage Credentials.org.apache.nifi.services.azure.storage.AzureStorageCredentialsService_v12org.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalseAuto Delete MessagesAuto Delete MessagesSpecifies whether the received message is to be automatically deleted from the queue.truetruetruefalsefalsetruefalsefalseNONEfalsefalseMessage Batch SizeMessage Batch SizeThe number of messages to be retrieved from the queue.32truefalsefalseNONEfalsefalseVisibility TimeoutVisibility TimeoutThe duration during which the retrieved message should be invisible to other consumers.30 secstruefalsefalseNONEfalsefalseRequest TimeoutRequest TimeoutThe timeout for read or write requests to Azure Queue Storage. Defaults to 1 second.10 secstruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalsesuccessAll successfully processed FlowFiles are routed to this relationshipfalseazure.queue.uriThe absolute URI of the configured Azure Queue Storageazure.queue.insertionTimeThe time when the message was inserted into the queue storageazure.queue.expirationTimeThe time when the message will expire from the queue storageazure.queue.messageIdThe ID of the retrieved messageazure.queue.popReceiptThe pop receipt of the retrieved messageINPUT_FORBIDDENorg.apache.nifi.processors.azure.storage.queue.PutAzureQueueStorage_v12org.apache.nifi.processors.azure.storage.queue.PutAzureQueueStorage_v12PROCESSORWrites the content of the incoming FlowFiles to the configured Azure Queue Storage.azuremicrosoftcloudstoragequeueenqueueQueue NameQueue NameName of the Azure Storage QueuetruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsestorage-endpoint-suffixEndpoint SuffixStorage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions).queue.core.windows.nettruefalsefalseNONEfalsefalseCredentials ServiceCredentials ServiceController Service used to obtain Azure Storage Credentials.org.apache.nifi.services.azure.storage.AzureStorageCredentialsService_v12org.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalseMessage Time To LiveMessage Time To LiveMaximum time to allow the message to be in the queue7 daystruefalsefalseNONEfalsefalseVisibility TimeoutVisibility TimeoutThe length of time during which the message will be invisible after it is read. If the processing unit fails to delete the message after it is read, then the message will reappear in the queue.30 secstruefalsefalseNONEfalsefalseRequest TimeoutRequest TimeoutThe timeout for read or write requests to Azure Queue Storage. Defaults to 1 second.10 secstruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalsesuccessAll successfully processed FlowFiles are routed to this relationshipfalsefailureUnsuccessful operations will be transferred to the failure relationship.falseINPUT_REQUIREDorg.apache.nifi.processors.azure.storage.queue.GetAzureQueueStorage_v12org.apache.nifi.services.azure.StandardAzureCredentialsControllerServiceCONTROLLER_SERVICEProvide credentials to use with an Azure client.azuresecuritycredentialsprovidersessioncredential-configuration-strategyCredential Configuration Strategydefault-credentialDefault Credentialdefault-credentialUses default credential chain. It first checks environment variables, before trying managed identity.Managed Identitymanaged-identityAzure Virtual Machine Managed Identity (it can only be used when NiFi is running on Azure)truefalsefalseNONEfalsefalsemanaged-identity-client-idManaged Identity Client IDClient ID of the managed identity. The property is required when User Assigned Managed Identity is used for authentication. It must be empty in case of System Assigned Managed Identity.falsetruefalseNONEfalsefalsecredential-configuration-strategyCredential Configuration Strategymanaged-identityorg.apache.nifi.services.azure.AzureCredentialsServiceorg.apache.nifinifi-azure-services-api-nar2.2.0org.apache.nifi.services.azure.cosmos.document.AzureCosmosDBClientServiceCONTROLLER_SERVICEProvides a controller service that configures a connection to Cosmos DB (Core SQL API) and provides access to that connection to other Cosmos DB-related components.azurecosmosdocumentserviceazure-cosmos-db-uriCosmos DB URICosmos DB URI, typically in the form of https://{databaseaccount}.documents.azure.com:443/ Note this host URL is for Cosmos DB with Core SQL API from Azure Portal (Overview->URI)falsetruefalseNONEfalsefalseazure-cosmos-db-keyCosmos DB Access KeyCosmos DB Access Key from Azure Portal (Settings->Keys). Choose a read-write key to enable database or container creation at run timefalsetruefalseNONEfalsefalseazure-cosmos-db-consistency-levelCosmos DB Consistency LevelChoose from five consistency levels on the consistency spectrum. Refer to Cosmos DB documentation for their differencesSESSIONSTRONGSTRONGBOUNDED_STALENESSBOUNDED_STALENESSSESSIONSESSIONCONSISTENT_PREFIXCONSISTENT_PREFIXEVENTUALEVENTUALfalsefalsefalseNONEfalsefalseorg.apache.nifi.services.azure.cosmos.AzureCosmosDBConnectionServiceorg.apache.nifinifi-azure-services-api-nar2.2.0org.apache.nifi.services.azure.data.explorer.StandardKustoIngestServiceCONTROLLER_SERVICESends batches of flowfile content or stream flowfile content to an Azure ADX cluster.AzureDataExplorerADXKustoingestazureAuthentication StrategyAuthentication StrategyAuthentication method for access to Azure Data ExplorerMANAGED_IDENTITYApplication CredentialsAPPLICATION_CREDENTIALSAzure Application Registration with Application KeyManaged IdentityMANAGED_IDENTITYAzure Managed IdentitytruefalsefalseNONEfalsefalseApplication Client IDApplication Client IDAzure Data Explorer Application Client Identifier for AuthenticationtruefalsefalseNONEfalsefalseApplication KeyApplication KeyAzure Data Explorer Application Key for AuthenticationtruetruefalseNONEfalsefalseAuthentication StrategyAuthentication StrategyAPPLICATION_CREDENTIALSApplication Tenant IDApplication Tenant IDAzure Data Explorer Application Tenant Identifier for AuthenticationtruefalsefalseNONEfalsefalseAuthentication StrategyAuthentication StrategyAPPLICATION_CREDENTIALSCluster URICluster URIAzure Data Explorer Cluster URItruefalsefalseNONEfalsefalseorg.apache.nifi.services.azure.data.explorer.KustoIngestServiceorg.apache.nifinifi-azure-services-api-nar2.2.0org.apache.nifi.services.azure.data.explorer.StandardKustoQueryServiceCONTROLLER_SERVICEStandard implementation of Kusto Query Service for Azure Data ExplorerAzureDataExplorerADXKustoCluster URICluster URIAzure Data Explorer Cluster URItruefalsefalseNONEfalsefalseAuthentication StrategyAuthentication StrategyAuthentication method for access to Azure Data ExplorerMANAGED_IDENTITYApplication CredentialsAPPLICATION_CREDENTIALSAzure Application Registration with Application KeyManaged IdentityMANAGED_IDENTITYAzure Managed IdentitytruefalsefalseNONEfalsefalseApplication Client IDApplication Client IDAzure Data Explorer Application Client Identifier for AuthenticationtruefalsefalseNONEfalsefalseApplication Tenant IDApplication Tenant IDAzure Data Explorer Application Tenant Identifier for AuthenticationtruefalsefalseNONEfalsefalseAuthentication StrategyAuthentication StrategyAPPLICATION_CREDENTIALSApplication KeyApplication KeyAzure Data Explorer Application Key for AuthenticationtruetruefalseNONEfalsefalseAuthentication StrategyAuthentication StrategyAPPLICATION_CREDENTIALSorg.apache.nifi.services.azure.data.explorer.KustoQueryServiceorg.apache.nifinifi-azure-services-api-nar2.2.0org.apache.nifi.services.azure.eventhub.AzureEventHubRecordSinkCONTROLLER_SERVICEFormat and send Records to Azure Event HubsazurerecordsinkService Bus EndpointService Bus EndpointProvides the domain for connecting to Azure Event Hubs.servicebus.windows.netAzure.servicebus.windows.netDefault Service Bus EndpointAzure China.servicebus.chinacloudapi.cnChina Service Bus EndpointAzure Germany.servicebus.cloudapi.deGermany Service Bus EndpointAzure US Government.servicebus.usgovcloudapi.netUnited States Government EndpointtruefalsefalseNONEfalsefalseEvent Hub NamespaceEvent Hub NamespaceProvides provides the host for connecting to Azure Event HubstruefalsefalseNONEfalsefalseEvent Hub NameEvent Hub NameProvides the Event Hub Name for connectionstruefalsefalseNONEfalsefalseTransport TypeTransport TypeAdvanced Message Queuing Protocol Transport Type for communication with Azure Event HubsAmqpAMQPAmqpAMQP over TCP on ports 5671 and 5672AMQP_WEB_SOCKETSAmqpWebSocketsAMQP over HTTPS with WebSockets on port 443truefalsefalseNONEfalsefalserecord-sink-record-writerRecord WriterSpecifies the Controller Service to use for writing out the records.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.2.0truefalsefalseNONEfalsefalseAuthentication StrategyAuthentication StrategyStrategy for authenticating to Azure Event HubsDEFAULT_AZURE_CREDENTIALShared Access KeySHARED_ACCESS_KEYAzure Event Hub shared access keyDefault Azure CredentialDEFAULT_AZURE_CREDENTIALThe Default Azure Credential will read credentials from standard environment variables and will also attempt to read Managed Identity credentials when running in Microsoft Azure environmentstruefalsefalseNONEfalsefalseShared Access PolicyShared Access PolicyThe name of the shared access policy. This policy must have Send claimsfalsefalsefalseNONEfalsefalseAuthentication StrategyAuthentication StrategySHARED_ACCESS_KEYShared Access Policy KeyShared Access Policy KeyThe primary or secondary key of the shared access policyfalsetruefalseNONEfalsefalseAuthentication StrategyAuthentication StrategySHARED_ACCESS_KEYPartition KeyPartition KeyA hint for Azure Event Hub message broker how to distribute messages across one or more partitionsfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseorg.apache.nifi.record.sink.RecordSinkServiceorg.apache.nifinifi-standard-services-api-nar2.2.0org.apache.nifi.services.azure.storage.ADLSCredentialsControllerServiceCONTROLLER_SERVICEDefines credentials for ADLS processors.azuremicrosoftcloudstorageadlscredentialsstorage-account-nameStorage Account NameThe storage account name. There are certain risks in allowing the account name to be stored as a FlowFile attribute. While it does provide for a more flexible flow by allowing the account name to be fetched dynamically from a FlowFile attribute, care must be taken to restrict access to the event provenance data (e.g., by strictly controlling the policies governing provenance for this processor). In addition, the provenance repositories may be put on encrypted disk partitions.truetruetrueFLOWFILE_ATTRIBUTESfalsefalsestorage-endpoint-suffixEndpoint SuffixStorage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions).dfs.core.windows.nettruefalsetrueENVIRONMENTfalsefalsecredentials-typeCredentials TypeCredentials type to be used for authenticating to AzureSAS_TOKENAccount KeyACCOUNT_KEYThe primary or secondary Account Key of the storage account that provides full access to the resources in the accountSAS TokenSAS_TOKENSAS (Shared Access Signature) Token generated for accessing resources in the storage accountManaged IdentityMANAGED_IDENTITYAzure Virtual Machine Managed Identity (it can only be used when NiFi is running on Azure)Service PrincipalSERVICE_PRINCIPALAzure Active Directory Service Principal with Client Id / Client Secret of a registered applicationtruefalsefalseNONEfalsefalsestorage-account-keyAccount KeyThe storage account key. This is an admin-like password providing access to every container in this account. It is recommended one uses Shared Access Signature (SAS) token, Managed Identity or Service Principal instead for fine-grained control with policies. There are certain risks in allowing the account key to be stored as a FlowFile attribute. While it does provide for a more flexible flow by allowing the account key to be fetched dynamically from a FlowFile attribute, care must be taken to restrict access to the event provenance data (e.g., by strictly controlling the policies governing provenance for this processor). In addition, the provenance repositories may be put on encrypted disk partitions.truetruetrueFLOWFILE_ATTRIBUTESfalsefalsecredentials-typeCredentials TypeACCOUNT_KEYstorage-sas-tokenSAS TokenShared Access Signature token (the leading '?' may be included) There are certain risks in allowing the SAS token to be stored as a FlowFile attribute. While it does provide for a more flexible flow by allowing the SAS token to be fetched dynamically from a FlowFile attribute, care must be taken to restrict access to the event provenance data (e.g., by strictly controlling the policies governing provenance for this processor). In addition, the provenance repositories may be put on encrypted disk partitions.truetruetrueFLOWFILE_ATTRIBUTESfalsefalsecredentials-typeCredentials TypeSAS_TOKENmanaged-identity-client-idManaged Identity Client IDClient ID of the managed identity. The property is required when User Assigned Managed Identity is used for authentication. It must be empty in case of System Assigned Managed Identity.falsetruefalseNONEfalsefalsecredentials-typeCredentials TypeMANAGED_IDENTITYservice-principal-tenant-idService Principal Tenant IDTenant ID of the Azure Active Directory hosting the Service Principal.truetruefalseNONEfalsefalsecredentials-typeCredentials TypeSERVICE_PRINCIPALservice-principal-client-idService Principal Client IDClient ID (or Application ID) of the Client/Application having the Service Principal.truetruefalseNONEfalsefalsecredentials-typeCredentials TypeSERVICE_PRINCIPALservice-principal-client-secretService Principal Client SecretPassword of the Client/Application.truetruefalseNONEfalsefalsecredentials-typeCredentials TypeSERVICE_PRINCIPALproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalsecredentials-typeCredentials TypeSERVICE_PRINCIPALMANAGED_IDENTITYorg.apache.nifi.services.azure.storage.ADLSCredentialsServiceorg.apache.nifinifi-azure-services-api-nar2.2.0org.apache.nifi.services.azure.storage.ADLSCredentialsControllerServiceLookupCONTROLLER_SERVICEProvides an ADLSCredentialsService that can be used to dynamically select another ADLSCredentialsService. This service requires an attribute named 'adls.credentials.name' to be passed in, and will throw an exception if the attribute is missing. The value of 'adls.credentials.name' will be used to select the ADLSCredentialsService that has been registered with that name. This will allow multiple ADLSCredentialsServices to be defined and registered, and then selected dynamically at runtime by tagging flow files with the appropriate 'adls.credentials.name' attribute.azuremicrosoftcloudstorageadlscredentialsThe name to register ADLSCredentialsServiceThe ADLSCredentialsServiceIf 'adls.credentials.name' attribute contains the name of the dynamic property, then the ADLSCredentialsService (registered in the value) will be selected.NONEorg.apache.nifi.services.azure.storage.ADLSCredentialsServiceorg.apache.nifinifi-azure-services-api-nar2.2.0org.apache.nifi.services.azure.storage.AzureBlobStorageFileResourceServiceCONTROLLER_SERVICEProvides an Azure Blob Storage file resource for other components.azuremicrosoftcloudstoragefileresourceblobstorage-credentials-serviceStorage CredentialsController Service used to obtain Azure Blob Storage Credentials.org.apache.nifi.services.azure.storage.AzureStorageCredentialsService_v12org.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalsecontainer-nameContainer NameName of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.${azure.container}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseblob-nameBlob NameThe full name of the blob${azure.blobname}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFetch a specific file from Azure Blob Storage. The service provides higher performance compared to fetch processors when the data should be moved between different storages without any transformation.INPUT_ALLOWED"Container Name" = "${azure.container}" "Blob Name" = "${azure.blobname}" The "Storage Credentials" property should specify an instance of the AzureStorageCredentialsService_v12 in order to provide credentials for accessing the storage container. org.apache.nifi.processors.azure.storage.FetchAzureBlobStorage_v12org.apache.nifi.fileresource.service.api.FileResourceServiceorg.apache.nifinifi-standard-services-api-nar2.2.0org.apache.nifi.services.azure.storage.AzureDataLakeStorageFileResourceServiceCONTROLLER_SERVICEProvides an Azure Data Lake Storage (ADLS) file resource for other components.azuremicrosoftcloudstorageadlsgen2fileresourcedatalakeadls-credentials-serviceADLS CredentialsController Service used to obtain Azure Credentials.org.apache.nifi.services.azure.storage.ADLSCredentialsServiceorg.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalsefilesystem-nameFilesystem NameName of the Azure Storage File System (also called Container). It is assumed to be already existing.${azure.filesystem}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedirectory-nameDirectory NameName of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. The root directory can be designated by the empty string value. In case of the PutAzureDataLakeStorage processor, the directory will be created if not already existing.${azure.directory}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsefile-nameFile NameThe filename${azure.filename}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFetch the specified file from Azure Data Lake Storage. The service provides higher performance compared to fetch processors when the data should be moved between different storages without any transformation.INPUT_ALLOWED"Filesystem Name" = "${azure.filesystem}" "Directory Name" = "${azure.directory}" "File Name" = "${azure.filename}" The "ADLS Credentials" property should specify an instance of the ADLSCredentialsService in order to provide credentials for accessing the filesystem. org.apache.nifi.processors.azure.storage.FetchAzureDataLakeStorageorg.apache.nifi.fileresource.service.api.FileResourceServiceorg.apache.nifinifi-standard-services-api-nar2.2.0org.apache.nifi.services.azure.storage.AzureStorageCredentialsControllerServiceLookup_v12CONTROLLER_SERVICEProvides an AzureStorageCredentialsService_v12 that can be used to dynamically select another AzureStorageCredentialsService_v12. This service requires an attribute named 'azure.storage.credentials.name' to be passed in, and will throw an exception if the attribute is missing. The value of 'azure.storage.credentials.name' will be used to select the AzureStorageCredentialsService_v12 that has been registered with that name. This will allow multiple AzureStorageCredentialsServices_v12 to be defined and registered, and then selected dynamically at runtime by tagging flow files with the appropriate 'azure.storage.credentials.name' attribute.azuremicrosoftcloudstorageblobqueuecredentialsThe name to register AzureStorageCredentialsService_v12The AzureStorageCredentialsService_v12If 'azure.storage.credentials.name' attribute contains the name of the dynamic property, then the AzureStorageCredentialsService_v12 (registered in the value) will be selected.NONEorg.apache.nifi.services.azure.storage.AzureStorageCredentialsService_v12org.apache.nifinifi-azure-services-api-nar2.2.0org.apache.nifi.services.azure.storage.AzureStorageCredentialsControllerService_v12CONTROLLER_SERVICEProvides credentials for Azure Storage processors using Azure Storage client library v12.azuremicrosoftcloudstorageblobcredentialsqueuestorage-account-nameStorage Account NameThe storage account name.truetruefalseNONEfalsefalsestorage-endpoint-suffixEndpoint SuffixStorage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions).blob.core.windows.nettruefalsefalseNONEfalsefalsecredentials-typeCredentials TypeCredentials type to be used for authenticating to AzureSAS_TOKENAccount KeyACCOUNT_KEYThe primary or secondary Account Key of the storage account that provides full access to the resources in the accountSAS TokenSAS_TOKENSAS (Shared Access Signature) Token generated for accessing resources in the storage accountManaged IdentityMANAGED_IDENTITYAzure Virtual Machine Managed Identity (it can only be used when NiFi is running on Azure)Service PrincipalSERVICE_PRINCIPALAzure Active Directory Service Principal with Client Id / Client Secret of a registered applicationtruefalsefalseNONEfalsefalsestorage-account-keyAccount KeyThe storage account key. This is an admin-like password providing access to every container in this account. It is recommended one uses Shared Access Signature (SAS) token, Managed Identity or Service Principal instead for fine-grained control with policies.truetruefalseNONEfalsefalsecredentials-typeCredentials TypeACCOUNT_KEYstorage-sas-tokenSAS TokenShared Access Signature token (the leading '?' may be included)truetruefalseNONEfalsefalsecredentials-typeCredentials TypeSAS_TOKENmanaged-identity-client-idManaged Identity Client IDClient ID of the managed identity. The property is required when User Assigned Managed Identity is used for authentication. It must be empty in case of System Assigned Managed Identity.falsetruefalseNONEfalsefalsecredentials-typeCredentials TypeMANAGED_IDENTITYservice-principal-tenant-idService Principal Tenant IDTenant ID of the Azure Active Directory hosting the Service Principal.truetruefalseNONEfalsefalsecredentials-typeCredentials TypeSERVICE_PRINCIPALservice-principal-client-idService Principal Client IDClient ID (or Application ID) of the Client/Application having the Service Principal.truetruefalseNONEfalsefalsecredentials-typeCredentials TypeSERVICE_PRINCIPALservice-principal-client-secretService Principal Client SecretPassword of the Client/Application.truetruefalseNONEfalsefalsecredentials-typeCredentials TypeSERVICE_PRINCIPALproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: SOCKS, HTTP In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.2.0falsefalsefalseNONEfalsefalsecredentials-typeCredentials TypeSERVICE_PRINCIPALMANAGED_IDENTITYorg.apache.nifi.services.azure.storage.AzureStorageCredentialsService_v12org.apache.nifinifi-azure-services-api-nar2.2.0org.apache.nifi.reporting.azure.loganalytics.AzureLogAnalyticsProvenanceReportingTaskREPORTING_TASKPublishes Provenance events to to a Azure Log Analytics workspace.azureprovenacereportinglog analyticsLog Analytics Workspace IdLog Analytics Workspace IdLog Analytics Workspace IdtruetruetrueENVIRONMENTfalsefalseLog Analytics Custom Log NameLog Analytics Custom Log NameLog Analytics Custom Log NamenifiprovenancefalsefalsetrueENVIRONMENTfalsefalseLog Analytics Workspace KeyLog Analytics Workspace KeyAzure Log Analytic Worskspace KeytruetruetrueENVIRONMENTfalsefalseApplication IDApplication IDThe Application ID to be included in the metrics sent to Azure Log Analytics WSnifitruefalsetrueENVIRONMENTfalsefalseInstance IDInstance IDId of this NiFi instance to be included in the metrics sent to Azure Log Analytics WS${hostname(true)}truefalsetrueENVIRONMENTfalsefalseJob NameJob NameThe name of the exporting jobnifi_reporting_jobfalsefalsetrueENVIRONMENTfalsefalseLog Analytics URL Endpoint FormatLog Analytics URL Endpoint FormatLog Analytics URL Endpoint Formathttps://{0}.ods.opinsights.azure.com/api/logs?api-version=2016-04-01falsefalsetrueENVIRONMENTfalsefalses2s-prov-task-event-filterEvent Type to IncludeComma-separated list of event types that will be used to filter the provenance events sent by the reporting task. Available event types are [CREATE, RECEIVE, FETCH, SEND, UPLOAD, REMOTE_INVOCATION, DOWNLOAD, DROP, EXPIRE, FORK, JOIN, CLONE, CONTENT_MODIFIED, ATTRIBUTES_MODIFIED, ROUTE, ADDINFO, REPLAY, UNKNOWN]. If no filter is set, all the events are sent. If multiple filters are set, the filters are cumulative.falsefalsetrueENVIRONMENTfalsefalses2s-prov-task-event-filter-excludeEvent Type to ExcludeComma-separated list of event types that will be used to exclude the provenance events sent by the reporting task. Available event types are [CREATE, RECEIVE, FETCH, SEND, UPLOAD, REMOTE_INVOCATION, DOWNLOAD, DROP, EXPIRE, FORK, JOIN, CLONE, CONTENT_MODIFIED, ATTRIBUTES_MODIFIED, ROUTE, ADDINFO, REPLAY, UNKNOWN]. If no filter is set, all the events are sent. If multiple filters are set, the filters are cumulative. If an event type is included in Event Type to Include and excluded here, then the exclusion takes precedence and the event will not be sent.falsefalsetrueENVIRONMENTfalsefalses2s-prov-task-type-filterComponent Type to IncludeRegular expression to filter the provenance events based on the component type. Only the events matching the regular expression will be sent. If no filter is set, all the events are sent. If multiple filters are set, the filters are cumulative.falsefalsetrueENVIRONMENTfalsefalses2s-prov-task-type-filter-excludeComponent Type to ExcludeRegular expression to exclude the provenance events based on the component type. The events matching the regular expression will not be sent. If no filter is set, all the events are sent. If multiple filters are set, the filters are cumulative. If a component type is included in Component Type to Include and excluded here, then the exclusion takes precedence and the event will not be sent.falsefalsetrueENVIRONMENTfalsefalses2s-prov-task-id-filterComponent ID to IncludeComma-separated list of component UUID that will be used to filter the provenance events sent by the reporting task. If no filter is set, all the events are sent. If multiple filters are set, the filters are cumulative.falsefalsetrueENVIRONMENTfalsefalses2s-prov-task-id-filter-excludeComponent ID to ExcludeComma-separated list of component UUID that will be used to exclude the provenance events sent by the reporting task. If no filter is set, all the events are sent. If multiple filters are set, the filters are cumulative. If a component UUID is included in Component ID to Include and excluded here, then the exclusion takes precedence and the event will not be sent.falsefalsetrueENVIRONMENTfalsefalses2s-prov-task-name-filterComponent Name to IncludeRegular expression to filter the provenance events based on the component name. Only the events matching the regular expression will be sent. If no filter is set, all the events are sent. If multiple filters are set, the filters are cumulative.falsefalsetrueENVIRONMENTfalsefalses2s-prov-task-name-filter-excludeComponent Name to ExcludeRegular expression to exclude the provenance events based on the component name. The events matching the regular expression will not be sent. If no filter is set, all the events are sent. If multiple filters are set, the filters are cumulative. If a component name is included in Component Name to Include and excluded here, then the exclusion takes precedence and the event will not be sent.falsefalsetrueENVIRONMENTfalsefalsestart-positionStart PositionIf the Reporting Task has never been run, or if its state has been reset by a user, specifies where in the stream of Provenance Events the Reporting Task should startbeginning-of-streamBeginning of Streambeginning-of-streamStart reading provenance Events from the beginning of the stream (the oldest event first)End of Streamend-of-streamStart reading provenance Events from the end of the stream, ignoring old eventstruefalsefalseNONEfalsefalseinclude-null-valuesInclude Null ValuesIndicate if null values should be included in records. Default will be falsefalsetruetruefalsefalsetruefalsefalseNONEfalsefalsePlatformPlatformThe value to use for the platform field in each event.nifitruefalsetrueENVIRONMENTfalsefalseInstance URLInstance URLThe URL of this instance to use in the Content URI of each event.http://${hostname(true)}:8080/nifitruefalsetrueENVIRONMENTfalsefalseBatch SizeBatch SizeSpecifies how many records to send in a single batch, at most.1000truefalsefalseNONEfalsefalseorg.apache.nifi.reporting.azure.loganalytics.AzureLogAnalyticsReportingTaskREPORTING_TASKSends JVM-metrics as well as Apache NiFi-metrics to a Azure Log Analytics workspace.Apache NiFi-metrics can be either configured global or on process-group level.azuremetricsreportinglog analyticsSend JVM MetricsSend JVM MetricsSend JVM Metrics in addition to the NiFi-metricsfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseLog Analytics Workspace IdLog Analytics Workspace IdLog Analytics Workspace IdtruetruetrueENVIRONMENTfalsefalseLog Analytics Custom Log NameLog Analytics Custom Log NameLog Analytics Custom Log NamenifimetricsfalsefalsetrueENVIRONMENTfalsefalseLog Analytics Workspace KeyLog Analytics Workspace KeyAzure Log Analytic Worskspace KeytruetruetrueENVIRONMENTfalsefalseApplication IDApplication IDThe Application ID to be included in the metrics sent to Azure Log Analytics WSnifitruefalsetrueENVIRONMENTfalsefalseInstance IDInstance IDId of this NiFi instance to be included in the metrics sent to Azure Log Analytics WS${hostname(true)}truefalsetrueENVIRONMENTfalsefalseProcess group ID(s)Process group ID(s)If specified, the reporting task will send metrics the configured ProcessGroup(s) only. Multiple IDs should be separated by a comma. If none of the group-IDs could be found or no IDs are defined, the Root Process Group is used and global metrics are sent.falsefalsetrueENVIRONMENTfalsefalseJob NameJob NameThe name of the exporting jobnifi_reporting_jobfalsefalsetrueENVIRONMENTfalsefalseLog Analytics URL Endpoint FormatLog Analytics URL Endpoint FormatLog Analytics URL Endpoint Formathttps://{0}.ods.opinsights.azure.com/api/logs?api-version=2016-04-01falsefalsetrueENVIRONMENTfalsefalseTIMER_DRIVEN1 min1org.apache.nifi.parameter.azure.AzureKeyVaultSecretsParameterProviderPARAMETER_PROVIDERFetches parameters from Azure Key Vault Secrets. Each secret becomes a Parameter, which map to a Parameter Group byadding a secret tag named 'group-name'.azurekeyvaultkeyvaultsecretsazure-credentials-serviceAzure Credentials ServiceController service used to obtain Azure credentials to be used with Key Vault client.org.apache.nifi.services.azure.AzureCredentialsServiceorg.apache.nifinifi-azure-services-api-nar2.2.0truefalsefalseNONEfalsefalsekey-vault-uriKey Vault URIVault URI of the Key Vault that contains the secretstruefalsefalseNONEfalsefalsegroup-name-patternGroup Name PatternA Regular Expression matching on the 'group-name' tag value that identifies Secrets whose parameters should be fetched. Any secrets without a 'group-name' tag value that matches this Regex will not be fetched..*truefalsefalseNONEfalsefalse