Package com.pulumi.aws.msk
Class Channel
- java.lang.Object
-
- com.pulumi.resources.Resource
-
- com.pulumi.resources.CustomResource
-
- com.pulumi.aws.msk.Channel
-
public class Channel extends com.pulumi.resources.CustomResourceManages an Amazon Managed Streaming for Apache Kafka (MSK) Channel. With [Amazon MSK Data Delivery](https://docs.aws.amazon.com/msk/latest/developerguide/msk-data-delivery.html), a channel delivers Apache Kafka data from an Amazon MSK Provisioned cluster that uses Express brokers directly to Amazon S3, without connectors or additional infrastructure to manage. A channel reads records from a Kafka topic and delivers them to one of two destinations: * **Amazon S3 general purpose buckets** — records are written in their source format (`JSON`, `ByteArray`, or `String`) as objects, for use cases such as log archival, compliance retention, and Kafka replay. * **Apache Iceberg tables on Amazon S3 Tables** — `JSON` records, validated against an AWS Glue Schema Registry schema, are materialized as Apache Iceberg tables. Records that cannot be processed are routed to a required dead-letter queue (DLQ) Amazon S3 bucket. Channels are supported only on Amazon MSK Provisioned clusters that use Express brokers; Standard brokers and Amazon MSK Serverless are not supported. > **Note:** A channel does not backfill previously produced data; only data produced after creation is delivered. For the Iceberg destination, a channel creates a new Iceberg table for each configuration. ## Example Usage ### Amazon S3 Destination
### Apache Iceberg Destinationpackage generated_program; import com.pulumi.Context; import com.pulumi.Pulumi; import com.pulumi.core.Output; import com.pulumi.aws.msk.Channel; import com.pulumi.aws.msk.ChannelArgs; import com.pulumi.aws.msk.inputs.ChannelTopicConfigurationArgs; import com.pulumi.aws.msk.inputs.ChannelTopicConfigurationRecordConverterArgs; import com.pulumi.aws.msk.inputs.ChannelS3DestinationArgs; import com.pulumi.aws.msk.inputs.ChannelS3DestinationDeadLetterQueueS3Args; import com.pulumi.aws.msk.inputs.ChannelS3DestinationStorageArgs; import java.util.ArrayList; import java.util.Arrays; import java.util.Map; import java.io.File; import java.nio.file.Files; import java.nio.file.Paths; public class App { public static void main(String[] args) { Pulumi.run(App::stack); } public static void stack(Context ctx) { var example = new Channel("example", ChannelArgs.builder() .topicConfiguration(ChannelTopicConfigurationArgs.builder() .recordConverter(ChannelTopicConfigurationRecordConverterArgs.builder() .valueConverter("BYTE_ARRAY") .build()) .topicArn(exampleAwsMskTopic.arn()) .build()) .s3Destination(ChannelS3DestinationArgs.builder() .deadLetterQueueS3(ChannelS3DestinationDeadLetterQueueS3Args.builder() .bucketArn(dlq.arn()) .build()) .storage(ChannelS3DestinationStorageArgs.builder() .bucketArn(exampleAwsS3Bucket.arn()) .compressionType("NONE") .storageClass("STANDARD") .build()) .serviceExecutionRoleArn(exampleAwsIamRole.arn()) .build()) .channelName("example") .clusterArn(exampleAwsMskCluster.arn()) .build()); } }
## Import ### Identity Schema #### Required * `arn` (String) ARN of the channel. * `clusterArn` (String) ARN that uniquely identifies the cluster. #### Optional * `accountId` (String) AWS Account where this resource is managed. * `region` (String) Region where this resource is managed. Using `pulumi import`, import Managed Streaming for Kafka Channel using the `arn` and `clusterArn`. For example: ```sh $ pulumi import aws:msk/channel:Channel example arn:aws:kafka:us-west-2:123456789012:channel/example/279c0212-d057-4dba-9aa9-1c4e5a25bfc7-3/a1b2c3d4,arn:aws:kafka:us-west-2:123456789012:cluster/example/279c0212-d057-4dba-9aa9-1c4e5a25bfc7-3 ```package generated_program; import com.pulumi.Context; import com.pulumi.Pulumi; import com.pulumi.core.Output; import com.pulumi.aws.msk.Channel; import com.pulumi.aws.msk.ChannelArgs; import com.pulumi.aws.msk.inputs.ChannelTopicConfigurationArgs; import com.pulumi.aws.msk.inputs.ChannelTopicConfigurationRecordConverterArgs; import com.pulumi.aws.msk.inputs.ChannelTopicConfigurationRecordSchemaArgs; import com.pulumi.aws.msk.inputs.ChannelIcebergDestinationArgs; import com.pulumi.aws.msk.inputs.ChannelIcebergDestinationCatalogArgs; import com.pulumi.aws.msk.inputs.ChannelIcebergDestinationDeadLetterQueueS3Args; import com.pulumi.aws.msk.inputs.ChannelIcebergDestinationDestinationTableArgs; import com.pulumi.aws.msk.inputs.ChannelIcebergDestinationDestinationTablePartitionSpecArgs; import com.pulumi.aws.msk.inputs.ChannelIcebergDestinationDestinationTablePartitionSpecSourceArgs; import com.pulumi.aws.msk.inputs.ChannelIcebergDestinationSchemaEvolutionArgs; import com.pulumi.aws.msk.inputs.ChannelIcebergDestinationTableCreationArgs; import java.util.ArrayList; import java.util.Arrays; import java.util.Map; import java.io.File; import java.nio.file.Files; import java.nio.file.Paths; public class App { public static void main(String[] args) { Pulumi.run(App::stack); } public static void stack(Context ctx) { var example = new Channel("example", ChannelArgs.builder() .topicConfiguration(ChannelTopicConfigurationArgs.builder() .recordConverter(ChannelTopicConfigurationRecordConverterArgs.builder() .valueConverter("JSON") .build()) .recordSchema(ChannelTopicConfigurationRecordSchemaArgs.builder() .gsrArn(exampleAwsGlueSchema.arn()) .build()) .topicArn(exampleAwsMskTopic.arn()) .build()) .icebergDestination(ChannelIcebergDestinationArgs.builder() .catalog(ChannelIcebergDestinationCatalogArgs.builder() .warehouseLocation(exampleAwsS3tablesTableBucket.arn()) .build()) .deadLetterQueueS3(ChannelIcebergDestinationDeadLetterQueueS3Args.builder() .bucketArn(dlq.arn()) .build()) .destinationTable(ChannelIcebergDestinationDestinationTableArgs.builder() .partitionSpec(ChannelIcebergDestinationDestinationTablePartitionSpecArgs.builder() .sources(ChannelIcebergDestinationDestinationTablePartitionSpecSourceArgs.builder() .sourceName("event_time") .build()) .partitionStrategy("TIME_HOUR") .build()) .destinationDatabaseName("example_namespace") .destinationTableName("example_table") .build()) .schemaEvolution(ChannelIcebergDestinationSchemaEvolutionArgs.builder() .enableSchemaEvolution(false) .build()) .tableCreation(ChannelIcebergDestinationTableCreationArgs.builder() .enableTableCreation(true) .build()) .appendOnly(true) .serviceExecutionRoleArn(exampleAwsIamRole.arn()) .build()) .channelName("example") .clusterArn(exampleAwsMskCluster.arn()) .build()); } }
-
-
Constructor Summary
Constructors Constructor Description Channel(java.lang.String name)Channel(java.lang.String name, ChannelArgs args)Channel(java.lang.String name, ChannelArgs args, com.pulumi.resources.CustomResourceOptions options)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description com.pulumi.core.Output<java.lang.String>arn()com.pulumi.core.Output<java.lang.String>channelName()com.pulumi.core.Output<java.lang.String>clusterArn()com.pulumi.core.Output<java.lang.String>destinationType()com.pulumi.core.Output<java.util.Optional<ChannelEncryptionConfiguration>>encryptionConfiguration()static Channelget(java.lang.String name, com.pulumi.core.Output<java.lang.String> id, ChannelState state, com.pulumi.resources.CustomResourceOptions options)Get an existing Host resource's state with the given name, ID, and optional extra properties used to qualify the lookup.com.pulumi.core.Output<java.util.Optional<ChannelIcebergDestination>>icebergDestination()com.pulumi.core.Output<java.util.Optional<ChannelLoggingInfo>>loggingInfo()com.pulumi.core.Output<java.lang.String>region()com.pulumi.core.Output<java.util.Optional<ChannelS3Destination>>s3Destination()com.pulumi.core.Output<java.util.Optional<java.util.Map<java.lang.String,java.lang.String>>>tags()com.pulumi.core.Output<java.util.Map<java.lang.String,java.lang.String>>tagsAll()com.pulumi.core.Output<java.util.Optional<ChannelTimeouts>>timeouts()com.pulumi.core.Output<ChannelTopicConfiguration>topicConfiguration()
-
-
-
Constructor Detail
-
Channel
public Channel(java.lang.String name)
- Parameters:
name- The _unique_ name of the resulting resource.
-
Channel
public Channel(java.lang.String name, ChannelArgs args)- Parameters:
name- The _unique_ name of the resulting resource.args- The arguments to use to populate this resource's properties.
-
Channel
public Channel(java.lang.String name, ChannelArgs args, @Nullable com.pulumi.resources.CustomResourceOptions options)- Parameters:
name- The _unique_ name of the resulting resource.args- The arguments to use to populate this resource's properties.options- A bag of options that control this resource's behavior.
-
-
Method Detail
-
arn
public com.pulumi.core.Output<java.lang.String> arn()
- Returns:
- ARN of the channel.
-
channelName
public com.pulumi.core.Output<java.lang.String> channelName()
- Returns:
- Name of the channel. Must be unique within the cluster. Changing this forces a new resource to be created.
-
clusterArn
public com.pulumi.core.Output<java.lang.String> clusterArn()
- Returns:
- ARN that uniquely identifies the cluster. Changing this forces a new resource to be created.
-
destinationType
public com.pulumi.core.Output<java.lang.String> destinationType()
- Returns:
- Type of destination configured for the channel.
-
encryptionConfiguration
public com.pulumi.core.Output<java.util.Optional<ChannelEncryptionConfiguration>> encryptionConfiguration()
- Returns:
- AWS KMS encryption configuration applied to data at rest. Changing this forces a new resource to be created. See `encryptionConfiguration` Block below.
-
icebergDestination
public com.pulumi.core.Output<java.util.Optional<ChannelIcebergDestination>> icebergDestination()
- Returns:
- Apache Iceberg destination for the channel. Exactly one of `icebergDestination` or `s3Destination` is required. With the exception of `dataFreshnessInSeconds`, changing an argument in this block forces a new resource to be created. See `icebergDestination` Block below.
-
loggingInfo
public com.pulumi.core.Output<java.util.Optional<ChannelLoggingInfo>> loggingInfo()
- Returns:
- Destinations to which the channel publishes operational logs. Changing this forces a new resource to be created. See `loggingInfo` Block below.
-
region
public com.pulumi.core.Output<java.lang.String> region()
- Returns:
- Region where this resource will be [managed](https://docs.aws.amazon.com/general/latest/gr/rande.html#regional-endpoints). Defaults to the Region set in the provider configuration.
-
s3Destination
public com.pulumi.core.Output<java.util.Optional<ChannelS3Destination>> s3Destination()
- Returns:
- Amazon S3 destination for the channel. Exactly one of `icebergDestination` or `s3Destination` is required. With the exception of `dataFreshnessInSeconds`, changing an argument in this block forces a new resource to be created. See `s3Destination` Block below.
-
tags
public com.pulumi.core.Output<java.util.Optional<java.util.Map<java.lang.String,java.lang.String>>> tags()
- Returns:
- Map of tags assigned to the resource. If configured with a provider `defaultTags` configuration block present, tags with matching keys will overwrite those defined at the provider-level.
-
tagsAll
public com.pulumi.core.Output<java.util.Map<java.lang.String,java.lang.String>> tagsAll()
- Returns:
- Map of tags assigned to the resource, including those inherited from the provider `defaultTags` configuration block.
-
timeouts
public com.pulumi.core.Output<java.util.Optional<ChannelTimeouts>> timeouts()
-
topicConfiguration
public com.pulumi.core.Output<ChannelTopicConfiguration> topicConfiguration()
- Returns:
- Configuration of the Apache Kafka topic that feeds the channel. Changing this forces a new resource to be created. See `topicConfiguration` Block below. The following arguments are optional:
-
get
public static Channel get(java.lang.String name, com.pulumi.core.Output<java.lang.String> id, @Nullable ChannelState state, @Nullable com.pulumi.resources.CustomResourceOptions options)
Get an existing Host resource's state with the given name, ID, and optional extra properties used to qualify the lookup.- Parameters:
name- The _unique_ name of the resulting resource.id- The _unique_ provider ID of the resource to lookup.state-options- Optional settings to control the behavior of the CustomResource.
-
-