Package com.pulumi.aws.bedrock

Class EvaluationJob

  • public class EvaluationJob
extends com.pulumi.resources.CustomResource
    Manages an Amazon Bedrock evaluation job. An evaluation job assesses model or knowledge base performance using either automated metrics or human workers. > **Note:** Amazon Bedrock does not support permanently deleting an evaluation job. Destroying this resource stops the job (if it is still running) using the [StopEvaluationJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_StopEvaluationJob.html) API, then removes it from Terraform state. Set `skipDestroy` to leave the job in its current state instead. ## Example Usage ### Automated Model Evaluation 
     
 package generated_program;
 
 import com.pulumi.Context;
 import com.pulumi.Pulumi;
 import com.pulumi.core.Output;
 import com.pulumi.aws.bedrock.EvaluationJob;
 import com.pulumi.aws.bedrock.EvaluationJobArgs;
 import com.pulumi.aws.bedrock.inputs.EvaluationJobEvaluationConfigArgs;
 import com.pulumi.aws.bedrock.inputs.EvaluationJobEvaluationConfigAutomatedArgs;
 import com.pulumi.aws.bedrock.inputs.EvaluationJobEvaluationConfigAutomatedDatasetMetricConfigArgs;
 import com.pulumi.aws.bedrock.inputs.EvaluationJobEvaluationConfigAutomatedDatasetMetricConfigDatasetArgs;
 import com.pulumi.aws.bedrock.inputs.EvaluationJobInferenceConfigArgs;
 import com.pulumi.aws.bedrock.inputs.EvaluationJobInferenceConfigModelArgs;
 import com.pulumi.aws.bedrock.inputs.EvaluationJobInferenceConfigModelBedrockModelArgs;
 import com.pulumi.aws.bedrock.inputs.EvaluationJobOutputDataConfigArgs;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Map;
 import java.io.File;
 import java.nio.file.Files;
 import java.nio.file.Paths;
 
 public class App {
     public static void main(String[] args) {
         Pulumi.run(App::stack);
     }
 
     public static void stack(Context ctx) {
         var example = new EvaluationJob("example", EvaluationJobArgs.builder()
             .evaluationConfig(EvaluationJobEvaluationConfigArgs.builder()
                 .automated(EvaluationJobEvaluationConfigAutomatedArgs.builder()
                     .datasetMetricConfigs(EvaluationJobEvaluationConfigAutomatedDatasetMetricConfigArgs.builder()
                         .dataset(EvaluationJobEvaluationConfigAutomatedDatasetMetricConfigDatasetArgs.builder()
                             .name("Builtin.Bold")
                             .build())
                         .taskType("Generation")
                         .metricNames("Builtin.Robustness")
                         .build())
                     .build())
                 .build())
             .inferenceConfig(EvaluationJobInferenceConfigArgs.builder()
                 .models(EvaluationJobInferenceConfigModelArgs.builder()
                     .bedrockModel(EvaluationJobInferenceConfigModelBedrockModelArgs.builder()
                         .modelIdentifier("amazon.nova-micro-v1:0")
                         .build())
                     .build())
                 .build())
             .outputDataConfig(EvaluationJobOutputDataConfigArgs.builder()
                 .s3Uri(String.format("s3://%s/output/", exampleAwsS3Bucket.id()))
                 .build())
             .jobName("example-job")
             .roleArn(exampleAwsIamRole.arn())
             .build());
 
     }
 }
    ## Import ### Identity Schema #### Required * `jobArn` (String) ARN of the evaluation job. Using `pulumi import`, import Bedrock Evaluation Job using the `jobArn`. For example: ```sh $ pulumi import aws:bedrock/evaluationJob:EvaluationJob example arn:aws:bedrock:us-west-2:123456789012:evaluation-job/abcdefgh1234 ```

    • Nested Class Summary

      • Nested classes/interfaces inherited from class com.pulumi.resources.CustomResource

        com.pulumi.resources.CustomResource.CustomResourceInternal

      • Nested classes/interfaces inherited from class com.pulumi.resources.Resource

        com.pulumi.resources.Resource.LazyField<T extends java.lang.Object>, com.pulumi.resources.Resource.LazyFields, com.pulumi.resources.Resource.ResourceInternal

    • Field Summary

      • Fields inherited from class com.pulumi.resources.Resource

        childResources, remote

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      com.pulumi.core.Output<java.lang.String> applicationType()  
      com.pulumi.core.Output<java.lang.String> createdAt()  
      com.pulumi.core.Output<java.util.Optional<java.lang.String>> customerEncryptionKeyId()  
      com.pulumi.core.Output<EvaluationJobEvaluationConfig> evaluationConfig()  
      com.pulumi.core.Output<java.util.List<java.lang.String>> failureMessages()  
      static EvaluationJob get​(java.lang.String name, com.pulumi.core.Output<java.lang.String> id, EvaluationJobState state, com.pulumi.resources.CustomResourceOptions options)
      Get an existing Host resource's state with the given name, ID, and optional extra properties used to qualify the lookup.
      com.pulumi.core.Output<EvaluationJobInferenceConfig> inferenceConfig()  
      com.pulumi.core.Output<java.lang.String> jobArn()  
      com.pulumi.core.Output<java.util.Optional<java.lang.String>> jobDescription()  
      com.pulumi.core.Output<java.lang.String> jobName()  
      com.pulumi.core.Output<java.lang.String> jobType()  
      com.pulumi.core.Output<java.lang.String> lastModifiedTime()  
      com.pulumi.core.Output<EvaluationJobOutputDataConfig> outputDataConfig()  
      com.pulumi.core.Output<java.lang.String> region()  
      com.pulumi.core.Output<java.lang.String> roleArn()  
      com.pulumi.core.Output<java.util.Optional<java.lang.Boolean>> skipDestroy()  
      com.pulumi.core.Output<java.lang.String> status()  
      com.pulumi.core.Output<java.util.Optional<java.util.Map<java.lang.String,​java.lang.String>>> tags()  
      com.pulumi.core.Output<java.util.Map<java.lang.String,​java.lang.String>> tagsAll()  
      com.pulumi.core.Output<java.util.Optional<EvaluationJobTimeouts>> timeouts()  

      • Methods inherited from class com.pulumi.resources.CustomResource

        getId, id, idFuture

      • Methods inherited from class com.pulumi.resources.Resource

        getChildResources, getResourceName, getResourceType, getUrn, pulumiChildResources, pulumiResourceName, pulumiResourceType, urn

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • EvaluationJob

        public EvaluationJob​(java.lang.String name)
        Parameters:
        name - The _unique_ name of the resulting resource.

      • EvaluationJob

        public EvaluationJob​(java.lang.String name,
                     EvaluationJobArgs args)
        Parameters:
        name - The _unique_ name of the resulting resource.
        args - The arguments to use to populate this resource's properties.

      • EvaluationJob

        public EvaluationJob​(java.lang.String name,
                     EvaluationJobArgs args,
                     @Nullable
                     com.pulumi.resources.CustomResourceOptions options)
        Parameters:
        name - The _unique_ name of the resulting resource.
        args - The arguments to use to populate this resource's properties.
        options - A bag of options that control this resource's behavior.

    • Method Detail

      • applicationType

        public com.pulumi.core.Output<java.lang.String> applicationType()
        Returns:
        Whether the evaluation job evaluates a model or a knowledge base. Valid values: `ModelEvaluation`, `RagEvaluation`.

      • createdAt

        public com.pulumi.core.Output<java.lang.String> createdAt()
        Returns:
        Date and time the evaluation job was created.

      • customerEncryptionKeyId

        public com.pulumi.core.Output<java.util.Optional<java.lang.String>> customerEncryptionKeyId()
        Returns:
        ARN of the customer managed KMS key to use to encrypt the evaluation job.

      • evaluationConfig

        public com.pulumi.core.Output<EvaluationJobEvaluationConfig> evaluationConfig()
        Returns:
        Configuration for either an automated or human-based evaluation job. See `evaluationConfig` Block below.

      • failureMessages

        public com.pulumi.core.Output<java.util.List<java.lang.String>> failureMessages()
        Returns:
        List of reasons the evaluation job failed to create, if applicable.

      • inferenceConfig

        public com.pulumi.core.Output<EvaluationJobInferenceConfig> inferenceConfig()
        Returns:
        Configuration for the inference model, or models, used for the evaluation job. See `inferenceConfig` Block below.

      • jobArn

        public com.pulumi.core.Output<java.lang.String> jobArn()
        Returns:
        ARN of the evaluation job.

      • jobDescription

        public com.pulumi.core.Output<java.util.Optional<java.lang.String>> jobDescription()
        Returns:
        Description of the evaluation job.

      • jobName

        public com.pulumi.core.Output<java.lang.String> jobName()
        Returns:
        Name for the evaluation job. Must be unique within your AWS account and Region, and consist of lowercase letters, numbers, and hyphens.

      • jobType

        public com.pulumi.core.Output<java.lang.String> jobType()
        Returns:
        Whether the evaluation job is automated or human-based.

      • lastModifiedTime

        public com.pulumi.core.Output<java.lang.String> lastModifiedTime()
        Returns:
        Date and time the evaluation job was last modified.

      • outputDataConfig

        public com.pulumi.core.Output<EvaluationJobOutputDataConfig> outputDataConfig()
        Returns:
        Configuration for the Amazon S3 location where the results of the evaluation job are stored. See `outputDataConfig` Block below.

      • region

        public com.pulumi.core.Output<java.lang.String> region()
        Returns:
        Region where this resource will be [managed](https://docs.aws.amazon.com/general/latest/gr/rande.html#regional-endpoints). Defaults to the Region set in the provider configuration.

      • roleArn

        public com.pulumi.core.Output<java.lang.String> roleArn()
        Returns:
        ARN of an IAM service role that Amazon Bedrock can assume to perform tasks on your behalf. See [Required permissions for model evaluations](https://docs.aws.amazon.com/bedrock/latest/userguide/model-evaluation-security.html). The following arguments are optional:

      • skipDestroy

        public com.pulumi.core.Output<java.util.Optional<java.lang.Boolean>> skipDestroy()
        Returns:
        Whether to leave the evaluation job in its current state when destroying the resource, instead of stopping it.

      • status

        public com.pulumi.core.Output<java.lang.String> status()
        Returns:
        Current status of the evaluation job.

      • tags

        public com.pulumi.core.Output<java.util.Optional<java.util.Map<java.lang.String,​java.lang.String>>> tags()
        Returns:
        Map of tags to assign to the evaluation job. If configured with a provider `defaultTags` configuration block present, tags with matching keys will overwrite those defined at the provider-level.

      • tagsAll

        public com.pulumi.core.Output<java.util.Map<java.lang.String,​java.lang.String>> tagsAll()
        Returns:
        Map of tags assigned to the resource, including those inherited from the provider `defaultTags` configuration block.

      • get

        public static EvaluationJob get​(java.lang.String name,
                                com.pulumi.core.Output<java.lang.String> id,
                                @Nullable
                                EvaluationJobState state,
                                @Nullable
                                com.pulumi.resources.CustomResourceOptions options)
        Get an existing Host resource's state with the given name, ID, and optional extra properties used to qualify the lookup.
        Parameters:
        name - The _unique_ name of the resulting resource.
        id - The _unique_ provider ID of the resource to lookup.
        state -
        options - Optional settings to control the behavior of the CustomResource.