java.lang.Object
- com.pulumi.resources.Resource
- - com.pulumi.resources.CustomResource
  - - com.pulumi.aws.bedrock.AgentcoreEvaluator

public class AgentcoreEvaluator
extends com.pulumi.resources.CustomResource

Manages an AWS Bedrock AgentCore Evaluator. An evaluator scores how an agent performs. You can configure it in one of two ways: an LLM-as-a-Judge evaluator that uses a model to score agent behavior against your instructions and a rating scale, or a code-based evaluator that runs a Lambda function you provide. ## Example Usage ### LLM-as-a-Judge with Numerical Rating Scale

 
 package generated_program;
 
 import com.pulumi.Context;
 import com.pulumi.Pulumi;
 import com.pulumi.core.Output;
 import com.pulumi.aws.bedrock.AgentcoreEvaluator;
 import com.pulumi.aws.bedrock.AgentcoreEvaluatorArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeRatingScaleArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeRatingScaleNumericalArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeModelConfigArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeModelConfigBedrockEvaluatorModelConfigArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeModelConfigBedrockEvaluatorModelConfigInferenceConfigArgs;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Map;
 import java.io.File;
 import java.nio.file.Files;
 import java.nio.file.Paths;
 
 public class App {
     public static void main(String[] args) {
         Pulumi.run(App::stack);
     }
 
     public static void stack(Context ctx) {
         var example = new AgentcoreEvaluator("example", AgentcoreEvaluatorArgs.builder()
             .evaluatorConfig(AgentcoreEvaluatorEvaluatorConfigArgs.builder()
                 .llmAsAJudge(AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeArgs.builder()
                     .ratingScale(AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeRatingScaleArgs.builder()
                         .numericals(                        
                             AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeRatingScaleNumericalArgs.builder()
                                 .definition("Not helpful at all.")
                                 .value(1.0)
                                 .label("1")
                                 .build(),
                             AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeRatingScaleNumericalArgs.builder()
                                 .definition("Extremely helpful.")
                                 .value(5.0)
                                 .label("5")
                                 .build())
                         .build())
                     .modelConfig(AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeModelConfigArgs.builder()
                         .bedrockEvaluatorModelConfig(AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeModelConfigBedrockEvaluatorModelConfigArgs.builder()
                             .inferenceConfig(AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeModelConfigBedrockEvaluatorModelConfigInferenceConfigArgs.builder()
                                 .maxTokens(1024)
                                 .temperature(0.0)
                                 .topP(1.0)
                                 .build())
                             .modelId("us.amazon.nova-2-lite-v1:0")
                             .build())
                         .build())
                     .instructions("Given the {context} and the {assistant_turn}, compare against {expected_response} and rate from 1 to 5.")
                     .build())
                 .build())
             .evaluatorName("helpfulness_evaluator")
             .description("Rates assistant helpfulness from 1 to 5")
             .level("TRACE")
             .build());
 
     }
 }

### LLM-as-a-Judge with Categorical Rating Scale

 
 package generated_program;
 
 import com.pulumi.Context;
 import com.pulumi.Pulumi;
 import com.pulumi.core.Output;
 import com.pulumi.aws.bedrock.AgentcoreEvaluator;
 import com.pulumi.aws.bedrock.AgentcoreEvaluatorArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeRatingScaleArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeRatingScaleCategoricalArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeModelConfigArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeModelConfigBedrockEvaluatorModelConfigArgs;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Map;
 import java.io.File;
 import java.nio.file.Files;
 import java.nio.file.Paths;
 
 public class App {
     public static void main(String[] args) {
         Pulumi.run(App::stack);
     }
 
     public static void stack(Context ctx) {
         var example = new AgentcoreEvaluator("example", AgentcoreEvaluatorArgs.builder()
             .evaluatorConfig(AgentcoreEvaluatorEvaluatorConfigArgs.builder()
                 .llmAsAJudge(AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeArgs.builder()
                     .ratingScale(AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeRatingScaleArgs.builder()
                         .categoricals(                        
                             AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeRatingScaleCategoricalArgs.builder()
                                 .definition("Friendly, helpful tone.")
                                 .label("POSITIVE")
                                 .build(),
                             AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeRatingScaleCategoricalArgs.builder()
                                 .definition("Neutral or terse tone.")
                                 .label("NEUTRAL")
                                 .build(),
                             AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeRatingScaleCategoricalArgs.builder()
                                 .definition("Unhelpful or dismissive tone.")
                                 .label("NEGATIVE")
                                 .build())
                         .build())
                     .modelConfig(AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeModelConfigArgs.builder()
                         .bedrockEvaluatorModelConfig(AgentcoreEvaluatorEvaluatorConfigLlmAsAJudgeModelConfigBedrockEvaluatorModelConfigArgs.builder()
                             .modelId("us.amazon.nova-2-lite-v1:0")
                             .build())
                         .build())
                     .instructions("Classify the tone of the {assistant_turn} given the {context}.")
                     .build())
                 .build())
             .evaluatorName("tone_evaluator")
             .level("SESSION")
             .build());
 
     }
 }

### Code-based Evaluator (Lambda)

 
 package generated_program;
 
 import com.pulumi.Context;
 import com.pulumi.Pulumi;
 import com.pulumi.core.Output;
 import com.pulumi.aws.bedrock.AgentcoreEvaluator;
 import com.pulumi.aws.bedrock.AgentcoreEvaluatorArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigCodeBasedArgs;
 import com.pulumi.aws.bedrock.inputs.AgentcoreEvaluatorEvaluatorConfigCodeBasedLambdaConfigArgs;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Map;
 import java.io.File;
 import java.nio.file.Files;
 import java.nio.file.Paths;
 
 public class App {
     public static void main(String[] args) {
         Pulumi.run(App::stack);
     }
 
     public static void stack(Context ctx) {
         var example = new AgentcoreEvaluator("example", AgentcoreEvaluatorArgs.builder()
             .evaluatorConfig(AgentcoreEvaluatorEvaluatorConfigArgs.builder()
                 .codeBased(AgentcoreEvaluatorEvaluatorConfigCodeBasedArgs.builder()
                     .lambdaConfig(AgentcoreEvaluatorEvaluatorConfigCodeBasedLambdaConfigArgs.builder()
                         .lambdaArn(exampleAwsLambdaFunction.arn())
                         .lambdaTimeoutInSeconds(60)
                         .build())
                     .build())
                 .build())
             .evaluatorName("lambda_evaluator")
             .level("TOOL_CALL")
             .build());
 
     }
 }

## Import ### Identity Schema #### Required * `evaluatorId` (String) Unique identifier of the evaluator. #### Optional * `accountId` (String) AWS Account where this resource is managed. * `region` (String) Region where this resource is managed. Using `pulumi import`, import Bedrock AgentCore Evaluator using the evaluator ID. For example: ```sh $ pulumi import aws:bedrock/agentcoreEvaluator:AgentcoreEvaluator example helpfulness_evaluator-abc1234567 ```

Nested Class Summary
- Nested classes/interfaces inherited from class com.pulumi.resources.CustomResource
  com.pulumi.resources.CustomResource.CustomResourceInternal
- Nested classes/interfaces inherited from class com.pulumi.resources.Resource
  com.pulumi.resources.Resource.LazyField<T extends java.lang.Object>, com.pulumi.resources.Resource.LazyFields, com.pulumi.resources.Resource.ResourceInternal

Field Summary
- Fields inherited from class com.pulumi.resources.Resource
  childResources, remote

Constructor Summary

Constructors
Constructor	Description
`AgentcoreEvaluator(java.lang.String name)`
`AgentcoreEvaluator(java.lang.String name, AgentcoreEvaluatorArgs args)`
`AgentcoreEvaluator(java.lang.String name, AgentcoreEvaluatorArgs args, com.pulumi.resources.CustomResourceOptions options)`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`com.pulumi.core.Output<java.lang.String>`	`createdAt()`
`com.pulumi.core.Output<java.util.Optional<java.lang.String>>`	`description()`
`com.pulumi.core.Output<java.lang.String>`	`evaluatorArn()`
`com.pulumi.core.Output<AgentcoreEvaluatorEvaluatorConfig>`	`evaluatorConfig()`
`com.pulumi.core.Output<java.lang.String>`	`evaluatorId()`
`com.pulumi.core.Output<java.lang.String>`	`evaluatorName()`
`static AgentcoreEvaluator`	`get(java.lang.String name, com.pulumi.core.Output<java.lang.String> id, AgentcoreEvaluatorState state, com.pulumi.resources.CustomResourceOptions options)`	Get an existing Host resource's state with the given name, ID, and optional extra properties used to qualify the lookup.
`com.pulumi.core.Output<java.util.Optional<java.lang.String>>`	`kmsKeyArn()`
`com.pulumi.core.Output<java.lang.String>`	`level()`
`com.pulumi.core.Output<java.lang.Boolean>`	`lockedForModification()`
`com.pulumi.core.Output<java.lang.String>`	`region()`
`com.pulumi.core.Output<java.lang.String>`	`status()`
`com.pulumi.core.Output<java.util.Optional<java.util.Map<java.lang.String,java.lang.String>>>`	`tags()`
`com.pulumi.core.Output<java.util.Map<java.lang.String,java.lang.String>>`	`tagsAll()`
`com.pulumi.core.Output<java.util.Optional<AgentcoreEvaluatorTimeouts>>`	`timeouts()`

Methods inherited from class com.pulumi.resources.CustomResource
getId, id, idFuture

Methods inherited from class com.pulumi.resources.Resource
getChildResources, getResourceName, getResourceType, getUrn, pulumiChildResources, pulumiResourceName, pulumiResourceType, urn

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - AgentcoreEvaluator
```
public AgentcoreEvaluator(java.lang.String name)
```
    Parameters:
    
    name - The _unique_ name of the resulting resource.
  - AgentcoreEvaluator
```
public AgentcoreEvaluator(java.lang.String name,
                          AgentcoreEvaluatorArgs args)
```
    Parameters:
    
    name - The _unique_ name of the resulting resource.
    
    args - The arguments to use to populate this resource's properties.
  - AgentcoreEvaluator
```
public AgentcoreEvaluator(java.lang.String name,
                          AgentcoreEvaluatorArgs args,
                          @Nullable
                          com.pulumi.resources.CustomResourceOptions options)
```
    Parameters:
    
    name - The _unique_ name of the resulting resource.
    
    args - The arguments to use to populate this resource's properties.
    
    options - A bag of options that control this resource's behavior.
- Method Detail
  - createdAt
```
public com.pulumi.core.Output<java.lang.String> createdAt()
```
    Returns:
    
    Timestamp when the evaluator was created.
  - description
```
public com.pulumi.core.Output<java.util.Optional<java.lang.String>> description()
```
    Returns:
    
    Description of the evaluator. Length 1–200.
  - evaluatorArn
```
public com.pulumi.core.Output<java.lang.String> evaluatorArn()
```
    Returns:
    
    ARN of the evaluator.
  - evaluatorConfig
```
public com.pulumi.core.Output<AgentcoreEvaluatorEvaluatorConfig> evaluatorConfig()
```
    Returns:
    
    Configuration that defines how the evaluator assesses agent performance. See `evaluatorConfig` below.
  - evaluatorId
```
public com.pulumi.core.Output<java.lang.String> evaluatorId()
```
    Returns:
    
    Unique identifier of the evaluator.
  - evaluatorName
```
public com.pulumi.core.Output<java.lang.String> evaluatorName()
```
    Returns:
    
    Name of the evaluator. Must match the pattern `^[a-zA-Z][a-zA-Z0-9_]{0,47}$`.
  - kmsKeyArn
```
public com.pulumi.core.Output<java.util.Optional<java.lang.String>> kmsKeyArn()
```
    Returns:
    
    ARN of a customer-managed KMS key used to encrypt the evaluator's sensitive data. Only symmetric encryption keys are supported.
  - level
```
public com.pulumi.core.Output<java.lang.String> level()
```
    Returns:
    
    Evaluation level that determines the scope of evaluation. Valid values: `TOOL_CALL`, `TRACE`, `SESSION`. The following arguments are optional:
  - lockedForModification
```
public com.pulumi.core.Output<java.lang.Boolean> lockedForModification()
```
    Returns:
    
    Whether the evaluator is locked because it is in use by an active online evaluation.
  - region
```
public com.pulumi.core.Output<java.lang.String> region()
```
    Returns:
    
    Region where this resource will be [managed](https://docs.aws.amazon.com/general/latest/gr/rande.html#regional-endpoints). Defaults to the Region set in the provider configuration.
  - status
```
public com.pulumi.core.Output<java.lang.String> status()
```
    Returns:
    
    Current status of the evaluator.
  - tags
```
public com.pulumi.core.Output<java.util.Optional<java.util.Map<java.lang.String,java.lang.String>>> tags()
```
    Returns:
    
    Map of tags assigned to the resource. If configured with a provider `defaultTags` configuration block present, tags with matching keys will overwrite those defined at the provider-level.
  - tagsAll
```
public com.pulumi.core.Output<java.util.Map<java.lang.String,java.lang.String>> tagsAll()
```
    Returns:
    
    Map of tags assigned to the resource, including those inherited from the provider `defaultTags` configuration block.
  - timeouts
```
public com.pulumi.core.Output<java.util.Optional<AgentcoreEvaluatorTimeouts>> timeouts()
```
  - get
```
public static AgentcoreEvaluator get(java.lang.String name,
                                     com.pulumi.core.Output<java.lang.String> id,
                                     @Nullable
                                     AgentcoreEvaluatorState state,
                                     @Nullable
                                     com.pulumi.resources.CustomResourceOptions options)
```
    Get an existing Host resource's state with the given name, ID, and optional extra properties used to qualify the lookup.
    
    Parameters:
    
    name - The _unique_ name of the resulting resource.
    
    id - The _unique_ provider ID of the resource to lookup.
    
    state -
    
    options - Optional settings to control the behavior of the CustomResource.

Class AgentcoreEvaluator

Nested Class Summary

Nested classes/interfaces inherited from class com.pulumi.resources.CustomResource

Nested classes/interfaces inherited from class com.pulumi.resources.Resource

Field Summary

Fields inherited from class com.pulumi.resources.Resource

Constructor Summary

Method Summary

Methods inherited from class com.pulumi.resources.CustomResource

Methods inherited from class com.pulumi.resources.Resource

Methods inherited from class java.lang.Object

Constructor Detail

AgentcoreEvaluator

AgentcoreEvaluator

AgentcoreEvaluator

Method Detail

createdAt

description

evaluatorArn

evaluatorConfig

evaluatorId

evaluatorName

kmsKeyArn

level

lockedForModification

region

status

tags

tagsAll

timeouts

get