Class ModelContainerMultiModelConfig


  • public final class ModelContainerMultiModelConfig
    extends java.lang.Object
    • Method Detail

      • modelCacheSetting

        public java.util.Optional<java.lang.String> modelCacheSetting()
        Returns:
        Whether to cache models for a multi-model endpoint. By default, multi-model endpoints cache models so that a model does not have to be loaded into memory each time it is invoked. Some use cases do not benefit from model caching. For example, if an endpoint hosts a large number of models that are each invoked infrequently, the endpoint might perform better if you disable model caching. To disable model caching, set the value of this parameter to `Disabled`. Allowed values are: `Enabled` and `Disabled`.