Package ai.onnxruntime

Class OrtModelCompilationOptions

java.lang.Object
ai.onnxruntime.OrtModelCompilationOptions
All Implemented Interfaces:
AutoCloseable
public final class OrtModelCompilationOptions extends Object implements AutoCloseable
Configuration options for compiling ONNX models.

  • Method Details

    • createFromSessionOptions

      public static OrtModelCompilationOptions createFromSessionOptions(OrtEnvironment env, OrtSession.SessionOptions sessionOptions) throws OrtException
      Creates a model compilation options from an existing SessionOptions.

      An OrtModelCompilationOptions object contains the settings used to generate a compiled ONNX model. The OrtSessionOptions object has the execution providers with which the model will be compiled.

      Parameters:
      env - The OrtEnvironment.
      sessionOptions - The session options to use.
      Returns:
      A constructed model compilation options instance.
      Throws:
      OrtException - If the construction failed.

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable

    • setInputModelPath

      public void setInputModelPath(String inputModelPath) throws OrtException
      Sets the file path to the input ONNX model.

      The input model's location must be set either to a path on disk with this method, or by supplying an in-memory reference with setInputModelFromBuffer(java.nio.ByteBuffer).

      Parameters:
      inputModelPath - The path to the model on disk.
      Throws:
      OrtException - If the set failed.

    • setInputModelFromBuffer

      public void setInputModelFromBuffer(ByteBuffer inputModelBuffer) throws OrtException
      Uses the supplied buffer as the input ONNX model.

      The input model's location must be set either to an in-memory reference with this method, or by supplying a path on disk with setInputModelPath(String).

      If the ByteBuffer is not direct it is copied into a direct buffer. In either case this object holds a reference to the buffer to prevent it from being GC'd.

      Parameters:
      inputModelBuffer - The buffer.
      Throws:
      OrtException - If the buffer could not be set.

    • setOutputModelPath

      public void setOutputModelPath(String outputModelPath) throws OrtException
      Sets the file path for the output compiled ONNX model.

      If this is unset it will append `_ctx` to the file name, e.g., my_model.onnx becomes my_model_ctx.onnx.

      Parameters:
      outputModelPath - The output model path.
      Throws:
      OrtException - If the path could not be set.

    • setOutputExternalInitializersPath

      public void setOutputExternalInitializersPath(String outputExternalInitializersPath, long sizeThreshold) throws OrtException
      Optionally sets the file that stores initializers for the compiled ONNX model. If unset then initializers are stored inside the model.

      Only initializers for nodes that were not compiled are stored in the external initializers file. Compiled nodes contain their initializer data within the `ep_cache_context` attribute of EPContext nodes.

      Parameters:
      outputExternalInitializersPath - Path to the file.
      sizeThreshold - Initializers larger than this threshold are stored in the file.
      Throws:
      OrtException - If the path could not be set.
      See Also:

    • setEpContextEmbedMode

      public void setEpContextEmbedMode(boolean embedEpContext) throws OrtException
      Enables or disables the embedding of EPContext binary data into the ep_cache_context attribute of EPContext nodes.

      Defaults to false. When enabled, the `ep_cache_context` attribute of EPContext nodes will store the context binary data, which may include weights for compiled subgraphs. When disabled, the `ep_cache_context` attribute of EPContext nodes will contain the path to the file containing the context binary data. The path is set by the execution provider creating the EPContext node.

      For more details see the EPContext design document.

      Parameters:
      embedEpContext - True to embed EPContext binary data into the EPContext node's ep_cache_context attribute.
      Throws:
      OrtException - If the set operation failed.

    • setCompilationFlags

      public void setCompilationFlags(EnumSet<OrtModelCompilationOptions.OrtCompileApiFlags> flags) throws OrtException
      Sets the specified compilation flags.
      Parameters:
      flags - The compilation flags.
      Throws:
      OrtException - If the set operation failed.

    • compileModel

      public void compileModel() throws OrtException
      Compiles the ONNX model with the configuration described by this instance of OrtModelCompilationOptions.
      Throws:
      OrtException - If the compilation failed.