ICudaEngine (JavaCPP Presets for TensorRT 8.4-1.5.8 API)

java.lang.Object
- org.bytedeco.javacpp.Pointer
- - org.bytedeco.tensorrt.nvinfer.INoCopy
  - - org.bytedeco.tensorrt.nvinfer.ICudaEngine

All Implemented Interfaces:

AutoCloseable
```
@Namespace(value="nvinfer1")
 @NoOffset
 @Properties(inherit=nvinfer.class)
public class ICudaEngine
extends INoCopy
```
\class ICudaEngine \brief An engine for executing inference on a built network, with functionally unsafe features. \warning Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Nested Class Summary
- Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer
  Pointer.CustomDeallocator, Pointer.Deallocator, Pointer.NativeDeallocator, Pointer.ReferenceCounter

Field Summary
- Fields inherited from class org.bytedeco.javacpp.Pointer
  address, capacity, limit, position

Constructor Summary

Constructors
Constructor and Description

ICudaEngine()
Default native constructor.

ICudaEngine(long size)
Native array allocator.

ICudaEngine(Pointer p)
Pointer cast constructor.

Constructors
Constructor and Description
`ICudaEngine()` Default native constructor.
`ICudaEngine(long size)` Native array allocator.
`ICudaEngine(Pointer p)` Pointer cast constructor.

Method Summary

All Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`boolean`	`bindingIsInput(int bindingIndex)` \brief Determine whether a binding is an input binding.
`IEngineInspector`	`createEngineInspector()` \brief Create a new engine inspector which prints the layer information in an engine or an execution context.
`IExecutionContext`	`createExecutionContext()` \brief Create an execution context.
`IExecutionContext`	`createExecutionContextWithoutDeviceMemory()` \brief create an execution context without any device memory allocated The memory for execution of this device context must be supplied by the application.
`void`	`destroy()` Deprecated. Use `delete` instead. Deprecated in TRT 8.0. \warning Calling destroy on a managed pointer will result in a double-free error.
`int`	`getBindingBytesPerComponent(int bindingIndex)` \brief Return the number of bytes per component of an element.
`int`	`getBindingComponentsPerElement(int bindingIndex)` \brief Return the number of components included in one element.
`nvinfer.DataType`	`getBindingDataType(int bindingIndex)` \brief Determine the required data type for a buffer from its binding index.
`Dims32`	`getBindingDimensions(int bindingIndex)` \brief Get the dimensions of a binding.
`nvinfer.TensorFormat`	`getBindingFormat(int bindingIndex)` \brief Return the binding format.
`String`	`getBindingFormatDesc(int bindingIndex)` \brief Return the human readable description of the tensor format.
`int`	`getBindingIndex(BytePointer name)`
`int`	`getBindingIndex(String name)` \brief Retrieve the binding index for a named tensor.
`String`	`getBindingName(int bindingIndex)` \brief Retrieve the name corresponding to a binding index.
`int`	`getBindingVectorizedDim(int bindingIndex)` \brief Return the dimension index that the buffer is vectorized.
`long`	`getDeviceMemorySize()` \brief Return the amount of device memory required by an execution context.
`nvinfer.EngineCapability`	`getEngineCapability()` \brief Determine what execution capability this engine has.
`IErrorRecorder`	`getErrorRecorder()` \brief Get the ErrorRecorder assigned to this interface.
`nvinfer.TensorLocation`	`getLocation(int bindingIndex)` \brief Get location of binding This lets you know whether the binding should be a pointer to device or host memory.
`int`	`getMaxBatchSize()` Deprecated. Deprecated in TensorRT 8.4.
`String`	`getName()` \brief Returns the name of the network associated with the engine.
`int`	`getNbBindings()` \brief Get the number of binding indices.
`int`	`getNbLayers()` \brief Get the number of layers in the network.
`int`	`getNbOptimizationProfiles()` \brief Get the number of optimization profiles defined for this engine.
`ICudaEngine`	`getPointer(long i)`
`Dims32`	`getProfileDimensions(int bindingIndex, int profileIndex, int select)`
`Dims32`	`getProfileDimensions(int bindingIndex, int profileIndex, nvinfer.OptProfileSelector select)` \brief Get the minimum / optimum / maximum dimensions for a particular binding under an optimization profile.
`IntBuffer`	`getProfileShapeValues(int profileIndex, int inputIndex, int select)`
`IntPointer`	`getProfileShapeValues(int profileIndex, int inputIndex, nvinfer.OptProfileSelector select)` \brief Get minimum / optimum / maximum values for an input shape binding under an optimization profile.
`nvinfer.ProfilingVerbosity`	`getProfilingVerbosity()` \brief Return the \ref ProfilingVerbosity the builder config was set to when the engine was built.
`int`	`getTacticSources()` \brief return the tactic sources required by this engine.
`boolean`	`hasImplicitBatchDimension()` \brief Query whether the engine was built with an implicit batch dimension.
`boolean`	`isExecutionBinding(int bindingIndex)` \brief True if pointer to tensor data is required for execution phase, false if nullptr can be supplied.
`boolean`	`isRefittable()` \brief Return true if an engine can be refit.
`boolean`	`isShapeBinding(int bindingIndex)` \brief True if tensor is required as input for shape calculations or output from them.
`ICudaEngine`	`position(long position)`
`IHostMemory`	`serialize()` \brief Serialize the network to a stream.
`void`	`setErrorRecorder(IErrorRecorder recorder)`

Methods inherited from class org.bytedeco.javacpp.Pointer
address, asBuffer, asByteBuffer, availablePhysicalBytes, calloc, capacity, capacity, close, deallocate, deallocate, deallocateReferences, deallocator, deallocator, equals, fill, formatBytes, free, getDirectBufferAddress, getPointer, getPointer, getPointer, hashCode, interruptDeallocatorThread, isNull, isNull, limit, limit, malloc, maxBytes, maxPhysicalBytes, memchr, memcmp, memcpy, memmove, memset, offsetAddress, offsetof, offsetof, parseBytes, physicalBytes, physicalBytesInaccurate, position, put, realloc, referenceCount, releaseReference, retainReference, setNull, sizeof, sizeof, toString, totalBytes, totalCount, totalPhysicalBytes, withDeallocator, zero

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

- Constructor Detail
  - ICudaEngine
```
public ICudaEngine()
```
    Default native constructor.
  - ICudaEngine
```
public ICudaEngine(long size)
```
    Native array allocator. Access with Pointer.position(long).
  - ICudaEngine
```
public ICudaEngine(Pointer p)
```
    Pointer cast constructor. Invokes Pointer(Pointer).
- Method Detail
  - position
```
public ICudaEngine position(long position)
```
    Overrides:
    
    position in class Pointer
  - getPointer
```
public ICudaEngine getPointer(long i)
```
    Overrides:
    
    getPointer in class Pointer
  - getNbBindings
```
@NoException(value=true)
public int getNbBindings()
```
    \brief Get the number of binding indices. There are separate binding indices for each optimization profile. This method returns the total over all profiles. If the engine has been built for K profiles, the first getNbBindings() / K bindings are used by profile number 0, the following getNbBindings() / K bindings are used by profile number 1 etc.
    
    See Also:
    
    getBindingIndex();
  - getBindingIndex
```
@NoException(value=true)
public int getBindingIndex(String name)
```
    \brief Retrieve the binding index for a named tensor. IExecutionContext::enqueueV2() and IExecutionContext::executeV2() require an array of buffers. Engine bindings map from tensor names to indices in this array. Binding indices are assigned at engine build time, and take values in the range [0 ... n-1] where n is the total number of inputs and outputs. To get the binding index of the name in an optimization profile with index k > 0, mangle the name by appending " [profile k]", as described for method getBindingName().
    
    Parameters:
    
    name - The tensor name.
    
    Returns:
    
    The binding index for the named tensor, or -1 if the name is not found.
    
    See Also:
    
    getBindingName()
  - getBindingIndex
```
@NoException(value=true)
public int getBindingIndex(@Cast(value="const char*")
                                                     BytePointer name)
```
  - getBindingName
```
@NoException(value=true)
public String getBindingName(int bindingIndex)
```
    \brief Retrieve the name corresponding to a binding index. This is the reverse mapping to that provided by getBindingIndex(). For optimization profiles with an index k > 0, the name is mangled by appending " [profile k]", with k written in decimal. For example, if the tensor in the INetworkDefinition had the name "foo", and bindingIndex refers to that tensor in the optimization profile with index 3, getBindingName returns "foo [profile 3]".
    
    Parameters:
    
    bindingIndex - The binding index.
    
    Returns:
    
    The name corresponding to the index, or nullptr if the index is out of range.
    
    See Also:
    
    getBindingIndex()
  - bindingIsInput
```
@Cast(value="bool")
 @NoException(value=true)
public boolean bindingIsInput(int bindingIndex)
```
    \brief Determine whether a binding is an input binding.
    
    Parameters:
    
    bindingIndex - The binding index.
    
    Returns:
    
    True if the index corresponds to an input binding and the index is in range.
    
    See Also:
    
    getBindingIndex()
  - getBindingDimensions
```
@ByVal
 @Cast(value="nvinfer1::Dims*")
 @NoException(value=true)
public Dims32 getBindingDimensions(int bindingIndex)
```
    \brief Get the dimensions of a binding.
    
    Parameters:
    
    bindingIndex - The binding index.
    
    Returns:
    
    The dimensions of the binding if the index is in range, otherwise Dims(). Has -1 for any dimension that varies within the optimization profile. For example, suppose an INetworkDefinition has an input with shape [-1,-1] that becomes a binding b in the engine. If the associated optimization profile specifies that b has minimum dimensions as [6,9] and maximum dimensions [7,9], getBindingDimensions(b) returns [-1,9], despite the second dimension being dynamic in the INetworkDefinition. Because each optimization profile has separate bindings, the returned value can differ across profiles. Consider another binding b' for the same network input, but for another optimization profile. If that other profile specifies minimum dimensions [5,8] and maximum dimensions [5,9], getBindingDimensions(b') returns [5,-1].
    
    See Also:
    
    getBindingIndex()
  - getBindingDataType
```
@NoException(value=true)
public nvinfer.DataType getBindingDataType(int bindingIndex)
```
    \brief Determine the required data type for a buffer from its binding index.
    
    Parameters:
    
    bindingIndex - The binding index.
    
    Returns:
    
    The type of the data in the buffer.
    
    See Also:
    
    getBindingIndex()
  - getMaxBatchSize
```
@Deprecated
 @NoException(value=true)
public int getMaxBatchSize()
```
    Deprecated. Deprecated in TensorRT 8.4.
    
    \brief Get the maximum batch size which can be used for inference. Should only be called if the engine is built from an INetworkDefinition with implicit batch dimension mode.
    
    Returns:
    
    The maximum batch size for this engine. \warning For an engine built from an INetworkDefinition with explicit batch dimension mode, this will always return 1.
  - getNbLayers
```
@NoException(value=true)
public int getNbLayers()
```
    \brief Get the number of layers in the network. The number of layers in the network is not necessarily the number in the original network definition, as layers may be combined or eliminated as the engine is optimized. This value can be useful when building per-layer tables, such as when aggregating profiling data over a number of executions.
    
    Returns:
    
    The number of layers in the network.
  - serialize
```
@NoException(value=true)
public IHostMemory serialize()
```
    \brief Serialize the network to a stream.
    
    Returns:
    
    A IHostMemory object that contains the serialized engine. The network may be deserialized with IRuntime::deserializeCudaEngine().
    
    See Also:
    
    IRuntime::deserializeCudaEngine()
  - createExecutionContext
```
@NoException(value=true)
public IExecutionContext createExecutionContext()
```
    \brief Create an execution context. If the engine supports dynamic shapes, each execution context in concurrent use must use a separate optimization profile. The first execution context created will call setOptimizationProfile(0) implicitly. For other execution contexts, setOptimizationProfile() must be called with unique profile index before calling execute or enqueue. If an error recorder has been set for the engine, it will also be passed to the execution context.
    
    See Also:
    
    IExecutionContext., IExecutionContext::setOptimizationProfile()
  - destroy
```
@Deprecated
 @NoException(value=true)
public void destroy()
```
    Deprecated. Use delete instead. Deprecated in TRT 8.0. \warning Calling destroy on a managed pointer will result in a double-free error.
    
    \brief Destroy this object;
  - getLocation
```
@NoException(value=true)
public nvinfer.TensorLocation getLocation(int bindingIndex)
```
    \brief Get location of binding This lets you know whether the binding should be a pointer to device or host memory.
    
    Parameters:
    
    bindingIndex - The binding index.
    
    Returns:
    
    The location of the bound tensor with given index.
    
    See Also:
    
    ITensor::getLocation()
  - createExecutionContextWithoutDeviceMemory
```
@NoException(value=true)
public IExecutionContext createExecutionContextWithoutDeviceMemory()
```
    \brief create an execution context without any device memory allocated The memory for execution of this device context must be supplied by the application.
  - getDeviceMemorySize
```
@Cast(value="size_t")
 @NoException(value=true)
public long getDeviceMemorySize()
```
    \brief Return the amount of device memory required by an execution context.
    
    See Also:
    
    IExecutionContext::setDeviceMemory()
  - isRefittable
```
@Cast(value="bool")
 @NoException(value=true)
public boolean isRefittable()
```
    \brief Return true if an engine can be refit.
    
    See Also:
    
    nvinfer1::createInferRefitter()
  - getBindingBytesPerComponent
```
@NoException(value=true)
public int getBindingBytesPerComponent(int bindingIndex)
```
    \brief Return the number of bytes per component of an element. The vector component size is returned if getBindingVectorizedDim() != -1.
    
    Parameters:
    
    bindingIndex - The binding Index.
    
    See Also:
    
    ICudaEngine::getBindingVectorizedDim()
  - getBindingComponentsPerElement
```
@NoException(value=true)
public int getBindingComponentsPerElement(int bindingIndex)
```
    \brief Return the number of components included in one element. The number of elements in the vectors is returned if getBindingVectorizedDim() != -1.
    
    Parameters:
    
    bindingIndex - The binding Index.
    
    See Also:
    
    ICudaEngine::getBindingVectorizedDim()
  - getBindingFormat
```
@NoException(value=true)
public nvinfer.TensorFormat getBindingFormat(int bindingIndex)
```
    \brief Return the binding format.
    
    Parameters:
    
    bindingIndex - The binding Index.
  - getBindingFormatDesc
```
@NoException(value=true)
public String getBindingFormatDesc(int bindingIndex)
```
    \brief Return the human readable description of the tensor format. The description includes the order, vectorization, data type, strides, and etc. Examples are shown as follows: Example 1: kCHW + FP32 "Row major linear FP32 format" Example 2: kCHW2 + FP16 "Two wide channel vectorized row major FP16 format" Example 3: kHWC8 + FP16 + Line Stride = 32 "Channel major FP16 format where C % 8 == 0 and H Stride % 32 == 0"
    
    Parameters:
    
    bindingIndex - The binding Index.
  - getBindingVectorizedDim
```
@NoException(value=true)
public int getBindingVectorizedDim(int bindingIndex)
```
    \brief Return the dimension index that the buffer is vectorized. Specifically -1 is returned if scalars per vector is 1.
    
    Parameters:
    
    bindingIndex - The binding Index.
  - getName
```
@NoException(value=true)
public String getName()
```
    \brief Returns the name of the network associated with the engine. The name is set during network creation and is retrieved after building or deserialization.
    
    Returns:
    
    A null-terminated C-style string representing the name of the network.
    
    See Also:
    
    INetworkDefinition::setName(), INetworkDefinition::getName()
  - getNbOptimizationProfiles
```
@NoException(value=true)
public int getNbOptimizationProfiles()
```
    \brief Get the number of optimization profiles defined for this engine.
    
    Returns:
    
    Number of optimization profiles. It is always at least 1.
    
    See Also:
    
    IExecutionContext::setOptimizationProfile()
  - getProfileDimensions
```
@ByVal
 @Cast(value="nvinfer1::Dims*")
 @NoException(value=true)
public Dims32 getProfileDimensions(int bindingIndex,
                                                                                                       int profileIndex,
                                                                                                       nvinfer.OptProfileSelector select)
```
    \brief Get the minimum / optimum / maximum dimensions for a particular binding under an optimization profile.
    
    Parameters:
    
    bindingIndex - The binding index, which must belong to the given profile, or be between 0 and bindingsPerProfile-1 as described below.
    
    profileIndex - The profile index, which must be between 0 and getNbOptimizationProfiles()-1.
    
    select - Whether to query the minimum, optimum, or maximum dimensions for this binding.
    
    Returns:
    
    The minimum / optimum / maximum dimensions for this binding in this profile. If the profileIndex or bindingIndex are invalid, return Dims with nbDims=-1. For backwards compatibility with earlier versions of TensorRT, if the bindingIndex does not belong to the current optimization profile, but is between 0 and bindingsPerProfile-1, where bindingsPerProfile = getNbBindings()/getNbOptimizationProfiles, then a corrected bindingIndex is used instead, computed by: profileIndex * bindingsPerProfile + bindingIndex % bindingsPerProfile Otherwise the bindingIndex is considered invalid.
  - getProfileDimensions
```
@ByVal
 @Cast(value="nvinfer1::Dims*")
 @NoException(value=true)
public Dims32 getProfileDimensions(int bindingIndex,
                                                                                                       int profileIndex,
                                                                                                       @Cast(value="nvinfer1::OptProfileSelector")
                                                                                                       int select)
```
  - getProfileShapeValues
```
@Const
 @NoException(value=true)
public IntPointer getProfileShapeValues(int profileIndex,
                                                                           int inputIndex,
                                                                           nvinfer.OptProfileSelector select)
```
    \brief Get minimum / optimum / maximum values for an input shape binding under an optimization profile.
    
    Parameters:
    
    profileIndex - The profile index (must be between 0 and getNbOptimizationProfiles()-1)
    
    inputIndex - The input index (must be between 0 and getNbBindings() - 1)
    
    select - Whether to query the minimum, optimum, or maximum shape values for this binding.
    
    Returns:
    
    If the binding is an input shape binding, return a pointer to an array that has the same number of elements as the corresponding tensor, i.e. 1 if dims.nbDims == 0, or dims.d[0] if dims.nbDims == 1, where dims = getBindingDimensions(inputIndex). The array contains the elementwise minimum / optimum / maximum values for this shape binding under the profile. If either of the indices is out of range, or if the binding is not an input shape binding, return nullptr. For backwards compatibility with earlier versions of TensorRT, a bindingIndex that does not belong to the profile is corrected as described for getProfileDimensions.
    
    See Also:
    
    ICudaEngine::getProfileDimensions
  - getProfileShapeValues
```
@Const
 @NoException(value=true)
public IntBuffer getProfileShapeValues(int profileIndex,
                                                                          int inputIndex,
                                                                          @Cast(value="nvinfer1::OptProfileSelector")
                                                                          int select)
```
  - isShapeBinding
```
@Cast(value="bool")
 @NoException(value=true)
public boolean isShapeBinding(int bindingIndex)
```
    \brief True if tensor is required as input for shape calculations or output from them. TensorRT evaluates a network in two phases: 1. Compute shape information required to determine memory allocation requirements and validate that runtime sizes make sense. 2. Process tensors on the device. Some tensors are required in phase 1. These tensors are called "shape tensors", and always have type Int32 and no more than one dimension. These tensors are not always shapes themselves, but might be used to calculate tensor shapes for phase 2. isShapeBinding(i) returns true if the tensor is a required input or an output computed in phase 1. isExecutionBinding(i) returns true if the tensor is a required input or an output computed in phase 2. For example, if a network uses an input tensor with binding i as an addend to an IElementWiseLayer that computes the "reshape dimensions" for IShuffleLayer, then isShapeBinding(i) == true. It's possible to have a tensor be required by both phases. For instance, a tensor can be used for the "reshape dimensions" and as the indices for an IGatherLayer collecting floating-point data. It's also possible to have a tensor be required by neither phase, but nonetheless shows up in the engine's inputs. For example, if an input tensor is used only as an input to IShapeLayer, only its shape matters and its values are irrelevant.
    
    See Also:
    
    isExecutionBinding()
  - isExecutionBinding
```
@Cast(value="bool")
 @NoException(value=true)
public boolean isExecutionBinding(int bindingIndex)
```
    \brief True if pointer to tensor data is required for execution phase, false if nullptr can be supplied. For example, if a network uses an input tensor with binding i ONLY as the "reshape dimensions" input of IShuffleLayer, then isExecutionBinding(i) is false, and a nullptr can be supplied for it when calling IExecutionContext::execute or IExecutionContext::enqueue.
    
    See Also:
    
    isShapeBinding()
  - getEngineCapability
```
@NoException(value=true)
public nvinfer.EngineCapability getEngineCapability()
```
    \brief Determine what execution capability this engine has. If the engine has EngineCapability::kSTANDARD, then all engine functionality is valid. If the engine has EngineCapability::kSAFETY, then only the functionality in safe engine is valid. If the engine has EngineCapability::kDLA_STANDALONE, then only serialize, destroy, and const-accessor functions are valid.
    
    Returns:
    
    The EngineCapability flag that the engine was built for.
  - setErrorRecorder
```
@NoException(value=true)
public void setErrorRecorder(IErrorRecorder recorder)
```
    See Also:
    
    /**
  - getErrorRecorder
```
@NoException(value=true)
public IErrorRecorder getErrorRecorder()
```
    \brief Get the ErrorRecorder assigned to this interface. Retrieves the assigned error recorder object for the given class. A nullptr will be returned if an error handler has not been set.
    
    Returns:
    
    A pointer to the IErrorRecorder object that has been registered.
    
    See Also:
    
    setErrorRecorder()
  - hasImplicitBatchDimension
```
@Cast(value="bool")
 @NoException(value=true)
public boolean hasImplicitBatchDimension()
```
    \brief Query whether the engine was built with an implicit batch dimension.
    
    Returns:
    
    True if tensors have implicit batch dimension, false otherwise. This is an engine-wide property. Either all tensors in the engine have an implicit batch dimension or none of them do. hasImplicitBatchDimension() is true if and only if the INetworkDefinition from which this engine was built was created with createNetworkV2() without NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag.
    
    See Also:
    
    createNetworkV2
  - getTacticSources
```
@Cast(value="nvinfer1::TacticSources")
 @NoException(value=true)
public int getTacticSources()
```
    \brief return the tactic sources required by this engine. The value returned is equal to zero or more tactics sources set at build time via \ref IBuilderConfig::setTacticSources(). Sources set by the latter but not returned by \ref ICudaEngine::getTacticSources do not reduce overall engine execution time, and can be removed from future builds to reduce build time.
    
    See Also:
    
    IBuilderConfig::setTacticSources()
  - getProfilingVerbosity
```
@NoException(value=true)
public nvinfer.ProfilingVerbosity getProfilingVerbosity()
```
    \brief Return the \ref ProfilingVerbosity the builder config was set to when the engine was built.
    
    Returns:
    
    the profiling verbosity the builder config was set to when the engine was built.
    
    See Also:
    
    IBuilderConfig::setProfilingVerbosity()
  - createEngineInspector
```
@NoException(value=true)
public IEngineInspector createEngineInspector()
```
    \brief Create a new engine inspector which prints the layer information in an engine or an execution context.
    
    See Also:
    
    IEngineInspector.

Class ICudaEngine

Nested Class Summary

Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer

Field Summary

Fields inherited from class org.bytedeco.javacpp.Pointer

Constructor Summary

Method Summary

Methods inherited from class org.bytedeco.javacpp.Pointer

Methods inherited from class java.lang.Object

Constructor Detail

ICudaEngine

ICudaEngine

ICudaEngine

Method Detail

position

getPointer

getNbBindings

getBindingIndex

getBindingIndex

getBindingName

bindingIsInput

getBindingDimensions

getBindingDataType

getMaxBatchSize

getNbLayers

serialize

createExecutionContext

destroy

getLocation

createExecutionContextWithoutDeviceMemory

getDeviceMemorySize

isRefittable

getBindingBytesPerComponent

getBindingComponentsPerElement

getBindingFormat

getBindingFormatDesc

getBindingVectorizedDim

getName

getNbOptimizationProfiles

getProfileDimensions

getProfileDimensions

getProfileShapeValues

getProfileShapeValues

isShapeBinding

isExecutionBinding

getEngineCapability

setErrorRecorder

getErrorRecorder

hasImplicitBatchDimension

getTacticSources

getProfilingVerbosity

createEngineInspector