The TensorRT API version 1 namespace. More...

Namespaces
	anonymous_namespace{NvInfer.h}

	anonymous_namespace{NvInferRuntime.h}

	plugin

	utility

	utils

Classes
class	CUDADriverWrapper

class	Dims
	Structure to define the dimensions of a tensor. More...

class	Dims2
	Descriptor for two-dimensional data. More...

class	Dims3
	Descriptor for three-dimensional data. More...

class	Dims4
	Descriptor for four-dimensional data. More...

class	DimsExprs

class	DimsHW
	Descriptor for two-dimensional spatial data. More...

class	DynamicPluginTensorDesc

class	IActivationLayer
	An Activation layer in a network definition. More...

class	IAlgorithm
	Describes a variation of execution of a layer. An algorithm is represented by IAlgorithmVariant and the IAlgorithmIOInfo for each of its inputs and outputs. An algorithm can be selected or reproduced using AlgorithmSelector::selectAlgorithms().". More...

class	IAlgorithmContext
	Describes the context and requirements, that could be fulfilled by one or more instances of IAlgorithm. More...

class	IAlgorithmIOInfo
	Carries information about input or output of the algorithm. IAlgorithmIOInfo for all the input and output along with IAlgorithmVariant denotes the variation of algorithm and can be used to select or reproduce an algorithm using IAlgorithmSelector::selectAlgorithms(). More...

class	IAlgorithmSelector
	Interface implemented by application for selecting and reporting algorithms of a layer provided by the builder. More...

class	IAlgorithmVariant
	provides a unique 128-bit identifier, which along with the input and output information denotes the variation of algorithm and can be used to select or reproduce an algorithm, using IAlgorithmSelector::selectAlgorithms() More...

class	IBuilder
	Builds an engine from a network definition. More...

class	IBuilderConfig
	Holds properties for configuring a builder to produce an engine. More...

class	IConcatenationLayer
	A concatenation layer in a network definition. More...

class	IConstantLayer
	Layer that represents a constant value. More...

class	IConvolutionLayer
	A convolution layer in a network definition. More...

class	ICudaEngine
	An engine for executing inference on a built network, with functionally unsafe features. More...

class	IDeconvolutionLayer
	A deconvolution layer in a network definition. More...

class	IDimensionExpr

class	IElementWiseLayer
	A elementwise layer in a network definition. More...

class	IErrorRecorder
	Reference counted application-implemented error reporting interface for TensorRT objects. More...

class	IExecutionContext
	Context for executing inference using an engine, with functionally unsafe features. More...

class	IExprBuilder

class	IFillLayer
	Generate an output tensor with specified mode. More...

class	IFullyConnectedLayer
	A fully connected layer in a network definition. This layer expects an input tensor of three or more non-batch dimensions. The input is automatically reshaped into an `MxV` tensor `X`, where `V` is a product of the last three dimensions and `M` is a product of the remaining dimensions (where the product over 0 dimensions is defined as 1). For example: More...

class	IGatherLayer

class	IGpuAllocator
	Application-implemented class for controlling allocation on the GPU. More...

class	IHostMemory
	Class to handle library allocated memory that is accessible to the user. More...

class	IIdentityLayer
	A layer that represents the identity function. More...

class	IInt8Calibrator
	Application-implemented interface for calibration. More...

class	IInt8EntropyCalibrator
	Entropy calibrator. More...

class	IInt8EntropyCalibrator2
	Entropy calibrator 2. More...

class	IInt8LegacyCalibrator
	Legacy calibrator left for backward compatibility with TensorRT 2.0. More...

class	IInt8MinMaxCalibrator
	MinMax Calibrator. More...

class	IIteratorLayer

class	ILayer
	Base class for all layer classes in a network definition. More...

class	ILogger
	Application-implemented logging interface for the builder, engine and runtime. More...

class	ILoop
	Helper for creating a recurrent subgraph. More...

class	ILoopBoundaryLayer

class	ILoopOutputLayer
	An ILoopOutputLayer is the sole way to get output from a loop. More...

class	ILRNLayer
	A LRN layer in a network definition. More...

class	IMatrixMultiplyLayer
	Layer that represents a Matrix Multiplication. More...

class	INetworkDefinition
	A network definition for input to the builder. More...

class	IOptimizationProfile
	Optimization profile for dynamic input dimensions and shape tensors. More...

class	IPaddingLayer
	Layer that represents a padding operation. More...

class	IParametricReLULayer
	Layer that represents a parametric ReLU operation. More...

class	IPlugin
	Plugin class for user-implemented layers. More...

class	IPluginCreator
	Plugin creator class for user implemented layers. More...

class	IPluginExt
	Plugin class for user-implemented layers. More...

class	IPluginFactory
	Plugin factory for deserialization. More...

class	IPluginRegistry
	Single registration point for all plugins in an application. It is used to find plugin implementations during engine deserialization. Internally, the plugin registry is considered to be a singleton so all plugins in an application are part of the same global registry. Note that the plugin registry is only supported for plugins of type IPluginV2 and should also have a corresponding IPluginCreator implementation. More...

class	IPluginV2
	Plugin class for user-implemented layers. More...

class	IPluginV2DynamicExt

class	IPluginV2Ext
	Plugin class for user-implemented layers. More...

class	IPluginV2IOExt
	Plugin class for user-implemented layers. More...

class	IPluginV2Layer
	Layer type for pluginV2. More...

class	IPoolingLayer
	A Pooling layer in a network definition. More...

class	IProfiler
	Application-implemented interface for profiling. More...

class	IRaggedSoftMaxLayer
	A RaggedSoftmax layer in a network definition. More...

class	IRecurrenceLayer

class	IReduceLayer
	Layer that represents a reduction operator across Shape, Int32, Float, and Half tensors. More...

class	IRefitter
	Updates weights in an engine. More...

class	IResizeLayer
	A resize layer in a network definition. More...

class	IRuntime
	Allows a serialized functionally unsafe engine to be deserialized. More...

class	IScaleLayer
	A Scale layer in a network definition. More...

class	ISelectLayer

class	IShapeLayer
	Layer type for getting shape of a tensor. More...

class	IShuffleLayer
	Layer type for shuffling data. More...

class	ISliceLayer
	Slices an input tensor into an output tensor based on the offset and strides. More...

class	ISoftMaxLayer
	A Softmax layer in a network definition. More...

class	ITensor
	A tensor in a network definition. More...

class	ITopKLayer
	Layer that represents a TopK reduction. More...

class	ITripLimitLayer

class	IUnaryLayer
	Layer that represents an unary operation. More...

struct	Permutation

class	PluginField
	Structure containing plugin attribute field names and associated data This information can be parsed to decode necessary plugin metadata. More...

struct	PluginFieldCollection

class	PluginRegistrar
	Register the plugin creator to the registry The static registry object will be instantiated when the plugin library is loaded. More...

struct	PluginTensorDesc
	Fields that a plugin might see for an input or output. More...

class	Weights
	An array of weights used as a layer parameter. More...

Typedefs
typedef uint32_t	QuantizationFlags
	Represents a collection of one or more QuantizationFlag values using binary OR operations. More...

typedef uint32_t	BuilderFlags
	Represents a collection of one or more QuantizationFlag values using binary OR operations, e.g., 1U << BuilderFlag::kFP16 \| 1U << BuilderFlag::kDEBUG. More...

using	TacticSources = uint32_t
	Represents a collection of one or more TacticSource values combine using bitwise-OR operations. More...

typedef uint32_t	NetworkDefinitionCreationFlags
	This bitset is capable of representing one or more NetworkDefinitionCreationFlag flags constructed with binary OR operations. e.g., 1U << NetworkDefinitionCreationFlag::kEXPLICIT_BATCH. More...

typedef uint32_t	TensorFormats
	It is capable of representing one or more TensorFormat by binary OR operations, e.g., 1U << TensorFormats::kCHW4 \| 1U << TensorFormats::kCHW32. More...

using	PluginFormat = TensorFormat
	PluginFormat is reserved for backward compatibility. More...

Enumerations
enum	LayerType : int32_t { LayerType::kCONVOLUTION = 0, LayerType::kFULLY_CONNECTED = 1, LayerType::kACTIVATION = 2, LayerType::kPOOLING = 3, LayerType::kLRN = 4, LayerType::kSCALE = 5, LayerType::kSOFTMAX = 6, LayerType::kDECONVOLUTION = 7, LayerType::kCONCATENATION = 8, LayerType::kELEMENTWISE = 9, LayerType::kPLUGIN = 10, LayerType::kRNN = 11, LayerType::kUNARY = 12, LayerType::kPADDING = 13, LayerType::kSHUFFLE = 14, LayerType::kREDUCE = 15, LayerType::kTOPK = 16, LayerType::kGATHER = 17, LayerType::kMATRIX_MULTIPLY = 18, LayerType::kRAGGED_SOFTMAX = 19, LayerType::kCONSTANT = 20, LayerType::kRNN_V2 = 21, LayerType::kIDENTITY = 22, LayerType::kPLUGIN_V2 = 23, LayerType::kSLICE = 24, LayerType::kSHAPE = 25, LayerType::kPARAMETRIC_RELU = 26, LayerType::kRESIZE = 27, LayerType::kTRIP_LIMIT = 28, LayerType::kRECURRENCE = 29, LayerType::kITERATOR = 30, LayerType::kLOOP_OUTPUT = 31, LayerType::kSELECT = 32, LayerType::kFILL = 33 }
	The type values of layer classes. More...

enum	PaddingMode : int32_t { PaddingMode::kEXPLICIT_ROUND_DOWN = 0, PaddingMode::kEXPLICIT_ROUND_UP = 1, PaddingMode::kSAME_UPPER = 2, PaddingMode::kSAME_LOWER = 3, PaddingMode::kCAFFE_ROUND_DOWN = 4, PaddingMode::kCAFFE_ROUND_UP = 5 }
	Enumerates the modes of padding to perform in convolution, deconvolution and pooling layer, padding mode takes precedence if setPaddingMode() and setPrePadding() are also used. More...

enum	PoolingType : int32_t { PoolingType::kMAX = 0, PoolingType::kAVERAGE = 1, PoolingType::kMAX_AVERAGE_BLEND = 2 }
	The type of pooling to perform in a pooling layer. More...

enum	ScaleMode : int32_t { ScaleMode::kUNIFORM = 0, ScaleMode::kCHANNEL = 1, ScaleMode::kELEMENTWISE = 2 }
	Controls how shift, scale and power are applied in a Scale layer. More...

enum	ElementWiseOperation : int32_t { ElementWiseOperation::kSUM = 0, ElementWiseOperation::kPROD = 1, ElementWiseOperation::kMAX = 2, ElementWiseOperation::kMIN = 3, ElementWiseOperation::kSUB = 4, ElementWiseOperation::kDIV = 5, ElementWiseOperation::kPOW = 6, ElementWiseOperation::kFLOOR_DIV = 7, ElementWiseOperation::kAND = 8, ElementWiseOperation::kOR = 9, ElementWiseOperation::kXOR = 10, ElementWiseOperation::kEQUAL = 11, ElementWiseOperation::kGREATER = 12, ElementWiseOperation::kLESS = 13 }
	Enumerates the binary operations that may be performed by an ElementWise layer. More...

enum	RNNOperation : int32_t { RNNOperation::kRELU = 0, RNNOperation::kTANH = 1, RNNOperation::kLSTM = 2, RNNOperation::kGRU = 3 }
	Enumerates the RNN operations that may be performed by an RNN layer. More...

enum	RNNDirection : int32_t { RNNDirection::kUNIDIRECTION = 0, RNNDirection::kBIDIRECTION = 1 }
	Enumerates the RNN direction that may be performed by an RNN layer. More...

enum	RNNInputMode : int32_t { RNNInputMode::kLINEAR = 0, RNNInputMode::kSKIP = 1 }
	Enumerates the RNN input modes that may occur with an RNN layer. More...

enum	RNNGateType : int32_t { RNNGateType::kINPUT = 0, RNNGateType::kOUTPUT = 1, RNNGateType::kFORGET = 2, RNNGateType::kUPDATE = 3, RNNGateType::kRESET = 4, RNNGateType::kCELL = 5, RNNGateType::kHIDDEN = 6 }
	Identifies an individual gate within an RNN cell. More...

enum	UnaryOperation : int32_t { UnaryOperation::kEXP = 0, UnaryOperation::kLOG = 1, UnaryOperation::kSQRT = 2, UnaryOperation::kRECIP = 3, UnaryOperation::kABS = 4, UnaryOperation::kNEG = 5, UnaryOperation::kSIN = 6, UnaryOperation::kCOS = 7, UnaryOperation::kTAN = 8, UnaryOperation::kSINH = 9, UnaryOperation::kCOSH = 10, UnaryOperation::kASIN = 11, UnaryOperation::kACOS = 12, UnaryOperation::kATAN = 13, UnaryOperation::kASINH = 14, UnaryOperation::kACOSH = 15, UnaryOperation::kATANH = 16, UnaryOperation::kCEIL = 17, UnaryOperation::kFLOOR = 18, UnaryOperation::kERF = 19, UnaryOperation::kNOT = 20 }
	Enumerates the unary operations that may be performed by a Unary layer. More...

enum	ReduceOperation : int32_t { ReduceOperation::kSUM = 0, ReduceOperation::kPROD = 1, ReduceOperation::kMAX = 2, ReduceOperation::kMIN = 3, ReduceOperation::kAVG = 4 }
	Enumerates the reduce operations that may be performed by a Reduce layer. More...

enum	SliceMode : int32_t { SliceMode::kDEFAULT = 0, SliceMode::kWRAP = 1 }
	Controls how ISliceLayer handles out of bounds coordinates. More...

enum	TopKOperation : int32_t { TopKOperation::kMAX = 0, TopKOperation::kMIN = 1 }
	Enumerates the operations that may be performed by a TopK layer. More...

enum	MatrixOperation : int32_t { MatrixOperation::kNONE, MatrixOperation::kTRANSPOSE, MatrixOperation::kVECTOR }
	Enumerates the operations that may be performed on a tensor by IMatrixMultiplyLayer before multiplication. More...

enum	ResizeMode : int32_t { ResizeMode::kNEAREST = 0, ResizeMode::kLINEAR = 1 }
	Enumerates various modes of resize in the resize layer. Resize mode set using setResizeMode(). More...

enum	LoopOutput : int32_t { LoopOutput::kLAST_VALUE = 0, LoopOutput::kCONCATENATE = 1, LoopOutput::kREVERSE = 2 }
	Enum that describes kinds of loop outputs. More...

enum	TripLimit : int32_t { TripLimit::kCOUNT = 0, TripLimit::kWHILE = 1 }
	Enum that describes kinds of trip limits. More...

enum	FillOperation : int32_t { FillOperation::kLINSPACE = 0, FillOperation::kRANDOM_UNIFORM = 1 }
	Enumerates the tensor fill operations that may performed by a fill layer. More...

enum	CalibrationAlgoType : int32_t { CalibrationAlgoType::kLEGACY_CALIBRATION = 0, CalibrationAlgoType::kENTROPY_CALIBRATION = 1, CalibrationAlgoType::kENTROPY_CALIBRATION_2 = 2, CalibrationAlgoType::kMINMAX_CALIBRATION = 3 }
	enum CalibrationAlgoType More...

enum	QuantizationFlag : int32_t { QuantizationFlag::kCALIBRATE_BEFORE_FUSION = 0 }
	List of valid flags for quantizing the network to int8. More...

enum	BuilderFlag : int32_t { BuilderFlag::kFP16 = 0, BuilderFlag::kINT8 = 1, BuilderFlag::kDEBUG = 2, BuilderFlag::kGPU_FALLBACK = 3, BuilderFlag::kSTRICT_TYPES = 4, BuilderFlag::kREFIT = 5, BuilderFlag::kDISABLE_TIMING_CACHE = 6, BuilderFlag::kTF32 = 7 }
	List of valid modes that the builder can enable when creating an engine from a network definition. More...

enum	ProfilingVerbosity : int32_t { ProfilingVerbosity::kDEFAULT = 0, ProfilingVerbosity::kNONE = 1, ProfilingVerbosity::kVERBOSE = 2 }
	List of verbosity levels of layer information exposed in NVTX annotations. More...

enum	TacticSource : int32_t { TacticSource::kCUBLAS = 0, TacticSource::kCUBLAS_LT = 1 }
	List of tactic sources for TensorRT. More...

enum	NetworkDefinitionCreationFlag : int32_t { NetworkDefinitionCreationFlag::kEXPLICIT_BATCH = 0, NetworkDefinitionCreationFlag::kEXPLICIT_PRECISION = 1 }
	List of immutable network properties expressed at network creation time. NetworkDefinitionCreationFlag is used with createNetworkV2 to specify immutable properties of the network. The createNetwork() function always had an implicit batch dimension being specified by the maxBatchSize builder parameter. createNetworkV2 with kDEFAULT flag mimics that behaviour. More...

enum	PluginType : int32_t { PluginType::kFASTERRCNN = 0, PluginType::kNORMALIZE = 1, PluginType::kPERMUTE = 2, PluginType::kPRIORBOX = 3, PluginType::kSSDDETECTIONOUTPUT = 4, PluginType::kCONCAT = 5, PluginType::kPRELU = 6, PluginType::kYOLOREORG = 7, PluginType::kYOLOREGION = 8, PluginType::kANCHORGENERATOR = 9 }
	The type values for the various plugins. More...

enum	EngineCapability : int32_t { EngineCapability::kDEFAULT = 0, EngineCapability::kSAFE_GPU = 1, EngineCapability::kSAFE_DLA = 2 }
	Forward declaration of IPluginFactory for use by other interfaces. More...

enum	DimensionOperation : int32_t { DimensionOperation::kSUM = 0, DimensionOperation::kPROD = 1, DimensionOperation::kMAX = 2, DimensionOperation::kMIN = 3, DimensionOperation::kSUB = 4, DimensionOperation::kEQUAL = 5, DimensionOperation::kLESS = 6, DimensionOperation::kFLOOR_DIV = 7, DimensionOperation::kCEIL_DIV = 8 }
	An operation on two IDimensionExpr, which represent integer expressions used in dimension computations. More...

enum	WeightsRole : int32_t { WeightsRole::kKERNEL = 0, WeightsRole::kBIAS = 1, WeightsRole::kSHIFT = 2, WeightsRole::kSCALE = 3, WeightsRole::kCONSTANT = 4 }
	How a layer uses particular Weights. More...

enum	DeviceType : int32_t { DeviceType::kGPU, DeviceType::kDLA }
	The device that this layer/network will execute on. More...

enum	OptProfileSelector : int32_t { OptProfileSelector::kMIN = 0, OptProfileSelector::kOPT = 1, OptProfileSelector::kMAX = 2 }
	When setting or querying optimization profile parameters (such as shape tensor inputs or dynamic dimensions), select whether we are interested in the minimum, optimum, or maximum values for these parameters. The minimum and maximum specify the permitted range that is supported at runtime, while the optimum value is used for the kernel selection. This should be the "typical" value that is expected to occur at runtime. More...

enum	ActivationType : int32_t { ActivationType::kRELU = 0, ActivationType::kSIGMOID = 1, ActivationType::kTANH = 2, ActivationType::kLEAKY_RELU = 3, ActivationType::kELU = 4, ActivationType::kSELU = 5, ActivationType::kSOFTSIGN = 6, ActivationType::kSOFTPLUS = 7, ActivationType::kCLIP = 8, ActivationType::kHARD_SIGMOID = 9, ActivationType::kSCALED_TANH = 10, ActivationType::kTHRESHOLDED_RELU = 11 }
	Enumerates the types of activation to perform in an activation layer. More...

enum	DataType : int32_t { DataType::kFLOAT = 0, DataType::kHALF = 1, DataType::kINT8 = 2, DataType::kINT32 = 3, DataType::kBOOL = 4 }
	The type of weights and tensors. More...

enum	DimensionType : int32_t { DimensionType::kSPATIAL = 0, DimensionType::kCHANNEL = 1, DimensionType::kINDEX = 2, DimensionType::kSEQUENCE = 3 }
	The type of data encoded across this dimension. More...

enum	TensorFormat : int32_t { TensorFormat::kLINEAR = 0, TensorFormat::kNCHW = kLINEAR, TensorFormat::kCHW2 = 1, TensorFormat::kNC2HW2 = kCHW2, TensorFormat::kHWC8 = 2, TensorFormat::kNHWC8 = kHWC8, TensorFormat::kCHW4 = 3, TensorFormat::kCHW16 = 4, TensorFormat::kCHW32 = 5, TensorFormat::kDHWC8 = 6, TensorFormat::kCDHW32 = 7, TensorFormat::kHWC = 8, TensorFormat::kDLA_LINEAR = 9, TensorFormat::kDLA_HWC4 = 10 }
	Format of the input/output tensors. More...

enum	PluginVersion : uint8_t { PluginVersion::kV2 = 0, PluginVersion::kV2_EXT = 1, PluginVersion::kV2_IOEXT = 2, PluginVersion::kV2_DYNAMICEXT = 3 }

enum	PluginFieldType : int32_t { PluginFieldType::kFLOAT16 = 0, PluginFieldType::kFLOAT32 = 1, PluginFieldType::kFLOAT64 = 2, PluginFieldType::kINT8 = 3, PluginFieldType::kINT16 = 4, PluginFieldType::kINT32 = 5, PluginFieldType::kCHAR = 6, PluginFieldType::kDIMS = 7, PluginFieldType::kUNKNOWN = 8 }

enum	TensorLocation : int32_t { TensorLocation::kDEVICE = 0, TensorLocation::kHOST = 1 }
	The location for tensor data storage, device or host. More...

enum	ErrorCode : int32_t { ErrorCode::kSUCCESS = 0, ErrorCode::kUNSPECIFIED_ERROR = 1, ErrorCode::kINTERNAL_ERROR = 2, ErrorCode::kINVALID_ARGUMENT = 3, ErrorCode::kINVALID_CONFIG = 4, ErrorCode::kFAILED_ALLOCATION = 5, ErrorCode::kFAILED_INITIALIZATION = 6, ErrorCode::kFAILED_EXECUTION = 7, ErrorCode::kFAILED_COMPUTATION = 8, ErrorCode::kINVALID_STATE = 9, ErrorCode::kUNSUPPORTED_STATE = 10 }
	Error codes that can be returned by TensorRT during execution. More...

Functions
template<>
constexpr int32_t	EnumMax< LayerType > ()
	Maximum number of elements in LayerType enum. More...

template<>
constexpr int32_t	EnumMax< PaddingMode > ()
	Maximum number of elements in PaddingMode enum. More...

template<>
constexpr int32_t	EnumMax< PoolingType > ()
	Maximum number of elements in PoolingType enum. More...

template<>
constexpr int32_t	EnumMax< ScaleMode > ()
	Maximum number of elements in ScaleMode enum. More...

template<>
constexpr int32_t	EnumMax< ElementWiseOperation > ()
	Maximum number of elements in ElementWiseOperation enum. More...

template<>
constexpr int32_t	EnumMax< RNNOperation > ()
	Maximum number of elements in RNNOperation enum. More...

template<>
constexpr int32_t	EnumMax< RNNDirection > ()
	Maximum number of elements in RNNDirection enum. More...

template<>
constexpr int32_t	EnumMax< RNNInputMode > ()
	Maximum number of elements in RNNInputMode enum. More...

template<>
constexpr int32_t	EnumMax< RNNGateType > ()
	Maximum number of elements in RNNGateType enum. More...

class	__attribute__ ((deprecated)) IOutputDimensionsFormula

template<>
constexpr int32_t	EnumMax< UnaryOperation > ()
	Maximum number of elements in UnaryOperation enum. More...

template<>
constexpr int32_t	EnumMax< ReduceOperation > ()
	Maximum number of elements in ReduceOperation enum. More...

template<>
constexpr int32_t	EnumMax< SliceMode > ()
	Maximum number of elements in SliceMode enum. More...

template<>
constexpr int32_t	EnumMax< TopKOperation > ()
	Maximum number of elements in TopKOperation enum. More...

template<>
constexpr int32_t	EnumMax< MatrixOperation > ()
	Maximum number of elements in MatrixOperation enum. More...

template<>
constexpr int32_t	EnumMax< ResizeMode > ()
	Maximum number of elements in ResizeMode enum. More...

template<>
constexpr int32_t	EnumMax< LoopOutput > ()
	Maximum number of elements in LoopOutput enum. More...

template<>
constexpr int32_t	EnumMax< TripLimit > ()
	Maximum number of elements in TripLimit enum. More...

template<>
constexpr int32_t	EnumMax< FillOperation > ()
	Maximum number of elements in FillOperation enum. More...

template<>
constexpr int32_t	EnumMax< CalibrationAlgoType > ()
	Maximum number of elements in CalibrationAlgoType enum. More...

template<>
constexpr int32_t	EnumMax< QuantizationFlag > ()
	Maximum number of quantization flags in QuantizationFlag enum. More...

template<>
constexpr int32_t	EnumMax< BuilderFlag > ()
	Maximum number of builder flags in BuilderFlag enum. More...

template<>
constexpr int32_t	EnumMax< ProfilingVerbosity > ()
	Maximum number of profile verbosity levels in ProfilingVerbosity enum. More...

template<>
constexpr int32_t	EnumMax< TacticSource > ()
	Maximum number of tactic sources in TacticSource enum. More...

template<>
constexpr int32_t	EnumMax< NetworkDefinitionCreationFlag > ()
	Maximum number of elements in NetworkDefinitionCreationFlag enum. More...

template<>
constexpr int32_t	EnumMax< PluginType > ()
	Maximum number of elements in PluginType enum. More...

template<>
constexpr int32_t	EnumMax< EngineCapability > ()
	Maximum number of elements in EngineCapability enum. More...

template<>
constexpr int32_t	EnumMax< DimensionOperation > ()
	Maximum number of elements in DimensionOperation enum. More...

template<>
constexpr int32_t	EnumMax< WeightsRole > ()
	Maximum number of elements in WeightsRole enum. More...

template<>
constexpr int32_t	EnumMax< DeviceType > ()
	Maximum number of elements in DeviceType enum. More...

template<>
constexpr int32_t	EnumMax< OptProfileSelector > ()

template<typename T >
constexpr int32_t	EnumMax ()
	Forward declare IGpuAllocator for use in other interfaces. More...

template<>
constexpr int32_t	EnumMax< ActivationType > ()
	Maximum number of elements in ActivationType enum. More...

template<>
constexpr int32_t	EnumMax< DataType > ()
	Maximum number of elements in DataType enum. More...

template<>
constexpr int32_t	EnumMax< DimensionType > ()
	Maximum number of elements in DimensionType enum. More...

template<>
constexpr int32_t	EnumMax< TensorFormat > ()
	Maximum number of elements in TensorFormat enum. More...

template<>
constexpr int32_t	EnumMax< TensorLocation > ()
	Maximum number of elements in TensorLocation enum. More...

template<>
constexpr int32_t	EnumMax< ILogger::Severity > ()
	Maximum number of elements in ILogger::Severity enum. More...

template<>
constexpr int32_t	EnumMax< ErrorCode > ()
	Maximum number of elements in ErrorCode enum. More...

void	cuErrCheck_ (CUresult stat, const CUDADriverWrapper &wrap, const char *file, int line)

Detailed Description

The TensorRT API version 1 namespace.

Typedef Documentation

◆ QuantizationFlags

typedef uint32_t nvinfer1::QuantizationFlags

Represents a collection of one or more QuantizationFlag values using binary OR operations.

See also: IBuilderConfig::getQuantizationFlags(), IBuilderConfig::setQuantizationFlags()

◆ BuilderFlags

typedef uint32_t nvinfer1::BuilderFlags

Represents a collection of one or more QuantizationFlag values using binary OR operations, e.g., 1U << BuilderFlag::kFP16 | 1U << BuilderFlag::kDEBUG.

See also: IBuilderConfig::getFlags(), ITensor::setFlags(),

◆ TacticSources

using nvinfer1::TacticSources = typedef uint32_t

Represents a collection of one or more TacticSource values combine using bitwise-OR operations.

See also: IBuilderConfig::setTacticSources(), IBuilderConfig::getTacticSources()

◆ NetworkDefinitionCreationFlags

nvinfer1::NetworkDefinitionCreationFlags

This bitset is capable of representing one or more NetworkDefinitionCreationFlag flags constructed with binary OR operations. e.g., 1U << NetworkDefinitionCreationFlag::kEXPLICIT_BATCH.

See also: IBuilder::createNetworkV2

◆ TensorFormats

typedef uint32_t nvinfer1::TensorFormats

It is capable of representing one or more TensorFormat by binary OR operations, e.g., 1U << TensorFormats::kCHW4 | 1U << TensorFormats::kCHW32.

See also: ITensor::getAllowedFormats(), ITensor::setAllowedFormats(),

◆ PluginFormat

using nvinfer1::PluginFormat = typedef TensorFormat

PluginFormat is reserved for backward compatibility.

See also: IPluginExt::getPluginFormats()

Enumeration Type Documentation

◆ LayerType

enum nvinfer1::LayerType : int32_t

strong

The type values of layer classes.

See also: ILayer::getType()

Enumerator
kCONVOLUTION	Convolution layer.
kFULLY_CONNECTED	Fully connected layer.
kACTIVATION	Activation layer.
kPOOLING	Pooling layer.
kLRN	LRN layer.
kSCALE	Scale layer.
kSOFTMAX	SoftMax layer.
kDECONVOLUTION	Deconvolution layer.
kCONCATENATION	Concatenation layer.
kELEMENTWISE	Elementwise layer.
kPLUGIN	Plugin layer.
kRNN	RNN layer.
kUNARY	UnaryOp operation Layer.
kPADDING	Padding layer.
kSHUFFLE	Shuffle layer.
kREDUCE	Reduce layer.
kTOPK	TopK layer.
kGATHER	Gather layer.
kMATRIX_MULTIPLY	Matrix multiply layer.
kRAGGED_SOFTMAX	Ragged softmax layer.
kCONSTANT	Constant layer.
kRNN_V2	RNNv2 layer.
kIDENTITY	Identity layer.
kPLUGIN_V2	PluginV2 layer.
kSLICE	Slice layer.
kSHAPE	Shape layer.
kPARAMETRIC_RELU	Parametric ReLU layer.
kRESIZE	Resize Layer.
kTRIP_LIMIT	Loop Trip limit layer.
kRECURRENCE	Loop Recurrence layer.
kITERATOR	Loop Iterator layer.
kLOOP_OUTPUT	Loop output layer.
kSELECT	Select layer.
kFILL	Fill layer.

◆ PaddingMode

enum nvinfer1::PaddingMode : int32_t

strong

Enumerates the modes of padding to perform in convolution, deconvolution and pooling layer, padding mode takes precedence if setPaddingMode() and setPrePadding() are also used.

There are three padding styles, EXPLICIT, SAME, and CAFFE, with each style having two variants. The EXPLICIT and CAFFE styles determine if the final sampling location is used or not. The SAME style determine if the asymmetry in the padding is on the pre or post padding.

Shorthand:
    I = dimensions of input image.
    B = prePadding, before the image data. For deconvolution, prePadding is set before output.
    A = postPadding, after the image data. For deconvolution, postPadding is set after output.
    P = delta between input and output
    S = stride
    F = filter
    O = output
    D = dilation
    M = I + B + A ; The image data plus any padding
    DK = 1 + D * (F - 1)

Formulas for Convolution:

EXPLICIT_ROUND_DOWN:
O = floor((M - DK) / S) + 1
CAFFE_ROUND_DOWN:
O = floor((I + B * 2 - DK) / S)
EXPLICIT_ROUND_UP:
O = ceil((M - DK) / S) + 1
CAFFE_ROUND_UP:
O = ceil((I + B * 2 - DK) / S)
SAME_UPPER:
O = ceil(I / S)

P = floor((I - 1) / S) * S + DK - I;

B = floor(P / 2)

A = P - B
SAME_LOWER:
O = ceil(I / S)

P = floor((I - 1) / S) * S + DK - I;

A = floor(P / 2)

B = P - A

Formulas for Deconvolution:

EXPLICIT_ROUND_DOWN:
CAFFE_ROUND_DOWN:
EXPLICIT_ROUND_UP:
CAFFE_ROUND_UP:
O = (I - 1) * S + DK - (B + A)
SAME_UPPER:
O = min(I * S, (I - 1) * S + DK)

P = max(DK - S, 0)

B = floor(P / 2)

A = P - B
SAME_LOWER:
O = min(I * S, (I - 1) * S + DK)

P = max(DK - S, 0)

A = floor(P / 2)

B = P - A

Formulas for Pooling:

EXPLICIT_ROUND_DOWN:
O = floor((M - F) / S) + 1
EXPLICIT_ROUND_UP:
O = ceil((M - F) / S) + 1
SAME_UPPER:
O = ceil(I / S)

P = floor((I - 1) / S) * S + F - I;

B = floor(P / 2)

A = P - B
SAME_LOWER:
O = ceil(I / S)

P = floor((I - 1) / S) * S + F - I;

A = floor(P / 2)

B = P - A
CAFFE_ROUND_DOWN:
EXPLICIT_ROUND_DOWN - ((EXPLICIT_ROUND_DOWN - 1) * S >= I + B)
CAFFE_ROUND_UP:
EXPLICIT_ROUND_UP - ((EXPLICIT_ROUND_UP - 1) * S >= I + B)

Pooling Example 1:

Given I = {6, 6}, B = {3, 3}, A = {2, 2}, S = {2, 2}, F = {3, 3}. What is O?

(B, A can be calculated for SAME_UPPER and SAME_LOWER mode)

EXPLICIT_ROUND_DOWN:
Computation:

M = {6, 6} + {3, 3} + {2, 2} ==> {11, 11}

O ==> floor((M - F) / S) + 1

==> floor(({11, 11} - {3, 3}) / {2, 2}) + {1, 1}

==> floor({8, 8} / {2, 2}) + {1, 1}

==> {5, 5}
EXPLICIT_ROUND_UP:
Computation:

M = {6, 6} + {3, 3} + {2, 2} ==> {11, 11}

O ==> ceil((M - F) / S) + 1

==> ceil(({11, 11} - {3, 3}) / {2, 2}) + {1, 1}

==> ceil({8, 8} / {2, 2}) + {1, 1}

==> {5, 5}

The sample points are {0, 2, 4, 6, 8} in each dimension.
SAME_UPPER:
Computation:

I = {6, 6}

S = {2, 2}

O = ceil(I / S) = {3, 3}

P = floor((I - 1) / S) * S + F - I

==> floor(({6, 6} - {1, 1}) / {2, 2}) * {2, 2} + {3, 3} - {6, 6}

==> {4, 4} + {3, 3} - {6, 6}

==> {1, 1}

B = floor({1, 1} / {2, 2})

==> {0, 0}

A = {1, 1} - {0, 0}

==> {1, 1}
SAME_LOWER:
Computation:

I = {6, 6}

S = {2, 2}

O = ceil(I / S) = {3, 3}

P = floor((I - 1) / S) * S + F - I

==> {1, 1}

A = floor({1, 1} / {2, 2})

==> {0, 0}

B = {1, 1} - {0, 0}

==> {1, 1}

The sample pointers are {0, 2, 4} in each dimension. SAMPLE_UPPER has {O0, O1, O2, pad} in output in each dimension. SAMPLE_LOWER has {pad, O0, O1, O2} in output in each dimension.

Pooling Example 2:

Given I = {6, 6}, B = {3, 3}, A = {3, 3}, S = {2, 2}, F = {3, 3}. What is O?

CAFFE_ROUND_DOWN:
Computation:

M = {6, 6} + {3, 3} + {3, 3} ==> {12, 12}

EXPLICIT_ROUND_DOWN ==> floor((M - F) / S) + 1

==> floor(({12, 12} - {3, 3}) / {2, 2}) + {1, 1}

==> {5, 5}

DIFF = (((EXPLICIT_ROUND_DOWN - 1) * S >= I + B) ? {1, 1} : {0, 0})

==> ({5, 5} - {1, 1}) * {2, 2} >= {6, 6} + {3, 3} ? {1, 1} : {0,0}

==> {0, 0}

O ==> EXPLICIT_ROUND_DOWN - DIFF

==> {5, 5} - {0, 0}

==> {5, 5}
CAFFE_ROUND_UP:
Computation:

M = {6, 6} + {3, 3} + {3, 3} ==> {12, 12}

EXPLICIT_ROUND_UP ==> ceil((M - F) / S) + 1

==> ceil(({12, 12} - {3, 3}) / {2, 2}) + {1, 1}

==> {6, 6}

DIFF = (((EXPLICIT_ROUND_UP - 1) * S >= I + B) ? {1, 1} : {0, 0})

==> ({6, 6} - {1, 1}) * {2, 2} >= {6, 6} + {3, 3} ? {1, 1} : {0,0}

==> {1, 1}

O ==> EXPLICIT_ROUND_UP - DIFF

==> {6, 6} - {1, 1}

==> {5, 5}

The sample points are {0, 2, 4, 6, 8} in each dimension.
CAFFE_ROUND_DOWN and CAFFE_ROUND_UP have two restrictions each on usage with pooling operations. This will cause getDimensions to return an empty dimension and also to reject the network at validation time.
For more information on original reference code, see https://github.com/BVLC/caffe/blob/master/src/caffe/layers/pooling_layer.cpp

Restriction 1:
CAFFE_ROUND_DOWN: B >= F is an error if (B - S) < F

CAFFE_ROUND_UP: (B + S) >= (F + 1) is an error if B < (F + 1)
Restriction 2:
CAFFE_ROUND_DOWN: (B - S) >= F is an error if B >= F

CAFFE_ROUND_UP: B >= (F + 1) is an error if (B + S) >= (F + 1)

Enumerator
kEXPLICIT_ROUND_DOWN	Use explicit padding, rounding output size down.
kEXPLICIT_ROUND_UP	Use explicit padding, rounding output size up.
kSAME_UPPER	Use SAME padding, with prePadding <= postPadding.
kSAME_LOWER	Use SAME padding, with prePadding >= postPadding.
kCAFFE_ROUND_DOWN	Use CAFFE padding, rounding output size down, uses prePadding value.
kCAFFE_ROUND_UP	Use CAFFE padding, rounding output size up, uses prePadding value.

◆ PoolingType

enum nvinfer1::PoolingType : int32_t

strong

The type of pooling to perform in a pooling layer.

Enumerator
kMAX
kAVERAGE
kMAX_AVERAGE_BLEND

◆ ScaleMode

enum nvinfer1::ScaleMode : int32_t

strong

Controls how shift, scale and power are applied in a Scale layer.

See also: IScaleLayer

Enumerator
kUNIFORM	Identical coefficients across all elements of the tensor.
kCHANNEL	Per-channel coefficients.
kELEMENTWISE	Elementwise coefficients.

◆ ElementWiseOperation

enum nvinfer1::ElementWiseOperation : int32_t

strong

Enumerates the binary operations that may be performed by an ElementWise layer.

See also: IElementWiseLayer

Enumerator
kSUM	Sum of the two elements.
kPROD	Product of the two elements.
kMAX	Maximum of the two elements.
kMIN	Minimum of the two elements.
kSUB	Substract the second element from the first.
kDIV	Divide the first element by the second.
kPOW	The first element to the power of the second element.
kFLOOR_DIV	Floor division of the first element by the second.
kAND	Logical AND of two elements.
kOR	Logical OR of two elements.
kXOR	Logical XOR of two elements.
kEQUAL	Check if two elements are equal.
kGREATER	Check if element in first tensor is greater than corresponding element in second tensor.
kLESS	Check if element in first tensor is less than corresponding element in second tensor.

◆ RNNOperation

enum nvinfer1::RNNOperation : int32_t

strong

Enumerates the RNN operations that may be performed by an RNN layer.

Equation definitions

In the equations below, we use the following naming convention:

t := current time step
 
i := input gate
o := output gate
f := forget gate
z := update gate
r := reset gate
c := cell gate
h := hidden gate
 
g[t] denotes the output of gate g at timestep t, e.g.
f[t] is the output of the forget gate f.
 
X[t] := input tensor for timestep t
C[t] := cell state for timestep t
H[t] := hidden state for timestep t
 
W[g] := W (input) parameter weight matrix for gate g
R[g] := U (recurrent) parameter weight matrix for gate g
Wb[g] := W (input) parameter bias vector for gate g
Rb[g] := U (recurrent) parameter bias vector for gate g
 
Unless otherwise specified, all operations apply pointwise
to elements of each operand tensor.
 
ReLU(X) := max(X, 0)
tanh(X) := hyperbolic tangent of X
sigmoid(X) := 1 / (1 + exp(-X))
exp(X) := e^X
 
A.B denotes matrix multiplication of A and B.
A*B denotes pointwise multiplication of A and B.

Equations

Depending on the value of RNNOperation chosen, each sub-layer of the RNN layer will perform one of the following operations:

::kRELU
 
  H[t] := ReLU(W[i].X[t] + R[i].H[t-1] + Wb[i] + Rb[i])
 
::kTANH
 
  H[t] := tanh(W[i].X[t] + R[i].H[t-1] + Wb[i] + Rb[i])
 
::kLSTM
 
  i[t] := sigmoid(W[i].X[t] + R[i].H[t-1] + Wb[i] + Rb[i])
  f[t] := sigmoid(W[f].X[t] + R[f].H[t-1] + Wb[f] + Rb[f])
  o[t] := sigmoid(W[o].X[t] + R[o].H[t-1] + Wb[o] + Rb[o])
  c[t] :=    tanh(W[c].X[t] + R[c].H[t-1] + Wb[c] + Rb[c])
 
  C[t] := f[t]*C[t-1] + i[t]*c[t]
  H[t] := o[t]*tanh(C[t])
 
::kGRU
 
  z[t] := sigmoid(W[z].X[t] + R[z].H[t-1] + Wb[z] + Rb[z])
  r[t] := sigmoid(W[r].X[t] + R[r].H[t-1] + Wb[r] + Rb[r])
  h[t] := tanh(W[h].X[t] + r[t]*(R[h].H[t-1] + Rb[h]) + Wb[h])
 
  H[t] := (1 - z[t])*h[t] + z[t]*H[t-1]

See also: IRNNLayer, IRNNv2Layer

Enumerator
kRELU	Single gate RNN w/ ReLU activation function.
kTANH	Single gate RNN w/ TANH activation function.
kLSTM	Four-gate LSTM network w/o peephole connections.
kGRU	Three-gate network consisting of Gated Recurrent Units.

◆ RNNDirection

enum nvinfer1::RNNDirection : int32_t

strong

Enumerates the RNN direction that may be performed by an RNN layer.

See also: IRNNLayer, IRNNv2Layer

Enumerator
kUNIDIRECTION	Network iterations from first input to last input.
kBIDIRECTION	Network iterates from first to last and vice versa and outputs concatenated.

◆ RNNInputMode

enum nvinfer1::RNNInputMode : int32_t

strong

Enumerates the RNN input modes that may occur with an RNN layer.

If the RNN is configured with RNNInputMode::kLINEAR, then for each gate g in the first layer of the RNN, the input vector X[t] (length E) is left-multiplied by the gate's corresponding weight matrix W[g] (dimensions HxE) as usual, before being used to compute the gate output as described by RNNOperation.

If the RNN is configured with RNNInputMode::kSKIP, then this initial matrix multiplication is "skipped" and W[g] is conceptually an identity matrix. In this case, the input vector X[t] must have length H (the size of the hidden state).

See also: IRNNLayer, IRNNv2Layer

Enumerator
kLINEAR	Perform the normal matrix multiplication in the first recurrent layer.
kSKIP	No operation is performed on the first recurrent layer.

◆ RNNGateType

enum nvinfer1::RNNGateType : int32_t

strong

Identifies an individual gate within an RNN cell.

See also: RNNOperation

Enumerator
kINPUT	Input gate (i).
kOUTPUT	Output gate (o).
kFORGET	Forget gate (f).
kUPDATE	Update gate (z).
kRESET	Reset gate (r).
kCELL	Cell gate (c).
kHIDDEN	Hidden gate (h).

◆ UnaryOperation

enum nvinfer1::UnaryOperation : int32_t

strong

Enumerates the unary operations that may be performed by a Unary layer.

See also: IUnaryLayer

Enumerator
kEXP	Exponentiation.
kLOG	Log (base e).
kSQRT	Square root.
kRECIP	Reciprocal.
kABS	Absolute value.
kNEG	Negation.
kSIN	Sine.
kCOS	Cosine.
kTAN	Tangent.
kSINH	Hyperbolic sine.
kCOSH	Hyperbolic cosine.
kASIN	Inverse sine.
kACOS	Inverse cosine.
kATAN	Inverse tangent.
kASINH	Inverse hyperbolic sine.
kACOSH	Inverse hyperbolic cosine.
kATANH	Inverse hyperbolic tangent.
kCEIL	Ceiling.
kFLOOR	Floor.
kERF	Gauss error function.
kNOT	Logical NOT.

◆ ReduceOperation

enum nvinfer1::ReduceOperation : int32_t

strong

Enumerates the reduce operations that may be performed by a Reduce layer.

The table shows the result of reducing across an empty volume of a given type.

Operation	kFLOAT and kHALF	kINT32	kINT8
kSUM	0	0	0
kPROD	1	1	1
kMAX	negative infinity	INT_MIN	-128
kMIN	positive infinity	INT_MAX	127
kAVG	NaN	0	-128

The current version of TensorRT usually performs reduction for kINT8 via kFLOAT or kHALF. The kINT8 values show the quantized representations of the floating-point values.

Enumerator
kSUM
kPROD
kMAX
kMIN
kAVG

◆ SliceMode

enum nvinfer1::SliceMode : int32_t

strong

Controls how ISliceLayer handles out of bounds coordinates.

See also: ISliceLayer

Enumerator
kDEFAULT	Fail with error when the coordinates are out of bounds. This is the default.
kWRAP	Coordinates wrap around periodically.

◆ TopKOperation

enum nvinfer1::TopKOperation : int32_t

strong

Enumerates the operations that may be performed by a TopK layer.

Enumerator
kMAX	Maximum of the elements.
kMIN	Minimum of the elements.

◆ MatrixOperation

enum nvinfer1::MatrixOperation : int32_t

strong

Enumerates the operations that may be performed on a tensor by IMatrixMultiplyLayer before multiplication.

Enumerator

kNONE

Treat x as a matrix if it has two dimensions, or as a collection of matrices if x has more than two dimensions, where the last two dimensions are the matrix dimensions.

x must have at least two dimensions.

kTRANSPOSE

Like kNONE, but transpose the matrix dimensions.

kVECTOR

Treat x as a vector if it has one dimension, or as a collection of vectors if x has more than one dimension.

x must have at least one dimension. The first input tensor with dimensions [M,K] used with MatrixOperation::kVECTOR is equivalent to a tensor with dimensions [M, 1, K] with MatrixOperation::kNONE, i.e. is treated as M row vectors of length K. If MatrixOperation::kTRANSPOSE is specified, then the dimensions are [M, K, 1].

The second input tensor with dimensions [M,K] used with MatrixOperation::kVECTOR is equivalent to a tensor with dimensions [M, K, 1] with MatrixOperation::kNONE, i.e. is treated as M column vectors of length K. If MatrixOperation::kTRANSPOSE is specified, then the dimensions are [M, 1, K].

◆ ResizeMode

enum nvinfer1::ResizeMode : int32_t

strong

Enumerates various modes of resize in the resize layer. Resize mode set using setResizeMode().

Enumerator
kNEAREST	ND (0 < N <= 8) nearest neighbor resizing.
kLINEAR	Can handle linear (1D), bilinear (2D), and trilinear (3D) resizing.

◆ LoopOutput

enum nvinfer1::LoopOutput : int32_t

strong

Enum that describes kinds of loop outputs.

Enumerator
kLAST_VALUE	Output value is value of tensor for last iteration.
kCONCATENATE	Output value is concatenation of values of tensor for each iteration, in forward order.
kREVERSE	Output value is concatenation of values of tensor for each iteration, in reverse order.

◆ TripLimit

enum nvinfer1::TripLimit : int32_t

strong

Enum that describes kinds of trip limits.

Enumerator
kCOUNT	Tensor is scalar of type kINT32 that contains the trip count.
kWHILE	Tensor is a scalar of type kBOOL. Loop terminates when value is false.

◆ FillOperation

enum nvinfer1::FillOperation : int32_t

strong

Enumerates the tensor fill operations that may performed by a fill layer.

See also: IFillLayer

Enumerator
kLINSPACE	Generate evenly spaced numbers over a specified interval.
kRANDOM_UNIFORM	Generate a tensor with random values drawn from a uniform distribution.

◆ CalibrationAlgoType

enum nvinfer1::CalibrationAlgoType : int32_t

strong

enum CalibrationAlgoType

Version of calibration algorithm to use.

Enumerator
kLEGACY_CALIBRATION
kENTROPY_CALIBRATION
kENTROPY_CALIBRATION_2
kMINMAX_CALIBRATION

◆ QuantizationFlag

enum nvinfer1::QuantizationFlag : int32_t

strong

List of valid flags for quantizing the network to int8.

See also: IBuilderConfig::setQuantizationFlag(), IBuilderConfig::getQuantizationFlag()

Enumerator

kCALIBRATE_BEFORE_FUSION

Run int8 calibration pass before layer fusion.

Only valid for IInt8LegacyCalibrator and IInt8EntropyCalibrator. We always run int8 calibration pass before layer fusion for IInt8MinMaxCalibrator and IInt8EntropyCalibrator2. Disabled by default.

◆ BuilderFlag

enum nvinfer1::BuilderFlag : int32_t

strong

List of valid modes that the builder can enable when creating an engine from a network definition.

See also: IBuilderConfig::setFlag(), IBuilderConfig::getFlag()

Enumerator
kFP16	Enable FP16 layer selection, with FP32 fallback.
kINT8	Enable Int8 layer selection, with FP32 fallback with FP16 fallback if kFP16 also specified.
kDEBUG	Enable debugging of layers via synchronizing after every layer.
kGPU_FALLBACK	Enable layers marked to execute on GPU if layer cannot execute on DLA.
kSTRICT_TYPES	Enables strict type constraints.
kREFIT	Enable building a refittable engine.
kDISABLE_TIMING_CACHE	Disable reuse of timing information across identical layers.
kTF32	Allow (but not require) computations on tensors of type DataType::kFLOAT to use TF32. TF32 computes inner products by rounding the inputs to 10-bit mantissas before multiplying, but accumulates the sum using 23-bit mantissas. Enabled by default.

◆ ProfilingVerbosity

enum nvinfer1::ProfilingVerbosity : int32_t

strong

List of verbosity levels of layer information exposed in NVTX annotations.

See also: IBuilderConfig::setProfilingVerbosity(), IBuilderConfig::getProfilingVerbosity()

Enumerator
kDEFAULT	Register layer names in NVTX message field.
kNONE	Turn off NVTX traces.
kVERBOSE	Register layer names in NVTX message field and register layer detail in NVTX JSON payload field.

◆ TacticSource

enum nvinfer1::TacticSource : int32_t

strong

List of tactic sources for TensorRT.

See also: TacticSources, IBuilderConfig::setTacticSources(), IBuilderConfig::getTacticSources()

Enumerator

kCUBLAS

cuBLAS tactics.

Note: Disabling kCUBLAS will cause the cublas handle passed to plugins in attachToContext to be null.

kCUBLAS_LT

cuBLAS LT tactics

◆ NetworkDefinitionCreationFlag

enum nvinfer1::NetworkDefinitionCreationFlag : int32_t

strong

List of immutable network properties expressed at network creation time. NetworkDefinitionCreationFlag is used with createNetworkV2 to specify immutable properties of the network. The createNetwork() function always had an implicit batch dimension being specified by the maxBatchSize builder parameter. createNetworkV2 with kDEFAULT flag mimics that behaviour.

See also: IBuilder::createNetworkV2

Enumerator

kEXPLICIT_BATCH

Dynamic shape support requires that the kEXPLICIT_BATCH flag is set.

With dynamic shapes, any of the input dimensions can vary at run-time, and there are no implicit dimensions in the network specification. This is specified by using the wildcard dimension value -1. Mark the network to be an explicit batch network

kEXPLICIT_PRECISION

Setting the network to be an explicit precision network has the following implications: 1) Precision of all input tensors to the network have to be specified with ITensor::setType() function 2) Precision of all layer output tensors in the network have to be specified using ILayer::setOutputType() function 3) The builder will not quantize the weights of any layer including those running in lower precision(INT8).

It will simply cast the weights into the required precision. 4) Dynamic ranges must not be provided to run the network in int8 mode. Dynamic ranges of each tensor in the explicit precision network is [-127,127]. 5) Quantizing and dequantizing activation values between higher (FP32) and lower (INT8) precision will be performed using explicit Scale layers with input/output precision set appropriately. Mark the network to be an explicit precision network

◆ PluginType

enum nvinfer1::PluginType : int32_t

strong

The type values for the various plugins.

See also: INvPlugin::getPluginType()

Enumerator
kFASTERRCNN	FasterRCNN fused plugin (RPN + ROI pooling).
kNORMALIZE	Normalize plugin.
kPERMUTE	Permute plugin.
kPRIORBOX	PriorBox plugin.
kSSDDETECTIONOUTPUT	SSD DetectionOutput plugin.
kCONCAT	Concat plugin.
kPRELU	YOLO PReLU Plugin.
kYOLOREORG	YOLO Reorg Plugin.
kYOLOREGION	YOLO Region Plugin.
kANCHORGENERATOR	SSD Grid Anchor Generator.

◆ EngineCapability

enum nvinfer1::EngineCapability : int32_t

strong

Forward declaration of IPluginFactory for use by other interfaces.

List of supported engine capability flows.

The EngineCapability determines the restrictions of a network during build time for what can be executed at runtime. EngineCapability::kDEFAULT does not provide any restrictions on functionality and the resulting serialized engine can be executed with TensorRT's standard runtime APIs in the nvinfer1 namespace. EngineCapabiltiy::kSAFE_GPU provides a restricted subset of network operations that are safety certified and the resulting serialized engine can be executed with TensorRT's safe runtime APIs in the nvinfer1::safe namespace. EngineCapability::kSAFE_DLA provides a restricted subset of network operations that are DLA compatible and the resulting serialized engine can be executed using NvMediaDLA's runtime APIs. See sampleNvmedia for an example of integrating NvMediaDLA APIs with TensorRT APIs.

Enumerator
kDEFAULT	Full capability, TensorRT mode without any restrictions using TensorRT nvinfer1 APIs.
kSAFE_GPU	Safety restricted capability, TensorRT flow that can only run on GPU devices via TensorRT nvinfer1::safe APIs.
kSAFE_DLA	Safety restricted capability, TensorRT flow that can only run on DLA devices via NvMediaDLA APIs.

◆ DimensionOperation

enum nvinfer1::DimensionOperation : int32_t

strong

An operation on two IDimensionExpr, which represent integer expressions used in dimension computations.

For example, given two IDimensionExpr x and y and an IExprBuilder& eb, eb.operation(DimensionOperation::kSUM, x, y) creates a representation of x+y.

See also: IDimensionExpr, IExprBuilder

Enumerator
kSUM	Sum of the two operands.
kPROD	Product of the two operands.
kMAX	Maximum of the two operands.
kMIN	Minimum of the two operands.
kSUB	Substract the second element from the first.
kEQUAL	1 if operands are equal, 0 otherwise.
kLESS	1 if first operand is less than second operand, 0 otherwise.
kFLOOR_DIV	Floor division of the first element by the second.
kCEIL_DIV	Division rounding up.

◆ WeightsRole

enum nvinfer1::WeightsRole : int32_t

strong

How a layer uses particular Weights.

The power weights of an IScaleLayer are omitted. Refitting those is not supported.

Enumerator
kKERNEL	kernel for IConvolutionLayer, IDeconvolutionLayer, or IFullyConnectedLayer
kBIAS	bias for IConvolutionLayer, IDeconvolutionLayer, or IFullyConnectedLayer
kSHIFT	shift part of IScaleLayer
kSCALE	scale part of IScaleLayer
kCONSTANT	weights for IConstantLayer

◆ DeviceType

enum nvinfer1::DeviceType : int32_t

strong

The device that this layer/network will execute on.

Enumerator
kGPU	GPU Device.
kDLA	DLA Core.

◆ OptProfileSelector

enum nvinfer1::OptProfileSelector : int32_t

strong

When setting or querying optimization profile parameters (such as shape tensor inputs or dynamic dimensions), select whether we are interested in the minimum, optimum, or maximum values for these parameters. The minimum and maximum specify the permitted range that is supported at runtime, while the optimum value is used for the kernel selection. This should be the "typical" value that is expected to occur at runtime.

See also: IOptimizationProfile::setDimensions(), IOptimizationProfile::setShapeValues()

Enumerator
kMIN	This is used to set or get the minimum permitted value for dynamic dimensions etc.
kOPT	This is used to set or get the value that is used in the optimization (kernel selection).
kMAX	This is used to set or get the maximum permitted value for dynamic dimensions etc.

◆ ActivationType

enum nvinfer1::ActivationType : int32_t

strong

Enumerates the types of activation to perform in an activation layer.

Enumerator
kRELU	Rectified linear activation.
kSIGMOID	Sigmoid activation.
kTANH	TanH activation.
kLEAKY_RELU	LeakyRelu activation: x>=0 ? x : alpha * x.
kELU	Elu activation: x>=0 ? x : alpha * (exp(x) - 1).
kSELU	Selu activation: x>0 ? beta * x : beta * (alpha*exp(x) - alpha)
kSOFTSIGN	Softsign activation: x / (1+\|x\|)
kSOFTPLUS	Parametric softplus activation: alphalog(exp(betax)+1)
kCLIP	Clip activation: max(alpha, min(beta, x))
kHARD_SIGMOID	Hard sigmoid activation: max(0, min(1, alpha*x+beta))
kSCALED_TANH	Scaled tanh activation: alphatanh(betax)
kTHRESHOLDED_RELU	Thresholded ReLU activation: x>alpha ? x : 0.

◆ DataType

enum nvinfer1::DataType : int32_t

strong

The type of weights and tensors.

Enumerator
kFLOAT	32-bit floating point format.
kHALF	IEEE 16-bit floating-point format.
kINT8	8-bit integer representing a quantized floating-point value.
kINT32	Signed 32-bit integer format.
kBOOL	8-bit boolean. 0 = false, 1 = true, other values undefined.

◆ DimensionType

enum nvinfer1::DimensionType : int32_t

strong

The type of data encoded across this dimension.

Enumerator
kSPATIAL	Elements correspond to different spatial data.
kCHANNEL	Elements correspond to different channels.
kINDEX	Elements correspond to different batch index.
kSEQUENCE	Elements correspond to different sequence values.

◆ TensorFormat

enum nvinfer1::TensorFormat : int32_t

strong

Format of the input/output tensors.

This enum is extended to be used by both plugins and reformat-free network I/O tensors.

See also: IPluginExt::getPluginFormats(), safe::ICudaEngine::getBindingFormat()

For more information about data formats, see the topic "Data Format Description" located in the TensorRT Developer Guide (https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html).

Enumerator
kLINEAR	Row major linear format. For a tensor with dimensions {N, C, H, W}, the W axis always has unit stride, and the stride of every other axis is at least the the product of of the next dimension times the next stride. the strides are the same as for a C array with dimensions [N][C][H][W].
kNCHW	Deprecated name of kLINEAR, provided for backwards compatibility and will be removed in TensorRT 8.0.
kCHW2	Two wide channel vectorized row major format. This format is bound to FP16. It is only available for dimensions >= 3. For a tensor with dimensions {N, C, H, W}, the memory layout is equivalent to a C array with dimensions [N][(C+1)/2][H][W][2], with the tensor coordinates (n, c, h, w) mapping to array subscript [n][c/2][h][w][c%2].
kNC2HW2	Deprecated name of kCHW2, provided for backwards compatibility and will be removed in TensorRT 8.0.
kHWC8	Eight channel format where C is padded to a multiple of 8. This format is bound to FP16. It is only available for dimensions >= 3. For a tensor with dimensions {N, C, H, W}, the memory layout is equivalent to the array with dimensions [N][H][W][(C+7)/8*8], with the tensor coordinates (n, c, h, w) mapping to array subscript [n][h][w][c].
kNHWC8	Deprecated name of kHWC8, provided for backwards compatibility and will be removed in TensorRT 8.0.
kCHW4	Four wide channel vectorized row major format. This format is bound to INT8 or FP16. It is only available for dimensions >= 3. For INT8, the C dimension must be a build-time constant. For a tensor with dimensions {N, C, H, W}, the memory layout is equivalent to a C array with dimensions [N][(C+3)/4][H][W][4], with the tensor coordinates (n, c, h, w) mapping to array subscript [n][c/4][h][w][c%4]. Deprecated usage: If running on the DLA, this format can be used for acceleration with the caveat that C must be equal or lesser than 4. If used as DLA input with allowGPUFallback disable, it needs to meet line stride requirement of DLA format. Column stride in bytes should be multiple of 32.
kCHW16	Sixteen wide channel vectorized row major format. This format is bound to FP16. It is only available for dimensions >= 3. For a tensor with dimensions {N, C, H, W}, the memory layout is equivalent to a C array with dimensions [N][(C+15)/16][H][W][16], with the tensor coordinates (n, c, h, w) mapping to array subscript [n][c/16][h][w][c%16]. For DLA usage, this format maps to the native format for FP16, and the tensor sizes are limited to C,H,W in the range [1,8192].
kCHW32	Thirty-two wide channel vectorized row major format. This format is only available for dimensions >= 3. For a tensor with dimensions {N, C, H, W}, the memory layout is equivalent to a C array with dimensions [N][(C+31)/32][H][W][32], with the tensor coordinates (n, c, h, w) mapping to array subscript [n][c/32][h][w][c%32]. For DLA usage, this format maps to the native format for INT8, and the tensor sizes are limited to C,H,W in the range [1,8192].
kDHWC8	Eight channel format where C is padded to a multiple of 8. This format is bound to FP16, and it is only available for dimensions >= 4. For a tensor with dimensions {N, C, D, H, W}, the memory layout is equivalent to an array with dimensions [N][D][H][W][(C+7)/8*8], with the tensor coordinates (n, c, d, h, w) mapping to array subscript [n][d][h][w][c].
kCDHW32	Thirty-two wide channel vectorized row major format. This format is bound to FP16 and INT8 and is only available for dimensions >= 4. For a tensor with dimensions {N, C, D, H, W}, the memory layout is equivalent to a C array with dimensions [N][(C+31)/32][D][H][W][32], with the tensor coordinates (n, c, d, h, w) mapping to array subscript [n][c/32][d][h][w][c%32].
kHWC	Non-vectorized channel-last format. This format is bound to FP32 and is only available for dimensions >= 3.
kDLA_LINEAR	DLA planar format. For a tensor with dimension {N, C, H, W}, the W axis always has unit stride. The stride for stepping along the H axis is rounded up to 64 bytes. The memory layout is equivalent to a C array with dimensions [N][C][H][roundUp(W, 64/elementSize)] where elementSize is 2 for FP16 and 1 for Int8, with the tensor coordinates (n, c, h, w) mapping to array subscript [n][c][h][w].
kDLA_HWC4	DLA image format. For a tensor with dimension {N, C, H, W} the C axis always has unit stride. The stride for stepping along the H axis is rounded up to 32 bytes. C can only be 1, 3 or 4. If C == 1, it will map to grayscale format. If C == 3 or C == 4, it will map to color image format. And if C == 3, the stride for stepping along the W axis needs to be padded to 4 in elements. When C is {1, 3, 4}, then C' is {1, 4, 4} respectively, the memory layout is equivalent to a C array with dimensions [N][H][roundUp(W, 32/C'/elementSize)][C'] where elementSize is 2 for FP16 and 1 for Int8. The tensor coordinates (n, c, h, w) mapping to array subscript [n][h][w][c].

◆ PluginVersion

enum nvinfer1::PluginVersion : uint8_t

strong

Enumerator
kV2
kV2_EXT	IPluginV2.
kV2_IOEXT	IPluginV2Ext.
kV2_DYNAMICEXT	IPluginV2IOExt.

◆ PluginFieldType

enum nvinfer1::PluginFieldType : int32_t

strong

Enumerator
kFLOAT16	FP16 field type.
kFLOAT32	FP32 field type.
kFLOAT64	FP64 field type.
kINT8	INT8 field type.
kINT16	INT16 field type.
kINT32	INT32 field type.
kCHAR	char field type.
kDIMS	nvinfer1::Dims field type.
kUNKNOWN

◆ TensorLocation

enum nvinfer1::TensorLocation : int32_t

strong

The location for tensor data storage, device or host.

Enumerator
kDEVICE	Data stored on device.
kHOST	Data stored on host.

◆ ErrorCode

enum nvinfer1::ErrorCode : int32_t

strong

Error codes that can be returned by TensorRT during execution.

the type of parser error

Enumerator
kSUCCESS	Execution completed successfully.
kUNSPECIFIED_ERROR	An error that does not fall into any other category. This error is included for forward compatibility
kINTERNAL_ERROR	A non-recoverable TensorRT error occurred.
kINVALID_ARGUMENT	An argument passed to the function is invalid in isolation. This is a violation of the API contract
kINVALID_CONFIG	An error occurred when comparing the state of an argument relative to other arguments. For example, the dimensions for concat differ between two tensors outside of the channel dimension. This error is triggered when an argument is correct in isolation, but not relative to other arguments. This is to help to distinguish from the simple errors from the more complex errors. This is a violation of the API contract.
kFAILED_ALLOCATION	An error occurred when performing an allocation of memory on the host or the device. A memory allocation error is normally fatal, but in the case where the application provided its own memory allocation routine, it is possible to increase the pool of available memory and resume execution.
kFAILED_INITIALIZATION	One, or more, of the components that TensorRT relies on did not initialize correctly. This is a system setup issue.
kFAILED_EXECUTION	An error occurred during execution that caused TensorRT to end prematurely, either an asynchronous error or other execution errors reported by CUDA/DLA. In a dynamic system, the data can be thrown away and the next frame can be processed or execution can be retried. This is either an execution error or a memory error.
kFAILED_COMPUTATION	An error occurred during execution that caused the data to become corrupted, but execution finished. Examples of this error are NaN squashing or integer overflow. In a dynamic system, the data can be thrown away and the next frame can be processed or execution can be retried. This is either a data corruption error, an input error, or a range error.
kINVALID_STATE	TensorRT was put into a bad state by incorrect sequence of function calls. An example of an invalid state is specifying a layer to be DLA only without GPU fallback, and that layer is not supported by DLA. This can occur in situations where a service is optimistically executing networks for multiple different configurations without checking proper error configurations, and instead throwing away bad configurations caught by TensorRT. This is a violation of the API contract, but can be recoverable. Example of a recovery: GPU fallback is disabled and conv layer with large filter(63x63) is specified to run on DLA. This will fail due to DLA not supporting the large kernel size. This can be recovered by either turning on GPU fallback or setting the layer to run on the GPU.
kUNSUPPORTED_STATE	An error occurred due to the network not being supported on the device due to constraints of the hardware or system. An example is running a unsafe layer in a safety certified context, or a resource requirement for the current network is greater than the capabilities of the target device. The network is otherwise correct, but the network and hardware combination is problematic. This can be recoverable. Examples: Scratch space requests larger than available device memory and can be recovered by increasing allowed workspace size. Tensor size exceeds the maximum element count and can be recovered by reducing the maximum batch size.

See also: RNNGateType

◆ attribute()

class nvinfer1::__attribute__ ( (deprecated) )

Application-implemented interface to compute the HW output dimensions of a layer from the layer input and parameters.

Parameters

inputDims	The input dimensions of the layer.
kernelSize	The kernel size (or window size, for a pooling layer) parameter of the layer operation.
stride	The stride parameter for the layer.
padding	The padding parameter of the layer.
dilation	The dilation parameter of the layer (only applicable to convolutions).
layerName	The name of the layer.

Returns: The output size of the layer

Note that for dilated convolutions, the dilation is applied to the kernel size before this routine is called.

◆ EnumMax< UnaryOperation >()

template<>

constexpr int32_t nvinfer1::EnumMax< UnaryOperation > ( )

inlineconstexpr

Maximum number of elements in UnaryOperation enum.

See also: UnaryOperation

◆ EnumMax< ReduceOperation >()

template<>

constexpr int32_t nvinfer1::EnumMax< ReduceOperation > ( )

inlineconstexpr

Maximum number of elements in ReduceOperation enum.

See also: ReduceOperation

◆ EnumMax< SliceMode >()

template<>

constexpr int32_t nvinfer1::EnumMax< SliceMode > ( )

inlineconstexpr

Maximum number of elements in SliceMode enum.

See also: SliceMode

◆ EnumMax< TopKOperation >()

template<>

constexpr int32_t nvinfer1::EnumMax< TopKOperation > ( )

inlineconstexpr

Maximum number of elements in TopKOperation enum.

See also: TopKOperation

◆ EnumMax< MatrixOperation >()

template<>

constexpr int32_t nvinfer1::EnumMax< MatrixOperation > ( )

inlineconstexpr

Maximum number of elements in MatrixOperation enum.

See also: DataType

◆ EnumMax< ResizeMode >()

template<>

constexpr int32_t nvinfer1::EnumMax< ResizeMode > ( )

inlineconstexpr

Maximum number of elements in ResizeMode enum.

See also: ResizeMode

◆ EnumMax< LoopOutput >()

template<>

constexpr int32_t nvinfer1::EnumMax< LoopOutput > ( )

inlineconstexpr

Maximum number of elements in LoopOutput enum.

See also: DataType

◆ EnumMax< TripLimit >()

template<>

constexpr int32_t nvinfer1::EnumMax< TripLimit > ( )

inlineconstexpr

Maximum number of elements in TripLimit enum.

See also: DataType

◆ EnumMax< FillOperation >()

template<>

constexpr int32_t nvinfer1::EnumMax< FillOperation > ( )

inlineconstexpr

Maximum number of elements in FillOperation enum.

See also: FillOperation

◆ EnumMax< CalibrationAlgoType >()

template<>

constexpr int32_t nvinfer1::EnumMax< CalibrationAlgoType > ( )

inlineconstexpr

Maximum number of elements in CalibrationAlgoType enum.

See also: DataType

◆ EnumMax< QuantizationFlag >()

template<>

constexpr int32_t nvinfer1::EnumMax< QuantizationFlag > ( )

inlineconstexpr

Maximum number of quantization flags in QuantizationFlag enum.

See also: QuantizationFlag

◆ EnumMax< BuilderFlag >()

template<>

constexpr int32_t nvinfer1::EnumMax< BuilderFlag > ( )

inlineconstexpr

Maximum number of builder flags in BuilderFlag enum.

See also: BuilderFlag

◆ EnumMax< ProfilingVerbosity >()

template<>

constexpr int32_t nvinfer1::EnumMax< ProfilingVerbosity > ( )

inlineconstexpr

Maximum number of profile verbosity levels in ProfilingVerbosity enum.

See also: ProfilingVerbosity

◆ EnumMax< TacticSource >()

template<>

constexpr int32_t nvinfer1::EnumMax< TacticSource > ( )

inlineconstexpr

Maximum number of tactic sources in TacticSource enum.

See also: TacticSource

◆ EnumMax< NetworkDefinitionCreationFlag >()

template<>

constexpr int32_t nvinfer1::EnumMax< NetworkDefinitionCreationFlag > ( )

inlineconstexpr

Maximum number of elements in NetworkDefinitionCreationFlag enum.

See also: NetworkDefinitionCreationFlag

◆ EnumMax< PluginType >()

template<>

constexpr int32_t nvinfer1::EnumMax< PluginType > ( )

inlineconstexpr

Maximum number of elements in PluginType enum.

See also: PluginType

◆ EnumMax< EngineCapability >()

template<>

constexpr int32_t nvinfer1::EnumMax< EngineCapability > ( )

inlineconstexpr

Maximum number of elements in EngineCapability enum.

See also: EngineCapability

◆ EnumMax< DimensionOperation >()

template<>

constexpr int32_t nvinfer1::EnumMax< DimensionOperation > ( )

inlineconstexpr

Maximum number of elements in DimensionOperation enum.

See also: DimensionOperation

◆ EnumMax< WeightsRole >()

template<>

constexpr int32_t nvinfer1::EnumMax< WeightsRole > ( )

inlineconstexpr

Maximum number of elements in WeightsRole enum.

See also: WeightsRole

◆ EnumMax< DeviceType >()

template<>

constexpr int32_t nvinfer1::EnumMax< DeviceType > ( )

inlineconstexpr

Maximum number of elements in DeviceType enum.

See also: DeviceType

◆ EnumMax< OptProfileSelector >()

template<>

constexpr int32_t nvinfer1::EnumMax< OptProfileSelector > ( )

inlineconstexpr

◆ EnumMax()

template<typename T >

constexpr int32_t nvinfer1::EnumMax ( )

inlineconstexpr

Forward declare IGpuAllocator for use in other interfaces.

Maximum number of elements in an enumeration type.

◆ EnumMax< ActivationType >()

template<>

constexpr int32_t nvinfer1::EnumMax< ActivationType > ( )

inlineconstexpr

Maximum number of elements in ActivationType enum.

See also: ActivationType

◆ EnumMax< DataType >()

template<>

constexpr int32_t nvinfer1::EnumMax< DataType > ( )

inlineconstexpr

Maximum number of elements in DataType enum.

See also: DataType

◆ EnumMax< DimensionType >()

template<>

constexpr int32_t nvinfer1::EnumMax< DimensionType > ( )

inlineconstexpr

Maximum number of elements in DimensionType enum.

See also: DimensionType

◆ EnumMax< TensorFormat >()

template<>

constexpr int32_t nvinfer1::EnumMax< TensorFormat > ( )

inlineconstexpr

Maximum number of elements in TensorFormat enum.

See also: TensorFormat

◆ EnumMax< TensorLocation >()

template<>

constexpr int32_t nvinfer1::EnumMax< TensorLocation > ( )

inlineconstexpr

Maximum number of elements in TensorLocation enum.

See also: TensorLocation

◆ EnumMax< ILogger::Severity >()

template<>

constexpr int32_t nvinfer1::EnumMax< ILogger::Severity > ( )

inlineconstexpr

Maximum number of elements in ILogger::Severity enum.

See also: ILogger::Severity

◆ EnumMax< ErrorCode >()

template<>

constexpr int32_t nvinfer1::EnumMax< ErrorCode > ( )

inlineconstexpr

Maximum number of elements in ErrorCode enum.

See also: ErrorCode

◆ cuErrCheck_()

void nvinfer1::cuErrCheck_	(	CUresult	stat,
		const CUDADriverWrapper &	wrap,
		const char *	file,
		int	line
	)

inline

Here is the call graph for this function:

Namespaces

Classes

Typedefs

Enumerations

Functions

Detailed Description

Typedef Documentation

◆ QuantizationFlags

◆ BuilderFlags

◆ TacticSources

◆ NetworkDefinitionCreationFlags

◆ TensorFormats

◆ PluginFormat

Enumeration Type Documentation

◆ LayerType

◆ PaddingMode

◆ PoolingType

◆ ScaleMode

◆ ElementWiseOperation

◆ RNNOperation

◆ RNNDirection

◆ RNNInputMode

◆ RNNGateType

◆ UnaryOperation

◆ ReduceOperation

◆ SliceMode

◆ TopKOperation

◆ MatrixOperation

◆ ResizeMode

◆ LoopOutput

◆ TripLimit

◆ FillOperation

◆ CalibrationAlgoType

◆ QuantizationFlag

◆ BuilderFlag

◆ ProfilingVerbosity

◆ TacticSource

◆ NetworkDefinitionCreationFlag

◆ PluginType

◆ EngineCapability

◆ DimensionOperation

◆ WeightsRole

◆ DeviceType

◆ OptProfileSelector

◆ ActivationType

◆ DataType

◆ DimensionType

◆ TensorFormat

◆ PluginVersion

◆ PluginFieldType

◆ TensorLocation

◆ ErrorCode

Function Documentation

◆ EnumMax< LayerType >()

◆ EnumMax< PaddingMode >()

◆ EnumMax< PoolingType >()

◆ EnumMax< ScaleMode >()

◆ EnumMax< ElementWiseOperation >()

◆ EnumMax< RNNOperation >()

◆ EnumMax< RNNDirection >()

◆ EnumMax< RNNInputMode >()

◆ EnumMax< RNNGateType >()

◆ __attribute__()

◆ EnumMax< UnaryOperation >()

◆ EnumMax< ReduceOperation >()

◆ EnumMax< SliceMode >()

◆ EnumMax< TopKOperation >()

◆ EnumMax< MatrixOperation >()

◆ EnumMax< ResizeMode >()

◆ EnumMax< LoopOutput >()

◆ EnumMax< TripLimit >()

◆ EnumMax< FillOperation >()

◆ EnumMax< CalibrationAlgoType >()

◆ EnumMax< QuantizationFlag >()

◆ EnumMax< BuilderFlag >()

◆ EnumMax< ProfilingVerbosity >()

◆ EnumMax< TacticSource >()

◆ EnumMax< NetworkDefinitionCreationFlag >()

◆ EnumMax< PluginType >()

◆ EnumMax< EngineCapability >()

◆ attribute()