SampleINT8API Class Reference

The SampleINT8API class implements INT8 inference on classification networks. More...

Collaboration diagram for SampleINT8API:

Public Member Functions
	SampleINT8API (const SampleINT8APIParams &params)
sample::Logger::TestResult	build ()
	Builds the network engine. More...
sample::Logger::TestResult	infer ()
	Runs the TensorRT inference engine for this sample. More...
sample::Logger::TestResult	teardown ()
	Used to clean up any state created in the sample class. More...

Public Attributes
SampleINT8APIParams	mParams
	Stores Sample Parameter. More...

Private Types
template<typename T >
using	SampleUniquePtr = std::unique_ptr< T, samplesCommon::InferDeleter >

Private Member Functions
void	getInputOutputNames ()
	Populates input and output mapping of the network. More...
bool	prepareInput (const samplesCommon::BufferManager &buffers)
	Reads the ppm input image, preprocesses, and stores the result in a managed buffer. More...
bool	verifyOutput (const samplesCommon::BufferManager &buffers) const
	Verifies that the output is correct and prints it. More...
bool	readPerTensorDynamicRangeValues ()
	Populate per tensor dynamic range values. More...
bool	setDynamicRange (SampleUniquePtr< nvinfer1::INetworkDefinition > &network)
	Sets custom dynamic range for network tensors. More...
void	setLayerPrecision (SampleUniquePtr< nvinfer1::INetworkDefinition > &network)
	Sets computation precision for network layers. More...
void	writeNetworkTensorNames (const SampleUniquePtr< nvinfer1::INetworkDefinition > &network)
	Write network tensor names to a file. More...

Private Attributes
std::shared_ptr< nvinfer1::ICudaEngine >	mEngine {nullptr}
	The TensorRT engine used to run the network. More...
std::map< std::string, std::string >	mInOut
	Input and output mapping of the network. More...
nvinfer1::Dims	mInputDims
	The dimensions of the input to the network. More...
nvinfer1::Dims	mOutputDims
	The dimensions of the output to the network. More...
std::unordered_map< std::string, float >	mPerTensorDynamicRangeMap
	Mapping from tensor name to max absolute dynamic range values. More...

Detailed Description

The SampleINT8API class implements INT8 inference on classification networks.

INT8 API usage for setting custom int8 range for each input layer. API showcase how to perform INT8 inference without calibration table

Member Typedef Documentation

SampleUniquePtr

template<typename T >

using SampleINT8API::SampleUniquePtr = std::unique_ptr<T, samplesCommon::InferDeleter>

private

Constructor & Destructor Documentation

SampleINT8API()

SampleINT8API::SampleINT8API ( const SampleINT8APIParams &  params )

inline

Member Function Documentation

build()

sample::Logger::TestResult SampleINT8API::build

(

)

Builds the network engine.

Creates the network, configures the builder and creates the network engine.

This function creates INT8 classification network by parsing the onnx model and builds the engine that will be used to run INT8 inference (mEngine)

Returns

Returns true if the engine was created successfully and false otherwise

Here is the call graph for this function:

infer()

sample::Logger::TestResult SampleINT8API::infer

(

)

Runs the TensorRT inference engine for this sample.

This function is the main execution function of the sample. It allocates the buffer, sets inputs, executes the engine, and verifies the output

Here is the call graph for this function:

teardown()

sample::Logger::TestResult SampleINT8API::teardown

(

)

Used to clean up any state created in the sample class.

getInputOutputNames()

void SampleINT8API::getInputOutputNames ( )

private

Populates input and output mapping of the network.

Here is the caller graph for this function:

prepareInput()

bool SampleINT8API::prepareInput ( const samplesCommon::BufferManager &  buffers )

private

Reads the ppm input image, preprocesses, and stores the result in a managed buffer.

Preprocess inputs and allocate host/device input buffers.

Here is the call graph for this function:

Here is the caller graph for this function:

verifyOutput()

bool SampleINT8API::verifyOutput ( const samplesCommon::BufferManager &  buffers ) const

private

Verifies that the output is correct and prints it.

Here is the call graph for this function:

Here is the caller graph for this function:

readPerTensorDynamicRangeValues()

bool SampleINT8API::readPerTensorDynamicRangeValues ( )

private

Populate per tensor dynamic range values.

Populate per tensor dyanamic range values.

Here is the caller graph for this function:

setDynamicRange()

bool SampleINT8API::setDynamicRange ( SampleUniquePtr< nvinfer1::INetworkDefinition > &  network )

private

Sets custom dynamic range for network tensors.

Here is the call graph for this function:

Here is the caller graph for this function:

setLayerPrecision()

void SampleINT8API::setLayerPrecision ( SampleUniquePtr< nvinfer1::INetworkDefinition > &  network )

private

Sets computation precision for network layers.

Here is the call graph for this function:

Here is the caller graph for this function:

writeNetworkTensorNames()

void SampleINT8API::writeNetworkTensorNames ( const SampleUniquePtr< nvinfer1::INetworkDefinition > &  network )

private

Write network tensor names to a file.

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

mParams

SampleINT8APIParams SampleINT8API::mParams

Stores Sample Parameter.

mEngine

std::shared_ptr<nvinfer1::ICudaEngine> SampleINT8API::mEngine {nullptr}

private

The TensorRT engine used to run the network.

mInOut

std::map<std::string, std::string> SampleINT8API::mInOut

private

Input and output mapping of the network.

mInputDims

nvinfer1::Dims SampleINT8API::mInputDims

private

The dimensions of the input to the network.

mOutputDims

nvinfer1::Dims SampleINT8API::mOutputDims

private

The dimensions of the output to the network.

mPerTensorDynamicRangeMap

std::unordered_map<std::string, float> SampleINT8API::mPerTensorDynamicRangeMap

private

Mapping from tensor name to max absolute dynamic range values.

---

The documentation for this class was generated from the following file:

• sampleINT8API.cpp