Inheritance diagram for SampleCharRNNLoop:

Collaboration diagram for SampleCharRNNLoop:

[legend]

Classes
struct	LstmIO

struct	LstmParams

Public Types
template<typename T >
using	SampleUniquePtr = std::unique_ptr< T, samplesCommon::InferDeleter >

Public Member Functions
	SampleCharRNNLoop (SampleCharRNNParams params)

bool	build ()
	Builds the network engine. More...

bool	infer ()
	Runs the TensorRT inference engine for this sample. More...

bool	teardown ()
	Used to clean up any state created in the sample class. More...

Protected Member Functions
nvinfer1::ILayer *	addLSTMLayers (SampleCharRNNBase::SampleUniquePtr< nvinfer1::INetworkDefinition > &network) final
	Add inputs to the TensorRT network and configure LSTM layers using network definition API. More...

nvinfer1::Weights	convertRNNWeights (nvinfer1::Weights input, int dataSize)
	Converts RNN weights from TensorFlow's format to TensorRT's format. More...

nvinfer1::Weights	convertRNNBias (nvinfer1::Weights input)
	Converts RNN Biases from TensorFlow's format to TensorRT's format. More...

nvinfer1::ITensor *	addReshape (SampleUniquePtr< nvinfer1::INetworkDefinition > &network, nvinfer1::ITensor &tensor, nvinfer1::Dims dims)

Protected Attributes
std::map< std::string, nvinfer1::Weights >	mWeightMap

std::vector< SampleUniquePtr< nvinfer1::IHostMemory > >	weightsMemory

SampleCharRNNParams	mParams

Private Member Functions
nvinfer1::ILayer *	addLSTMCell (SampleUniquePtr< nvinfer1::INetworkDefinition > &network, const LstmIO &inputTensors, nvinfer1::ITensor *sequenceSize, const LstmParams &params, LstmIO &outputTensors)

std::map< std::string, nvinfer1::Weights >	loadWeights (const std::string file)
	Load requested weights from a formatted file into a map. More...

void	constructNetwork (SampleUniquePtr< nvinfer1::IBuilder > &builder, SampleUniquePtr< nvinfer1::INetworkDefinition > &network, SampleUniquePtr< nvinfer1::IBuilderConfig > &config)
	Create full model using the TensorRT network definition API and build the engine. More...

void	copyEmbeddingToInput (samplesCommon::BufferManager &buffers, const char &c)
	Looks up the embedding tensor for a given char and copies it to input buffer. More...

bool	stepOnce (samplesCommon::BufferManager &buffers, SampleUniquePtr< nvinfer1::IExecutionContext > &context, cudaStream_t &stream)
	Perform one time step of inference with the TensorRT execution context. More...

void	copyRNNOutputsToInputs (samplesCommon::BufferManager &buffers)
	Copies Ct/Ht output from the RNN to the Ct-1/Ht-1 input buffers for next time step. More...

Private Attributes
std::shared_ptr< nvinfer1::ICudaEngine >	mEngine {nullptr}
	The TensorRT engine used to run the network. More...

Member Typedef Documentation

◆ SampleUniquePtr

template<typename T >

using SampleCharRNNBase::SampleUniquePtr = std::unique_ptr<T, samplesCommon::InferDeleter>

inherited

Constructor & Destructor Documentation

◆ SampleCharRNNLoop()

SampleCharRNNLoop::SampleCharRNNLoop ( SampleCharRNNParams params )

inline

Member Function Documentation

◆ addLSTMLayers()

nvinfer1::ILayer * SampleCharRNNLoop::addLSTMLayers ( SampleCharRNNBase::SampleUniquePtr< nvinfer1::INetworkDefinition > & network )

finalprotectedvirtual

Add inputs to the TensorRT network and configure LSTM layers using network definition API.

Implements SampleCharRNNBase.

Here is the call graph for this function:

◆ addLSTMCell()

nvinfer1::ILayer * SampleCharRNNLoop::addLSTMCell	(	SampleUniquePtr< nvinfer1::INetworkDefinition > &	network,
		const LstmIO &	inputTensors,
		nvinfer1::ITensor *	sequenceSize,
		const LstmParams &	params,
		LstmIO &	outputTensors
	)

private

Here is the call graph for this function:

Here is the caller graph for this function:

◆ build()

bool SampleCharRNNBase::build ( )

inherited

Builds the network engine.

Creates the network, configures the builder and creates the network engine.

This function loads weights from a trained TensorFlow model, creates the network using the TensorRT network definition API, and builds a TensorRT engine.

Returns: Returns true if the engine was created successfully and false otherwise

Here is the call graph for this function:

◆ infer()

bool SampleCharRNNBase::infer ( )

inherited

Runs the TensorRT inference engine for this sample.

This function is the main execution function of the sample. It allocates the buffer, sets inputs, executes the engine, and verifies the output.

Here is the call graph for this function:

◆ teardown()

bool SampleCharRNNBase::teardown ( )

inherited

Used to clean up any state created in the sample class.

◆ convertRNNWeights()

nvinfer1::Weights SampleCharRNNBase::convertRNNWeights	(	nvinfer1::Weights	orig,
		int	dataSize
	)

protectedinherited

Converts RNN weights from TensorFlow's format to TensorRT's format.

Parameters

input Weights that are stored in TensorFlow's format.

Returns: Converted weights in TensorRT's format.

Note: TensorFlow weight parameters for BasicLSTMCell are formatted as: Each [WR][icfo] is hiddenSize sequential elements. CellN Row 0: WiT, WcT, WfT, WoT CellN Row 1: WiT, WcT, WfT, WoT ... CellN RowM-1: WiT, WcT, WfT, WoT CellN RowM+0: RiT, RcT, RfT, RoT CellN RowM+1: RiT, RcT, RfT, RoT ... CellNRow2M-1: RiT, RcT, RfT, RoT

TensorRT expects the format to laid out in memory: CellN: Wi, Wc, Wf, Wo, Ri, Rc, Rf, Ro

Here is the caller graph for this function:

◆ convertRNNBias()

nvinfer1::Weights SampleCharRNNBase::convertRNNBias ( nvinfer1::Weights input )

protectedinherited

Converts RNN Biases from TensorFlow's format to TensorRT's format.

Parameters

input Biases that are stored in TensorFlow's format.

Returns: Converted bias in TensorRT's format.

Note: TensorFlow bias parameters for BasicLSTMCell are formatted as: CellN: Bi, Bc, Bf, Bo

TensorRT expects the format to be: CellN: Wi, Wc, Wf, Wo, Ri, Rc, Rf, Ro

Since tensorflow already combines U and W, we double the size and set all of U to zero.

Here is the caller graph for this function:

◆ addReshape()

nvinfer1::ITensor * SampleCharRNNBase::addReshape	(	SampleUniquePtr< nvinfer1::INetworkDefinition > &	network,
		nvinfer1::ITensor &	tensor,
		nvinfer1::Dims	dims
	)

protectedinherited

Here is the call graph for this function:

Here is the caller graph for this function:

◆ loadWeights()

std::map< std::string, nvinfer1::Weights > SampleCharRNNBase::loadWeights ( const std::string file )

privateinherited

Load requested weights from a formatted file into a map.

Parameters

file	Path to weights file. File has to be the formatted dump from the dumpTFWts.py script. Otherwise, this function will not work as intended.

Returns: A map containing the extracted weights.

Note: Weight V2 files are in a very simple space delimited format. <number of buffers> for each buffer: [name] [type] [shape] <data as binary blob>
Note: type is the integer value of the DataType enum in NvInfer.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ constructNetwork()

void SampleCharRNNBase::constructNetwork	(	SampleUniquePtr< nvinfer1::IBuilder > &	builder,
		SampleUniquePtr< nvinfer1::INetworkDefinition > &	network,
		SampleUniquePtr< nvinfer1::IBuilderConfig > &	config
	)

privateinherited

Create full model using the TensorRT network definition API and build the engine.

Parameters

weightMap	Map that contains all the weights required by the model.
modelStream	The stream within which the engine is serialized once built.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ copyEmbeddingToInput()

void SampleCharRNNBase::copyEmbeddingToInput	(	samplesCommon::BufferManager &	buffers,
		const char &	c
	)

privateinherited

Looks up the embedding tensor for a given char and copies it to input buffer.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ stepOnce()

bool SampleCharRNNBase::stepOnce	(	samplesCommon::BufferManager &	buffers,
		SampleUniquePtr< nvinfer1::IExecutionContext > &	context,
		cudaStream_t &	stream
	)

privateinherited

Perform one time step of inference with the TensorRT execution context.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ copyRNNOutputsToInputs()

void SampleCharRNNBase::copyRNNOutputsToInputs ( samplesCommon::BufferManager & buffers )

privateinherited

Copies Ct/Ht output from the RNN to the Ct-1/Ht-1 input buffers for next time step.

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

◆ mWeightMap

std::map<std::string, nvinfer1::Weights> SampleCharRNNBase::mWeightMap

protectedinherited

◆ weightsMemory

std::vector<SampleUniquePtr<nvinfer1::IHostMemory> > SampleCharRNNBase::weightsMemory

protectedinherited

◆ mParams

SampleCharRNNParams SampleCharRNNBase::mParams

protectedinherited

◆ mEngine

std::shared_ptr<nvinfer1::ICudaEngine> SampleCharRNNBase::mEngine {nullptr}

privateinherited

The TensorRT engine used to run the network.

The documentation for this class was generated from the following file:

sampleCharRNN.cpp

Classes

Public Types

Public Member Functions

Protected Member Functions

Protected Attributes

Private Member Functions

Private Attributes

Member Typedef Documentation

◆ SampleUniquePtr

Constructor & Destructor Documentation

◆ SampleCharRNNLoop()

Member Function Documentation

◆ addLSTMLayers()

◆ addLSTMCell()

◆ build()

◆ infer()

◆ teardown()

◆ convertRNNWeights()

◆ convertRNNBias()

◆ addReshape()

◆ loadWeights()

◆ constructNetwork()

◆ copyEmbeddingToInput()

◆ stepOnce()

◆ copyRNNOutputsToInputs()

Member Data Documentation

◆ mWeightMap

◆ weightsMemory

◆ mParams

◆ mEngine