SampleCharRNNBase Class Reference

The SampleCharRNNBase class implements the char_rnn sample. More...

Inheritance diagram for SampleCharRNNBase:

Collaboration diagram for SampleCharRNNBase:

Public Types
template<typename T >
using	SampleUniquePtr = std::unique_ptr< T, samplesCommon::InferDeleter >

Public Member Functions
	SampleCharRNNBase (const SampleCharRNNParams &params)
virtual	~SampleCharRNNBase ()=default
bool	build ()
	Builds the network engine. More...
bool	infer ()
	Runs the TensorRT inference engine for this sample. More...
bool	teardown ()
	Used to clean up any state created in the sample class. More...

Protected Member Functions
virtual nvinfer1::ILayer *	addLSTMLayers (SampleUniquePtr< nvinfer1::INetworkDefinition > &network)=0
	Add inputs to the TensorRT network and configure LSTM layers using network definition API. More...
nvinfer1::Weights	convertRNNWeights (nvinfer1::Weights input, int dataSize)
	Converts RNN weights from TensorFlow's format to TensorRT's format. More...
nvinfer1::Weights	convertRNNBias (nvinfer1::Weights input)
	Converts RNN Biases from TensorFlow's format to TensorRT's format. More...
nvinfer1::ITensor *	addReshape (SampleUniquePtr< nvinfer1::INetworkDefinition > &network, nvinfer1::ITensor &tensor, nvinfer1::Dims dims)

Protected Attributes
std::map< std::string, nvinfer1::Weights >	mWeightMap
std::vector< SampleUniquePtr< nvinfer1::IHostMemory > >	weightsMemory
SampleCharRNNParams	mParams

Private Member Functions
std::map< std::string, nvinfer1::Weights >	loadWeights (const std::string file)
	Load requested weights from a formatted file into a map. More...
void	constructNetwork (SampleUniquePtr< nvinfer1::IBuilder > &builder, SampleUniquePtr< nvinfer1::INetworkDefinition > &network, SampleUniquePtr< nvinfer1::IBuilderConfig > &config)
	Create full model using the TensorRT network definition API and build the engine. More...
void	copyEmbeddingToInput (samplesCommon::BufferManager &buffers, const char &c)
	Looks up the embedding tensor for a given char and copies it to input buffer. More...
bool	stepOnce (samplesCommon::BufferManager &buffers, SampleUniquePtr< nvinfer1::IExecutionContext > &context, cudaStream_t &stream)
	Perform one time step of inference with the TensorRT execution context. More...
void	copyRNNOutputsToInputs (samplesCommon::BufferManager &buffers)
	Copies Ct/Ht output from the RNN to the Ct-1/Ht-1 input buffers for next time step. More...

Private Attributes
std::shared_ptr< nvinfer1::ICudaEngine >	mEngine {nullptr}
	The TensorRT engine used to run the network. More...

Detailed Description

The SampleCharRNNBase class implements the char_rnn sample.

It uses weights from a trained TensorFlow model and creates the network using the TensorRT network definition API

Member Typedef Documentation

SampleUniquePtr

template<typename T >

using SampleCharRNNBase::SampleUniquePtr = std::unique_ptr<T, samplesCommon::InferDeleter>

Constructor & Destructor Documentation

SampleCharRNNBase()

SampleCharRNNBase::SampleCharRNNBase ( const SampleCharRNNParams &  params )

inline

~SampleCharRNNBase()

virtual SampleCharRNNBase::~SampleCharRNNBase ( )

virtualdefault

Member Function Documentation

build()

bool SampleCharRNNBase::build

(

)

Builds the network engine.

Creates the network, configures the builder and creates the network engine.

This function loads weights from a trained TensorFlow model, creates the network using the TensorRT network definition API, and builds a TensorRT engine.

Returns

Returns true if the engine was created successfully and false otherwise

Here is the call graph for this function:

infer()

bool SampleCharRNNBase::infer

(

)

Runs the TensorRT inference engine for this sample.

This function is the main execution function of the sample. It allocates the buffer, sets inputs, executes the engine, and verifies the output.

Here is the call graph for this function:

teardown()

bool SampleCharRNNBase::teardown

(

)

Used to clean up any state created in the sample class.

addLSTMLayers()

virtual nvinfer1::ILayer* SampleCharRNNBase::addLSTMLayers ( SampleUniquePtr< nvinfer1::INetworkDefinition > &  network )

protectedpure virtual

Add inputs to the TensorRT network and configure LSTM layers using network definition API.

Implemented in SampleCharRNNLoop, and SampleCharRNNv2.

Here is the caller graph for this function:

convertRNNWeights()

nvinfer1::Weights SampleCharRNNBase::convertRNNWeights ( nvinfer1::Weights  orig, int  dataSize  )

protected

Converts RNN weights from TensorFlow's format to TensorRT's format.

Parameters

input

Weights that are stored in TensorFlow's format.

Returns

Converted weights in TensorRT's format.

Note

TensorFlow weight parameters for BasicLSTMCell are formatted as: Each [WR][icfo] is hiddenSize sequential elements. CellN Row 0: WiT, WcT, WfT, WoT CellN Row 1: WiT, WcT, WfT, WoT ... CellN RowM-1: WiT, WcT, WfT, WoT CellN RowM+0: RiT, RcT, RfT, RoT CellN RowM+1: RiT, RcT, RfT, RoT ... CellNRow2M-1: RiT, RcT, RfT, RoT

TensorRT expects the format to laid out in memory: CellN: Wi, Wc, Wf, Wo, Ri, Rc, Rf, Ro

Here is the caller graph for this function:

convertRNNBias()

nvinfer1::Weights SampleCharRNNBase::convertRNNBias ( nvinfer1::Weights  input )

protected

Converts RNN Biases from TensorFlow's format to TensorRT's format.

Parameters

input

Biases that are stored in TensorFlow's format.

Returns

Converted bias in TensorRT's format.

Note

TensorFlow bias parameters for BasicLSTMCell are formatted as: CellN: Bi, Bc, Bf, Bo

TensorRT expects the format to be: CellN: Wi, Wc, Wf, Wo, Ri, Rc, Rf, Ro

Since tensorflow already combines U and W, we double the size and set all of U to zero.

Here is the caller graph for this function:

addReshape()

nvinfer1::ITensor * SampleCharRNNBase::addReshape ( SampleUniquePtr< nvinfer1::INetworkDefinition > &  network, nvinfer1::ITensor &  tensor, nvinfer1::Dims  dims  )

protected

Here is the call graph for this function:

Here is the caller graph for this function:

loadWeights()

std::map< std::string, nvinfer1::Weights > SampleCharRNNBase::loadWeights ( const std::string  file )

private

Load requested weights from a formatted file into a map.

Parameters

file	Path to weights file. File has to be the formatted dump from the dumpTFWts.py script. Otherwise, this function will not work as intended.

Returns

A map containing the extracted weights.

Note

Weight V2 files are in a very simple space delimited format. <number of buffers> for each buffer: [name] [type] [shape] <data as binary blob>
Note: type is the integer value of the DataType enum in NvInfer.h.

Here is the call graph for this function:

Here is the caller graph for this function:

constructNetwork()

void SampleCharRNNBase::constructNetwork ( SampleUniquePtr< nvinfer1::IBuilder > &  builder, SampleUniquePtr< nvinfer1::INetworkDefinition > &  network, SampleUniquePtr< nvinfer1::IBuilderConfig > &  config  )

private

Create full model using the TensorRT network definition API and build the engine.

Parameters

weightMap	Map that contains all the weights required by the model.
modelStream	The stream within which the engine is serialized once built.

Here is the call graph for this function:

Here is the caller graph for this function:

copyEmbeddingToInput()

void SampleCharRNNBase::copyEmbeddingToInput ( samplesCommon::BufferManager &  buffers, const char &  c  )

private

Looks up the embedding tensor for a given char and copies it to input buffer.

Here is the call graph for this function:

Here is the caller graph for this function:

stepOnce()

bool SampleCharRNNBase::stepOnce ( samplesCommon::BufferManager &  buffers, SampleUniquePtr< nvinfer1::IExecutionContext > &  context, cudaStream_t &  stream  )

private

Perform one time step of inference with the TensorRT execution context.

Here is the call graph for this function:

Here is the caller graph for this function:

copyRNNOutputsToInputs()

void SampleCharRNNBase::copyRNNOutputsToInputs ( samplesCommon::BufferManager &  buffers )

private

Copies Ct/Ht output from the RNN to the Ct-1/Ht-1 input buffers for next time step.

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

mWeightMap

std::map<std::string, nvinfer1::Weights> SampleCharRNNBase::mWeightMap

protected

weightsMemory

std::vector<SampleUniquePtr<nvinfer1::IHostMemory> > SampleCharRNNBase::weightsMemory

protected

mParams

SampleCharRNNParams SampleCharRNNBase::mParams

protected

mEngine

std::shared_ptr<nvinfer1::ICudaEngine> SampleCharRNNBase::mEngine {nullptr}

private

The TensorRT engine used to run the network.

---

The documentation for this class was generated from the following file:

• sampleCharRNN.cpp