nvinfer1::utils Namespace Reference

Functions
bool	reshapeWeights (const Weights &input, const int32_t *shape, const int32_t *shapeOrder, void *data, int32_t nbDims)
	Reformat the input weights of the given shape based on the new order of dimensions. More...
bool	reorderSubBuffers (void *input, const int32_t *order, int32_t num, int32_t size)
	Takes an input stream and re-orders num chunks of the data given the size and order. More...
bool	transposeSubBuffers (void *input, DataType type, int32_t num, int32_t height, int32_t width)
	Transpose num sub-buffers of height * width. More...

Function Documentation

reshapeWeights()

bool nvinfer1::utils::reshapeWeights	(	const Weights &	input,
		const int32_t *	shape,
		const int32_t *	shapeOrder,
		void *	data,
		int32_t	nbDims
	)

Reformat the input weights of the given shape based on the new order of dimensions.

Parameters

input	The input weights to reshape.
shape	The shape of the weights.
shapeOrder	The order of the dimensions to process for the output.
data	The location where the output data is placed.
nbDims	The number of dimensions to process.

Take the weights specified by input with the dimensions specified by shape and re-order the weights based on the new dimensions specified by shapeOrder. The size of each dimension and the input data is not modified. The output volume pointed to by data must be the same as he input volume.

Example usage: float *out = new float[N*C*H*W]; Weights input{DataType::kFLOAT, {0 ... N*C*H*W-1}, N*C*H*W size}; int32_t order[4]{1, 0, 3, 2}; int32_t shape[4]{C, N, W, H}; reshapeWeights(input, shape, order, out, 4); Weights reshaped{input.type, out, input.count};

Input Matrix{3, 2, 3, 2}: { 0 1}, { 2 3}, { 4 5} <– {0, 0, *, *} { 6 7}, { 8 9}, {10 11} <– {0, 1, *, *} {12 13}, {14 15}, {16 17} <– {1, 0, *, *} {18 19}, {20 21}, {22 23} <– {1, 1, *, *} {24 25}, {26 27}, {28 29} <– {2, 0, *, *} {30 31}, {32 33}, {34 35} <– {2, 1, *, *}

Output Matrix{2, 3, 2, 3}: { 0 2 4}, { 1 3 5} <– {0, 0, *, *} {12 14 16}, {13 15 17} <– {0, 1, *, *} {24 26 28}, {25 27 29} <– {0, 2, *, *} { 6 8 10}, { 7 9 11} <– {1, 0, *, *} {18 20 22}, {19 21 23} <– {1, 1, *, *} {30 32 34}, {31 33 35} <– {1, 2, *, *}

Returns

True on success, false on failure.

reorderSubBuffers()

bool nvinfer1::utils::reorderSubBuffers	(	void *	input,
		const int32_t *	order,
		int32_t	num,
		int32_t	size
	)

Takes an input stream and re-orders num chunks of the data given the size and order.

Parameters

input	The input data to re-order.
order	The new order of the data sub-buffers.
num	The number of data sub-buffers to re-order.
size	The size of each data sub-buffer in bytes.

In some frameworks, the ordering of the sub-buffers within a dimension is different than the way that TensorRT expects them. TensorRT expects the gate/bias sub-buffers for LSTM's to be in fico order. TensorFlow however formats the sub-buffers in icfo order. This helper function solves this in a generic fashion.

Example usage output of reshapeWeights above: int32_t indir[1]{1, 0} int32_t stride = W*H; for (int32_t x = 0, y = N*C; x < y; ++x) reorderSubBuffers(out + x * stride, indir, H, W);

Input Matrix{2, 3, 2, 3}: { 0 2 4}, { 1 3 5} <– {0, 0, *, *} {12 14 16}, {13 15 17} <– {0, 1, *, *} {24 26 28}, {25 27 29} <– {0, 2, *, *} { 6 8 10}, { 7 9 11} <– {1, 0, *, *} {18 20 22}, {19 21 23} <– {1, 1, *, *} {30 32 34}, {31 33 35} <– {1, 2, *, *}

Output Matrix{2, 3, 2, 3}: { 1 3 5}, { 0 2 4} <– {0, 0, *, *} {13 15 17}, {12 14 16} <– {0, 1, *, *} {25 27 29}, {24 26 28} <– {0, 2, *, *} { 7 9 11}, { 6 8 10} <– {1, 0, *, *} {19 21 23}, {18 20 22} <– {1, 1, *, *} {31 33 35}, {30 32 34} <– {1, 2, *, *}

Returns

True on success, false on failure.

See also

reshapeWeights()

transposeSubBuffers()

bool nvinfer1::utils::transposeSubBuffers	(	void *	input,
		DataType	type,
		int32_t	num,
		int32_t	height,
		int32_t	width
	)

Transpose num sub-buffers of height * width.

Parameters

input	The input data to transpose.
type	The type of the data to transpose.
num	The number of data sub-buffers to transpose.
height	The size of the height dimension to transpose.
width	The size of the width dimension to transpose.

Returns

True on success, false on failure.