polygraphy.comparator.comparator.Comparator Class Reference

Inheritance diagram for polygraphy.comparator.comparator.Comparator:

Collaboration diagram for polygraphy.comparator.comparator.Comparator:

Static Public Member Functions
def	run (runners, data_loader=None, warm_up=None, use_subprocess=None, subprocess_timeout=None, subprocess_polling_interval=None, save_inputs_path=None)
def	postprocess (run_results, postprocess_func)
def	default_comparisons (run_results)
def	compare_accuracy (run_results, fail_fast=False, comparisons=None, compare_func=None)
def	validate (run_results, check_finite=None, check_nan=None, fail_fast=None)

Member Function Documentation

run()

def polygraphy.comparator.comparator.Comparator.run (   runners,   data_loader = None,   warm_up = None,   use_subprocess = None,   subprocess_timeout = None,   subprocess_polling_interval = None,   save_inputs_path = None  )

static

Runs the supplied runners sequentially.

Args:
    runners (List[BaseRunner]):
    A list of runners to run.
    data_loader (Generator -> OrderedDict[str, numpy.ndarray]):
    A generator or iterable that yields a dictionary that maps input names to input numpy buffers.
    In the simplest case, this can be a `List[Dict[str, numpy.ndarray]]` .

    In case you don't know details about the inputs ahead of time, you can access the
    `input_metadata` property in your data loader, which will be set to an `TensorMetadata`
    instance by this function.
    Note that this does not work for generators or lists.

    The number of iterations run by this function is controlled by the number of items supplied
    by the data loader.

    Defaults to an instance of `DataLoader`.
    warm_up (int):
    The number of warm up runs to perform for each runner before timing.
    Defaults to 0.
    use_subprocess (bool):
    Whether each runner should be run in a subprocess. This allows each runner to have exclusive
    access to the GPU. When using a subprocess, runners and loaders will never be modified.
    subprocess_timeout (int):
    The timeout before a subprocess is killed automatically. This is useful for handling processes
    that never terminate. A value of None disables the timeout. Defaults to None.
    subprocess_polling_interval (int):
    The polling interval, in seconds, for checking whether a subprocess has completed or crashed.
    In rare cases, omitting this parameter when subprocesses are enabled may cause this function
    to hang indefinitely if the subprocess crashes.
    A value of 0 disables polling. Defaults to 30 seconds.
    save_inputs_path (str):
    [EXPERIMENTAL] Path at which to save inputs used during inference. This will include all inputs generated by
    the provided data_loader, and will be saved as a pickled List[Dict[str, numpy.ndarray]].

Returns:
    RunResults:
    A mapping of runner names to the results of their inference.
    The ordering of `runners` is preserved in this mapping.

Here is the caller graph for this function:

postprocess()

def polygraphy.comparator.comparator.Comparator.postprocess (   run_results,   postprocess_func  )

static

Applies post processing to all the outputs in the provided run results.
This is a convenience function to avoid the need for manual iteration over the run_results dictionary.

Args:
    run_results (RunResults): The result of Comparator.run().
    postprocess_func (Callable(IterationResult) -> IterationResult):
    The function to apply to each ``IterationResult``.

Returns:
    RunResults: The updated run results.

default_comparisons()

def polygraphy.comparator.comparator.Comparator.default_comparisons (   run_results )

static

compare_accuracy()

def polygraphy.comparator.comparator.Comparator.compare_accuracy (   run_results,   fail_fast = False,   comparisons = None,   compare_func = None  )

static

Args:
    run_results (RunResults): The result of Comparator.run()


    fail_fast (bool): Whether to exit after the first failure
    comparisons (List[Tuple[int, int]]):
    Comparisons to perform, specified by runner indexes. For example, [(0, 1), (1, 2)]
    would compare the first runner with the second, and the second with the third.
    By default, this compares each result to the subsequent one.
    compare_func (Callable(IterationResult, IterationResult) -> OrderedDict[str, bool]):
    A function that takes in two IterationResults, and returns a dictionary that maps output
    names to a boolean (or anything convertible to a boolean) indicating whether outputs matched.
    The order of arguments to this function is guaranteed to be the same as the ordering of the
    tuples contained in `comparisons`.

Returns:
    AccuracyResult:
    A summary of the results of the comparisons. The order of the keys (i.e. runner pairs) is
    guaranteed to be the same as the order of `comparisons`. For more details, see the AccuracyResult
    docstring (e.g. help(AccuracyResult)).

validate()

def polygraphy.comparator.comparator.Comparator.validate (   run_results,   check_finite = None,   check_nan = None,   fail_fast = None  )

static

Checks output validity.

Args:
    run_results (Dict[str, List[IterationResult]]): The result of Comparator.run().
    check_finite (bool): Whether to fail on non-finite values. Defaults to False.
    check_nan (bool): Whether to fail on NaNs. Defaults to True.
    fail_fast (bool): Whether to fail after the first invalid value. Defaults to False.

Returns:
    bool: True if all outputs were valid, False otherwise.

---

The documentation for this class was generated from the following file:

• comparator/comparator.py