This class allows you to easily run multiple identical copies of a neural network to process many files at once. More...

#include "DarkHelpThreads.hpp"

Collaboration diagram for DarkHelp::DHThreads:

[legend]

Public Types
using	ResultsMap = std::map< std::string, DarkHelp::PredictionResults >
	The key is the input image filename, while the value contains a copy of the prediction results. More...

Public Member Functions
	DHThreads ()
	Constructor. More...

	DHThreads (const DarkHelp::Config &c, const size_t workers, const std::filesystem::path &output_directory=".")
	Constructor. More...

	DHThreads (const std::filesystem::path &filename, const std::string &key, const size_t workers, const std::filesystem::path &output_directory=".", const DarkHelp::EDriver &driver=DarkHelp::EDriver::kDarknet)
	Constructor. More...

	~DHThreads ()
	Destructor. This will immediately stop all the worker threads and discard any results. More...

std::string	add_image (cv::Mat image)
	Add a single image to be processed. More...

DHThreads &	add_images (const std::filesystem::path &dir)
	Can be used to add a single image, or a subdirectory. More...

size_t	files_remaining () const
	Get the number of images and files that have not yet been processed. More...

DarkHelp::NN *	get_nn (const size_t idx)
	Gain access to the neural network for the given worker thread. More...

ResultsMap	get_results ()
	Get all of the available prediction results. More...

DHThreads &	init (const DarkHelp::Config &c, const size_t workers, const std::filesystem::path &output_directory=".")
	Mostly for use with the default constructor. More...

DHThreads &	init (const std::filesystem::path &filename, const std::string &key, const size_t workers, const std::filesystem::path &output_directory=".", const DarkHelp::EDriver &driver=DarkHelp::EDriver::kDarknet)
	Similar to the other init() call, but uses the name of the "bundle" file introduced in DarkHelp v1.8. More...

size_t	networks_loaded () const
	Get the number of worker threads which have loaded a copy of the neural network. More...

DHThreads &	purge ()
	Removes all input files, waits for all worker threads to finish processing, clears out any results, and resets the image index (similar to reset_image_index()). More...

DHThreads &	reset_image_index ()
	Reset to zero the image index used by add_image() to generate the image filename. More...

DHThreads &	restart ()
	Starts all of the processing threads. More...

DHThreads &	stop ()
	Causes the threads to stop processing and exit. More...

ResultsMap	wait_for_results ()
	Returns when there are zero input files remaining and all worker threads have finished processing images. More...

Public Attributes
std::atomic< bool >	annotate_output_images
	Determines if annotated output images are created. More...

DarkHelp::Config	cfg
	A copy of the configuration to use when instantiating each of the DarkHelp::NN objects. More...

std::atomic< bool >	detele_input_file_after_processing
	Determines whether the input images are deleted once they are processed. More...

Private Member Functions
void	run (const size_t id)
	The method that each worker thread runs to process images. More...

Private Attributes
std::atomic< size_t >	files_processing
	The number of worker threads which are currently processing an image. More...

std::deque< std::string >	input_files
	Used to keep track of all the input files remaining to be processed. More...

std::mutex	input_image_and_file_lock
	Lock used to protect access to both input_files and input_images. More...

std::atomic< size_t >	input_image_index
	Used by add_image() to generate an image filename. More...

std::map< std::string, cv::Mat >	input_images
	Used to keep track of all the input images remaining to be processed. More...

std::vector< DarkHelp::NN * >	networks
	Address to the worker thread neural networks. More...

std::filesystem::path	output_dir
	The directory where the output will be saved. More...

std::atomic< bool >	stop_requested
	If the threads need to stop, set this variable to `true`. More...

std::vector< std::thread >	threads
	The threads that were started. More...

std::atomic< size_t >	threads_ready
	Track the number of worker threads which have loaded the neural network. More...

size_t	worker_threads_to_start
	The number of threads to start. More...


std::condition_variable	trigger
	Used to signal the worker threads when more work becomes available. More...

std::mutex	trigger_lock
	Used to signal the worker threads when more work becomes available. More...


ResultsMap	all_results
	The prediction results for all the image file which have been processed. More...

std::mutex	results_lock
	The prediction results for all the image file which have been processed. More...

Detailed Description

This class allows you to easily run multiple identical copies of a neural network to process many files at once.

If you have enough vram, or if you're running the CPU-only version of Darknet on a device with many cores, then you can use this class to load multiple copies of a neural network. Each copy will run on a different thread, and can process images independently of each other. This works especially well when the network is loaded once and re-used many times to process a large set of images.

For example, on a RTX 3050 with 4 GiB of vram, a YOLOv4-tiny network measuring 768x576 takes 29.4 seconds to process 1540 image files.

With this class, the same set of 1540 images only takes 8.8 seconds to process using 13 identical instances of the network loaded on the GPU.

(Each instance of the network consumes 289 MiB of vram, which is why 13 copies can be loaded at once on a GPU with 4 GiB of vram.)

Note this header file is not included by DarkHelp.hpp. To use this functionality you'll need to explicitely include this header file.

Since: 2024-03-26

Member Typedef Documentation

◆ ResultsMap

using DarkHelp::DHThreads::ResultsMap = std::map<std::string, DarkHelp::PredictionResults>

The key is the input image filename, while the value contains a copy of the prediction results.

See also: DarkHelp::PredictionResults (a vector of results); DarkHelp::PredictionResult (an individual result)

Since: 2024-03-26

Constructor & Destructor Documentation

◆ DHThreads() [1/3]

DarkHelp::DHThreads::DHThreads ( )

Constructor.

No worker threads are started with this constructor. You'll need to manually call init().

Since: 2024-03-26

◆ DHThreads() [2/3]

DarkHelp::DHThreads::DHThreads	(	const DarkHelp::Config &	c,
		const size_t	workers,
		const std::filesystem::path &	output_directory = `"."`
	)

Constructor.

Parameters

[in]	c	The DarkHelp configuration that each thread should use to load the neural network.
[in]	workers	The number of worker threads to create. Each of these threads will also load a copy of the neural network, so the number you use here should depend on the amount of vram available and the neural network dimensions.
[in]	output_directory	The directory where the output files will be saved when annotate_output_images is enabled. If the directory does not exist, then it will be created. If it already exists, then it is left as-is, meaning existing files will remain.

The DHThreads constructor will automatically call init() to ensure all the threads and the neural networks are running.

Call add_image() or add_images() to get the threads to start processing images. Then call wait_for_results() or get_results().

Since: 2024-03-26

References init().

Here is the call graph for this function:

◆ DHThreads() [3/3]

DarkHelp::DHThreads::DHThreads	(	const std::filesystem::path &	filename,
		const std::string &	key,
		const size_t	workers,
		const std::filesystem::path &	output_directory = `"."`,
		const DarkHelp::EDriver &	driver = `DarkHelp::EDriver::kDarknet`
	)

Constructor.

This is similar to the other constructor, but uses the name of the "bundle" file introduced in DarkHelp v1.8. Run the CLI tool DarkHelpCombine to create a .dh bundle file.

Unlike the other DHThreads constructor, this one will wait until all the worker threads have fully loaded the neural network before returning to the caller. (This is required due to the use of the obfuscated bundle file.)

See also: DarkHelp::combine()

Since: 2024-04-16

References init().

Here is the call graph for this function:

◆ ~DHThreads()

DarkHelp::DHThreads::~DHThreads ( )

Destructor. This will immediately stop all the worker threads and discard any results.

Member Function Documentation

◆ add_image()

std::string DarkHelp::DHThreads::add_image ( cv::Mat image )

Add a single image to be processed.

Unlike add_images(), this does not require the image to be written to disk. The cv::Mat must be in standard OpenCV BGR format, and the lifetime of the image data must extend to when the image will be picked up and processed by one of the worker threads executing run().

Returns: A "virtual" filename will be created and returned to represent the OpenCV image. This filename is needed to correctly interpret the results from wait_for_results(). The filename generated contains a numerical value which is assigned in increasing sequential order, until one of purge(), restart() or reset_image_index() are called.

Note: Images added via add_image() will be processed before filenames added via add_images(). This is done to ensure that memory is freed up as quickly as possible (filenames barely take any memory).

See also: add_images(); reset_image_index()

Since: 2024-04-01

Referenced by main().

Here is the caller graph for this function:

◆ add_images()

DarkHelp::DHThreads & DarkHelp::DHThreads::add_images ( const std::filesystem::path & dir )

Can be used to add a single image, or a subdirectory.

If a subdirectory, then recurse looking for all images. Call this as many times as necessary until all images have been added. Image processing by the worker threads will start immediately. Additional images can be added at any time, even while the worker threads have already started processing the first set of images.

Note: Images added via add_image() will be processed before filenames added via add_images(). This is done to ensure that memory is freed up as quickly as possible.

See also: add_image()

Since: 2024-03-26

Referenced by main().

Here is the caller graph for this function:

◆ files_remaining()

size_t DarkHelp::DHThreads::files_remaining ( ) const

inline

Get the number of images and files that have not yet been processed.

This is the number of unprocessed images and files combined, plus the images which are in the middle of being processed.

Since: 2024-03-26

References files_processing, input_files, and input_images.

◆ get_nn()

DarkHelp::NN * DarkHelp::DHThreads::get_nn ( const size_t idx )

Gain access to the neural network for the given worker thread.

This will return nullptr if the given worker thread has not loaded a neural network.

Since: 2024-03-26

Referenced by main().

Here is the caller graph for this function:

◆ get_results()

DarkHelp::DHThreads::ResultsMap DarkHelp::DHThreads::get_results ( )

Get all of the available prediction results.

You may call this as often you like to get intermediate results (if any are available), or you can indirectly call this via wait_for_results().

Note: Once the results are read and returned, the structure in which the results are stored internally is cleared.

The difference between wait_for_results() and get_results() is that wait_for_results() will wait until all the results are available, while get_results() will immediately return with whatever results are available, even if the results are empty.

Since: 2024-03-26

◆ init() [1/2]

DarkHelp::DHThreads & DarkHelp::DHThreads::init	(	const DarkHelp::Config &	c,
		const size_t	workers,
		const std::filesystem::path &	output_directory = `"."`
	)

Mostly for use with the default constructor.

Parameters are the same as the other constructor.

Since: 2024-03-26

References DarkHelp::edit_cfg_file(), and DarkHelp::verify_cfg_and_weights().

Referenced by DHThreads().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ init() [2/2]

DarkHelp::DHThreads & DarkHelp::DHThreads::init	(	const std::filesystem::path &	filename,
		const std::string &	key,
		const size_t	workers,
		const std::filesystem::path &	output_directory = `"."`,
		const DarkHelp::EDriver &	driver = `DarkHelp::EDriver::kDarknet`
	)

Similar to the other init() call, but uses the name of the "bundle" file introduced in DarkHelp v1.8.

Note: This will not delete the obfuscated bundle file from disk.

Since: 2024-04-16

References DarkHelp::extract(), and init().

Here is the call graph for this function:

◆ networks_loaded()

size_t DarkHelp::DHThreads::networks_loaded ( ) const

inline

Get the number of worker threads which have loaded a copy of the neural network.

Since: 2024-03-26

References threads_ready.

◆ purge()

DarkHelp::DHThreads & DarkHelp::DHThreads::purge ( )

Removes all input files, waits for all worker threads to finish processing, clears out any results, and resets the image index (similar to reset_image_index()).

Unlike the call to stop(), calling purge() does not terminate the worker threads or unload the neural networks, so you can immediately call add_image() or add_images() without needing to restart().

Since: 2024-03-26

◆ reset_image_index()

DarkHelp::DHThreads & DarkHelp::DHThreads::reset_image_index ( )

Reset to zero the image index used by add_image() to generate the image filename.

Since: 2024-04-01

◆ restart()

DarkHelp::DHThreads & DarkHelp::DHThreads::restart ( )

Starts all of the processing threads.

This is automatically called by init(), but may also be called manually if stop() was called. Calling restart() when the threads are already running will cause the existing threads to stop, input files to be cleared, and existing results to be reset.

Warning: If a "bundle" file was used when loading the neural network, then the .cfg, .names, and .weights files do not exist on disk and the call to restart() will fail since the neural network cannot be loaded.

See also: init()

Since: 2024-03-26

References run().

Here is the call graph for this function:

◆ run()

void DarkHelp::DHThreads::run ( const size_t id )

private

The method that each worker thread runs to process images.

See also: restart()

References DarkHelp::NN::annotate(), and DarkHelp::NN::predict().

Referenced by restart().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ stop()

DarkHelp::DHThreads & DarkHelp::DHThreads::stop ( )

Causes the threads to stop processing and exit.

All results and input files are cleared. Does not return until all threads have joined. Worker threads can be restarted by calling either init() or restart().

See also: purge()

Since: 2024-03-26

◆ wait_for_results()

DarkHelp::DHThreads::ResultsMap DarkHelp::DHThreads::wait_for_results ( )

Returns when there are zero input files remaining and all worker threads have finished processing images.

Note this will clear out the results since it internally calls get_results().

The difference between wait_for_results() and get_results() is that wait_for_results() will wait until all the results are available, while get_results() will immediately return with whatever results are available at this point in time.

Since: 2024-03-26

Referenced by main().

Here is the caller graph for this function:

Member Data Documentation

◆ all_results

ResultsMap DarkHelp::DHThreads::all_results

private

The prediction results for all the image file which have been processed.

See also: get_results()

◆ annotate_output_images

std::atomic<bool> DarkHelp::DHThreads::annotate_output_images

Determines if annotated output images are created.

Default value is false.

Since: 2024-03-26

◆ cfg

DarkHelp::Config DarkHelp::DHThreads::cfg

A copy of the configuration to use when instantiating each of the DarkHelp::NN objects.

This is only referenced by restart(). Meaning if you change cfg after the DHThreads() consructor or init(), you'll need to call stop() or restart().

Since: 2024-03-26

◆ detele_input_file_after_processing

std::atomic<bool> DarkHelp::DHThreads::detele_input_file_after_processing

Determines whether the input images are deleted once they are processed.

Default value is false, meaning files are not deleted.

Since: 2024-03-26

◆ files_processing

std::atomic<size_t> DarkHelp::DHThreads::files_processing

private

The number of worker threads which are currently processing an image.

Referenced by files_remaining().

◆ input_files

std::deque<std::string> DarkHelp::DHThreads::input_files

private

Used to keep track of all the input files remaining to be processed.

See also: add_image(); add_images(); input_image_and_file_lock

Referenced by files_remaining().

◆ input_image_and_file_lock

std::mutex DarkHelp::DHThreads::input_image_and_file_lock

private

Lock used to protect access to both input_files and input_images.

◆ input_image_index

std::atomic<size_t> DarkHelp::DHThreads::input_image_index

private

Used by add_image() to generate an image filename.

◆ input_images

std::map<std::string, cv::Mat> DarkHelp::DHThreads::input_images

private

Used to keep track of all the input images remaining to be processed.

See also: add_image(); add_images(); input_image_and_file_lock

Referenced by files_remaining().

◆ networks

std::vector<DarkHelp::NN *> DarkHelp::DHThreads::networks

private

Address to the worker thread neural networks.

See also: get_nn()

◆ output_dir

std::filesystem::path DarkHelp::DHThreads::output_dir

private

The directory where the output will be saved.

◆ results_lock

std::mutex DarkHelp::DHThreads::results_lock

private

The prediction results for all the image file which have been processed.

See also: get_results()

◆ stop_requested

std::atomic<bool> DarkHelp::DHThreads::stop_requested

private

If the threads need to stop, set this variable to true.

See also: stop()

◆ threads

std::vector<std::thread> DarkHelp::DHThreads::threads

private

The threads that were started.

See also: worker_threads_to_start

◆ threads_ready

std::atomic<size_t> DarkHelp::DHThreads::threads_ready

private

Track the number of worker threads which have loaded the neural network.

Referenced by networks_loaded().

◆ trigger

std::condition_variable DarkHelp::DHThreads::trigger

private

Used to signal the worker threads when more work becomes available.

◆ trigger_lock

std::mutex DarkHelp::DHThreads::trigger_lock

private

Used to signal the worker threads when more work becomes available.

◆ worker_threads_to_start

size_t DarkHelp::DHThreads::worker_threads_to_start

private

The number of threads to start.

The documentation for this class was generated from the following files:

Public Types

Public Member Functions

Public Attributes

Private Member Functions

Private Attributes

Detailed Description

Member Typedef Documentation

◆ ResultsMap

Constructor & Destructor Documentation

◆ DHThreads() [1/3]

◆ DHThreads() [2/3]

◆ DHThreads() [3/3]

◆ ~DHThreads()

Member Function Documentation

◆ add_image()

◆ add_images()

◆ files_remaining()

◆ get_nn()

◆ get_results()

◆ init() [1/2]

◆ init() [2/2]

◆ networks_loaded()

◆ purge()

◆ reset_image_index()

◆ restart()

◆ run()

◆ stop()

◆ wait_for_results()

Member Data Documentation

◆ all_results

◆ annotate_output_images

◆ cfg

◆ detele_input_file_after_processing

◆ files_processing

◆ input_files

◆ input_image_and_file_lock

◆ input_image_index

◆ input_images

◆ networks

◆ output_dir

◆ results_lock

◆ stop_requested

◆ threads

◆ threads_ready

◆ trigger

◆ trigger_lock

◆ worker_threads_to_start