Instantiate one of these objects by giving it the name of the .cfg and .weights file, then call DarkHelp::NN::predict() as often as necessary to determine what the images contain. More...
#include "DarkHelpNN.hpp"
Public Member Functions | |
NN () | |
Constructor. When using this constructor, the neural network remains uninitialized until DarkHelp::NN::init() is called. More... | |
NN (const bool delete_combined_bundle_once_loaded, const std::string &filename, const std::string &key="", const EDriver driver=EDriver::kDarknet) | |
Constructor. More... | |
NN (const char *cfg_filename, const char *weights_filename, const char *names_filename="", const bool verify_files_first=true, const EDriver driver=EDriver::kDarknet) | |
Constructor. More... | |
NN (const Config &cfg) | |
Constructor. More... | |
NN (const std::string &cfg_filename, const std::string &weights_filename, const std::string &names_filename="", const bool verify_files_first=true, const EDriver driver=EDriver::kDarknet) | |
Constructor. More... | |
~NN () | |
Destructor. This automatically calls reset() to release memory allocated by the neural network. More... | |
cv::Mat | annotate (const float new_threshold=-1.0f) |
Takes the most recent DarkHelp::NN::prediction_results, and applies them to the most recent DarkHelp::NN::original_image. More... | |
NN & | clear () |
Clear out the images and the predictions stored internally within this object. More... | |
std::string | duration_string () |
Return DarkHelp::NN::duration as a text string which can then be added to the image during annotation. More... | |
bool | empty () const |
Only returns true if both original_image and prediction_results are both empty. More... | |
int | image_channels () |
Return the number of channels defined in the .cfg file. Usually, this will be 3 . More... | |
NN & | init () |
Initialize ("load") the darknet neural network. More... | |
NN & | init (const bool delete_combined_bundle_once_loaded, const std::string &filename, const std::string &key="", const EDriver driver=EDriver::kDarknet) |
Initialize ("load") the darknet neural network using a "bundle" .dh file created using the DarkHelpCombine command-line tool. More... | |
NN & | init (const std::string &cfg_filename, const std::string &weights_filename, const std::string &names_filename="", const bool verify_files_first=true, const EDriver driver=EDriver::kDarknet) |
Initialize ("load") the darknet neural network. More... | |
bool | is_initialized () const |
Determines if a neural network has been loaded. More... | |
bool | is_loaded () const |
Alias for is_initialized(). Returns true if the neural network has been loaded. More... | |
cv::Size | network_size () |
Determine the size of the network. For example, 416x416, or 608x480. More... | |
PredictionResults | predict (const std::string &image_filename, const float new_threshold=-1.0f) |
Use the neural network to predict what is contained in this image. More... | |
PredictionResults | predict (cv::Mat mat, const float new_threshold=-1.0f) |
Use the neural network to predict what is contained in this image. More... | |
PredictionResults | predict_tile (cv::Mat mat, const float new_threshold=-1.0f) |
Similar to DarkHelp::NN::predict(), but automatically breaks the images down into individual tiles if it is significantly larger than the network dimensions. More... | |
NN & | reset () |
The opposite of DarkHelp::NN::init(). More... | |
NN & | snap_annotation (PredictionResult &pred) |
Snap only the given annotation. More... | |
NN & | snap_annotations () |
Snap all the annotations. More... | |
NN (const NN &)=delete | |
We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More... | |
NN (NN &&)=delete | |
We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More... | |
NN & | operator= (const NN &)=delete |
We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More... | |
NN & | operator= (NN &&)=delete |
We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More... | |
Static Public Member Functions | |
static std::string | version () |
Get a version string for the DarkHelp library. E.g., could be 1.5.13-1 . More... | |
Public Attributes | |
cv::Mat | annotated_image |
The most recent output produced by DarkHelp::NN::annotate(). More... | |
cv::Mat | binary_inverted_image |
Intended mostly for internal purpose, this is only useful when annotation "snapping" is enabled. More... | |
Config | config |
Configuratin for the neural network. More... | |
void * | darknet_net |
The Darknet network pointer will only be set when the driver is DarkHelp::EDriver::kDarknet in DarkHelp::NN::init(). More... | |
std::chrono::high_resolution_clock::duration | duration |
The length of time it took to initially load the network and weights (after the DarkHelp object has been constructed), or the length of time DarkHelp::NN::predict() took to run on the last image to be processed. More... | |
size_t | horizontal_tiles |
The number of horizontal tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict(). More... | |
VStr | names |
A vector of names corresponding to the identified classes. More... | |
cv::Mat | original_image |
The most recent image handled by DarkHelp::NN::predict(). More... | |
PredictionResults | prediction_results |
A copy of the most recent results after applying the neural network to an image. This is set by DarkHelp::NN::predict(). More... | |
cv::Size | tile_size |
The size that was used for each individual tile by DarkHelp::NN::predict_tile(). More... | |
size_t | vertical_tiles |
The number of vertical tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict(). More... | |
Protected Member Functions | |
NN & | name_prediction (PredictionResult &pred) |
Give a consistent name to the given production result. More... | |
PredictionResults | predict_internal (cv::Mat mat, const float new_threshold=-1.0f) |
Used by all the other DarkHelp::NN::predict() calls to do the actual network prediction. More... | |
void | predict_internal_darknet () |
Called from DarkHelp::NN::predict_internal(). More... | |
void | predict_internal_opencv () |
Called from DarkHelp::NN::predict_internal(). More... | |
Protected Attributes | |
cv::Size | network_dimensions |
Size of the neural network, e.g., 416x416 or 608x608 . More... | |
int | number_of_channels |
The number of channels defined in the .cfg file. This is normally set to 3 . More... | |
Instantiate one of these objects by giving it the name of the .cfg and .weights file, then call DarkHelp::NN::predict() as often as necessary to determine what the images contain.
For example:
Instead of calling DarkHelp::NN::annotate(), you can get the detection results and iterate through them:
Instead of writing your own loop, you can also use the std::ostream
operator<<()
like this:
|
delete |
We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.
So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.
|
delete |
We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.
So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.
DarkHelp::NN::~NN | ( | ) |
DarkHelp::NN::NN | ( | ) |
Constructor. When using this constructor, the neural network remains uninitialized until DarkHelp::NN::init() is called.
References reset().
DarkHelp::NN::NN | ( | const Config & | cfg | ) |
Constructor.
This constructor automatically calls DarkHelp::NN::init() to load the neural network. The neural network filenames within the cfg
object must not be empty otherwise the call to DarkHelp::NN::init() will fail.
References config, and init().
DarkHelp::NN::NN | ( | const std::string & | cfg_filename, |
const std::string & | weights_filename, | ||
const std::string & | names_filename = "" , |
||
const bool | verify_files_first = true , |
||
const EDriver | driver = EDriver::kDarknet |
||
) |
Constructor.
This constructor automatically calls DarkHelp::NN::init() with the given parameters.
verify_files_first
is set to true
(the default value). This is because DarkHelp::NN::init() will call DarkHelp::verify_cfg_and_weights() to correctly determine which is the
.cfg,
.weights, and
.names file, and swap the names around as necessary so Darknet is given the correct filenames. References init().
DarkHelp::NN::NN | ( | const char * | cfg_filename, |
const char * | weights_filename, | ||
const char * | names_filename = "" , |
||
const bool | verify_files_first = true , |
||
const EDriver | driver = EDriver::kDarknet |
||
) |
Constructor.
Similar to the previous constructor, but takes char*
instead of std::string
to prevent them from being converted to a bool
and causing the "bundle" constructor from being called.
DarkHelp::NN::NN | ( | const bool | delete_combined_bundle_once_loaded, |
const std::string & | filename, | ||
const std::string & | key = "" , |
||
const EDriver | driver = EDriver::kDarknet |
||
) |
Constructor.
This constructor takes a bundle that has been prepared with the DarkHelpCombine
command-line tool. That tool will combine the .cfg file, the
.names file, and the
.weights file into a single bundle. Optionally, an obfuscation key may also be required to extract the 3 files.
[in] | delete_combined_bundle_once_loaded | If this is set to true then the combined bundle filename will be deleted from the drive once the neural network is loaded. If you don't want the bundle file to be deleted you must pass false for this parameter. |
[in] | filename | This is the file that was previously created using the DarkHelpCombine command-line tool. It can have any name. By default, this file was created with a .dh file extension, though you can rename it. |
[in] | key | This is the passphrase key that was used when you ran the DarkHelpCombine command-line tool. If the key is empty, then no obfuscation was done on the file. |
[in] | driver | Determines the backend driver to use. See EDriver for details. Default is to use Darknet. |
For example, to create a bundle file that can be used with this constructor, you could run the following in the CLI:
To create a bundle file that is not obfuscated, you'd use an empty passphrase key:
References init().
cv::Mat DarkHelp::NN::annotate | ( | const float | new_threshold = -1.0f | ) |
Takes the most recent DarkHelp::NN::prediction_results, and applies them to the most recent DarkHelp::NN::original_image.
The output annotated image is stored in DarkHelp::NN::annotated_image as well as returned to the caller.
This is an example of what an annotated image looks like:
[in] | new_threshold | Which threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then DarkHelp::Config::threshold will be set to this new value. |
Turning down the threshold in DarkHelp::NN::annotate() wont bring back predictions that were excluded due to a higher threshold originally used with DarkHelp::NN::predict(). Here is an example:
In the previous example, when annotate() is called with the lower threshold of 25%, the predictions had already been capped at 75%. This means any prediction between >= 25% and < 75% were excluded from the prediction results. The only way to get those predictions is to re-run predict() with a value of 0.25.
1
.std::logic_error | if an attempt is made to annotate an empty image |
References DarkHelp::duration_string(), DarkHelp::get_default_annotation_colours(), and DarkHelp::pixelate_rectangles().
Referenced by Annotate(), main(), process_image(), process_video(), and DarkHelp::DHThreads::run().
DarkHelp::NN & DarkHelp::NN::clear | ( | ) |
Clear out the images and the predictions stored internally within this object.
This makes it seem as if the NN
object has not yet been used to process any images. Unlike reset(), this does not change any configuration settings, it only clears the images and the predictions. If a neural network has been loaded, calling clear()
does not unload that neural network, and is_initialized() will continue to return true
.
Calling this method between images is not necessary, but is included for completeness.
References clear().
Referenced by clear().
std::string DarkHelp::NN::duration_string | ( | ) |
Return DarkHelp::NN::duration as a text string which can then be added to the image during annotation.
For example, this might return "912 microseconds"
or "375 milliseconds"
.
References DarkHelp::duration_string().
Referenced by configure(), GetPredictionResults(), init(), and process_image().
bool DarkHelp::NN::empty | ( | ) | const |
Only returns true
if both original_image and prediction_results are both empty.
This will only happen in the following situations:
int DarkHelp::NN::image_channels | ( | ) |
Return the number of channels defined in the .cfg file. Usually, this will be 3
.
DarkHelp::NN & DarkHelp::NN::init | ( | ) |
Initialize ("load") the darknet neural network.
This uses the values within DarkHelp::NN::config and is called automatically if the network files have been specified to the constructor. You only need to manually call init()
if the default constructor without filenames is used.
std::invalid_argument | if the .cfg or .weights filenames have not been set. |
std::runtime_error | if the call to darknet's load_network_custom() has failed. |
std::invalid_argument | if there is a blank line in the .names file. |
std::invalid_argument | if the network dimensions cannot be read from the .cfg file |
std::invalid_argument | if the channels= ... line in the .cfg file is not 1 or 3 |
References DarkHelp::edit_cfg_file(), DarkHelp::kDarknet, DarkHelp::kMax, DarkHelp::kMin, DarkHelp::kOpenCVCPU, and DarkHelp::toggle_output_redirection().
Referenced by NN().
DarkHelp::NN & DarkHelp::NN::init | ( | const bool | delete_combined_bundle_once_loaded, |
const std::string & | filename, | ||
const std::string & | key = "" , |
||
const EDriver | driver = EDriver::kDarknet |
||
) |
Initialize ("load") the darknet neural network using a "bundle" .dh file created using the
DarkHelpCombine
command-line tool.
[in] | delete_combined_bundle_once_loaded | If this is set to true then the combined bundle filename will be deleted from the drive once the neural network is loaded. If you don't want the bundle file to be deleted you must pass false for this parameter. |
[in] | filename | This is the file that was previously created using the DarkHelpCombine command-line tool. It can have any name. By default, this file was created with a .dh file extension, though you can rename it. |
[in] | key | This is the passphrase key that was used when you ran the DarkHelpCombine command-line tool. If the key is empty, then no obfuscation was done on the file. |
[in] | driver | Determines the backend driver to use. See EDriver for details. Default is to use Darknet. |
References DarkHelp::extract(), and init().
DarkHelp::NN & DarkHelp::NN::init | ( | const std::string & | cfg_filename, |
const std::string & | weights_filename, | ||
const std::string & | names_filename = "" , |
||
const bool | verify_files_first = true , |
||
const EDriver | driver = EDriver::kDarknet |
||
) |
Initialize ("load") the darknet neural network.
If verify_files_first
has been enabled (the default) then this method will also call the static method DarkHelp::verify_cfg_and_weights() to perform some last-minute validation prior to darknet loading the neural network.
References init(), DarkHelp::kDarknet, and DarkHelp::verify_cfg_and_weights().
Referenced by configure(), and init().
bool DarkHelp::NN::is_initialized | ( | ) | const |
Determines if a neural network has been loaded.
For example:
NN
object has been created with the default constructor, is_initialized()
returns false
. is_initialized()
returns true
. is_initialized()
would continue to return true
. is_initialized()
returns false
.References DarkHelp::kDarknet.
Referenced by is_loaded().
|
inline |
Alias for is_initialized(). Returns true
if the neural network has been loaded.
References is_initialized().
|
protected |
Give a consistent name to the given production result.
This gets called by both DarkHelp::NN::predict_internal() and DarkHelp::NN::predict_tile() and is intended for internal use only.
References DarkHelp::PredictionResult::all_probabilities, DarkHelp::PredictionResult::best_class, DarkHelp::PredictionResult::best_probability, and DarkHelp::PredictionResult::name.
cv::Size DarkHelp::NN::network_size | ( | ) |
Determine the size of the network. For example, 416x416, or 608x480.
Referenced by configure(), GetPredictionResults(), and init().
We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.
So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.
We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.
So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.
DarkHelp::PredictionResults DarkHelp::NN::predict | ( | const std::string & | image_filename, |
const float | new_threshold = -1.0f |
||
) |
Use the neural network to predict what is contained in this image.
This results in a call to either DarkHelp::NN::predict_internal() or DarkHelp::NN::predict_tile() depending on how DarkHelp::Config::enable_tiles has been set.
[in] | image_filename | The name of the image file to load from disk and analyze. The member DarkHelp::NN::original_image will be set to this image. If the image is larger or smaller than the dimensions of the neural network, then Darknet will stretch the image to match the exact size of the neural network. Stretching the image does not maintain the the aspect ratio. |
[in] | new_threshold | Which threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then DarkHelp::Config::threshold will be set to this new value. The threshold must be either -1, or a value between 0.0 and 1.0 meaning 0% to 100%. |
std::invalid_argument | if the image failed to load. |
Referenced by main(), Predict(), PredictFN(), process_image(), process_video(), and DarkHelp::DHThreads::run().
DarkHelp::PredictionResults DarkHelp::NN::predict | ( | cv::Mat | mat, |
const float | new_threshold = -1.0f |
||
) |
Use the neural network to predict what is contained in this image.
This results in a call to either DarkHelp::NN::predict_internal() or DarkHelp::NN::predict_tile() depending on how DarkHelp::Config::enable_tiles has been set.
[in] | mat | A OpenCV2 image which has already been loaded and which needs to be analyzed. The member DarkHelp::NN::original_image will be set to this image. If the image is larger or smaller than the dimensions of the neural network, then Darknet will stretch the image to match the exact size of the neural network. Stretching the image does not maintain the the aspect ratio. |
[in] | new_threshold | Which threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then DarkHelp::Config::threshold will be set to this new value. The threshold must be either -1, or a value between 0.0 and 1.0 meaning 0% to 100%. |
std::invalid_argument | if the image is empty. |
|
protected |
Used by all the other DarkHelp::NN::predict() calls to do the actual network prediction.
This uses the image stored in DarkHelp::NN::original_image.
std::logic_error | if the DarkHelp object has not been initialized. |
std::logic_error | if the network is invalid. |
std::logic_error | if the image is invalid. |
References DarkHelp::kAscending, DarkHelp::kDarknet, DarkHelp::kDescending, DarkHelp::kInvalid, and DarkHelp::kPageOrder.
|
protected |
Called from DarkHelp::NN::predict_internal().
References DarkHelp::PredictionResult::all_probabilities, DarkHelp::PredictionResult::best_class, DarkHelp::PredictionResult::best_probability, convert_opencv_mat_to_darknet_image(), DarkHelp::fast_resize_ignore_aspect_ratio(), DarkHelp::fix_out_of_bound_normalized_rect(), DarkHelp::PredictionResult::original_point, DarkHelp::PredictionResult::original_size, DarkHelp::PredictionResult::rect, DarkHelp::slow_resize_ignore_aspect_ratio(), and DarkHelp::PredictionResult::tile.
|
protected |
Called from DarkHelp::NN::predict_internal().
References DarkHelp::fast_resize_ignore_aspect_ratio(), DarkHelp::fix_out_of_bound_normalized_rect(), DarkHelp::slow_resize_ignore_aspect_ratio(), and DarkHelp::PredictionResult::tile.
DarkHelp::PredictionResults DarkHelp::NN::predict_tile | ( | cv::Mat | mat, |
const float | new_threshold = -1.0f |
||
) |
Similar to DarkHelp::NN::predict(), but automatically breaks the images down into individual tiles if it is significantly larger than the network dimensions.
This is explained in details in Image Tiling.
false
(which is the default).Here is a visual representation of a large image broken into 4 tiles for processing by Darknet. It is important to understand that neither the individual image tiles nor their results are returned to the caller. DarkHelp only returns the final results once each tile has been processed and the vectors have been merged together.
std::invalid_argument | if the image is empty. |
DarkHelp::NN & DarkHelp::NN::reset | ( | ) |
The opposite of DarkHelp::NN::init().
This is automatically called by the destructor.
Referenced by NN(), and ~NN().
DarkHelp::NN & DarkHelp::NN::snap_annotation | ( | DarkHelp::PredictionResult & | pred | ) |
Snap only the given annotation.
References DarkHelp::PredictionResult::original_point, DarkHelp::PredictionResult::original_size, and DarkHelp::PredictionResult::rect.
DarkHelp::NN & DarkHelp::NN::snap_annotations | ( | ) |
Snap all the annotations.
This is automatically called from predict() when DarkHelp::Config::snapping_enabled is set to true
. When set to false
, you can manually invoke this method to get the annotations to snap, or you can also manually call DarkHelp::NN::snap_annotation() on specific annotations as needed.
Image | Setting |
---|---|
snapping_enabled=false | |
snapping_enabled=true |
|
static |
Get a version string for the DarkHelp library. E.g., could be 1.5.13-1
.
cv::Mat DarkHelp::NN::annotated_image |
The most recent output produced by DarkHelp::NN::annotate().
Referenced by main().
cv::Mat DarkHelp::NN::binary_inverted_image |
Intended mostly for internal purpose, this is only useful when annotation "snapping" is enabled.
Config DarkHelp::NN::config |
Configuratin for the neural network.
This includes both settings for the neural network itself and everything needed to annotate images/frames.
Referenced by configure(), display_current_msg(), EnableAnnotationAutoHideLabels(), EnableAnnotationIncludeDuration(), EnableAnnotationIncludeTimestamp(), EnableAnnotationPixelate(), EnableAnnotationSuppressAllLabels(), EnableCombineTilePredictions(), EnableIncludeAllNames(), EnableNamesIncludePercentage(), EnableOnlyCombineSimilarPredictions(), EnableSnapping(), EnableTiles(), EnableUseFastImageResize(), GetPredictionResults(), init(), main(), NN(), process_image(), process_video(), SetAnnotationFontScale(), SetAnnotationFontThickness(), SetAnnotationLineThickness(), SetAnnotationPixelateSize(), SetAnnotationShadePredictions(), SetBinaryThresholdBlockSize(), SetBinaryThresholdConstant(), SetNonMaximalSuppression(), SetSnappingHorizontalTolerance(), SetSnappingLimitGrow(), SetSnappingLimitShrink(), SetSnappingVerticalTolerance(), SetThreshold(), SetTileEdgeFactor(), and SetTileRectFactor().
void* DarkHelp::NN::darknet_net |
The Darknet network pointer will only be set when the driver is DarkHelp::EDriver::kDarknet in DarkHelp::NN::init().
std::chrono::high_resolution_clock::duration DarkHelp::NN::duration |
The length of time it took to initially load the network and weights (after the DarkHelp object has been constructed), or the length of time DarkHelp::NN::predict() took to run on the last image to be processed.
If using DarkHelp::NN::predict_tile(), then this will store the sum of all durations across the entire set of tiles.
Referenced by process_video().
size_t DarkHelp::NN::horizontal_tiles |
The number of horizontal tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict().
This is set to 1
if calling DarkHelp::NN::predict(). It may be > 1 if calling DarkHelp::NN::predict_tile() with an image large enough to require multiple tiles.
Referenced by GetPredictionResults(), process_image(), and process_video().
VStr DarkHelp::NN::names |
A vector of names corresponding to the identified classes.
This is typically setup in the constructor, but can be manually set afterwards.
Referenced by configure(), GetPredictionResults(), init(), and process_image().
|
protected |
Size of the neural network, e.g., 416x416
or 608x608
.
|
protected |
The number of channels defined in the .cfg file. This is normally set to 3
.
cv::Mat DarkHelp::NN::original_image |
The most recent image handled by DarkHelp::NN::predict().
Referenced by GetPredictionResults().
PredictionResults DarkHelp::NN::prediction_results |
A copy of the most recent results after applying the neural network to an image. This is set by DarkHelp::NN::predict().
Referenced by GetPredictionResults(), Predict(), and PredictFN().
cv::Size DarkHelp::NN::tile_size |
The size that was used for each individual tile by DarkHelp::NN::predict_tile().
This will be the size of the network when calling DarkHelp::NN::predict().
For example, if the network is 416x416
, and the image used with DarkHelp::NN::predict_tile() measures 1280x960
, then:
3
2
"(427, 480)"
Referenced by GetPredictionResults(), process_image(), and process_video().
size_t DarkHelp::NN::vertical_tiles |
The number of vertical tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict().
This is set to 1
if calling DarkHelp::NN::predict(). It may be > 1 if calling DarkHelp::NN::predict_tile() with an image large enough to require multiple tiles.
Referenced by GetPredictionResults(), process_image(), and process_video().