guibot.finder module¶
SUMMARY¶
Computer vision finders (CV backends) to perform find targets on screen.
INTERFACE¶
- class guibot.finder.CVParameter(value, min_val=None, max_val=None, delta=10.0, tolerance=1.0, fixed=True, enumerated=False)[source]¶
Bases:
object
A class for a single parameter used for CV backend configuration.
- static from_string(raw)[source]¶
Parse a CV parameter from string.
- Parameters:
raw (str) – string representation for the parameter
- Returns:
parameter parsed from the representation
- Return type:
- Raises:
ValueError
if unsupported type is encountered
- random_value(mu=None, sigma=None)[source]¶
Return a random value of the CV parameter given its range and type.
- Parameters:
mu (bool or int or float or str or None) – mean for a normal distribution, uniform distribution if None
sigma (bool or int or float or str or None) – standard deviation for a normal distribution, quarter range if None (maximal range is equivalent to maximal data type values)
- Returns:
a random value comforming to the CV parameter range and type
- Return type:
bool or int or float or str or None
Note
Only uniform distribution is used for boolean values.
- class guibot.finder.Finder(configure=True, synchronize=True)[source]¶
Bases:
LocalConfig
Base for all image matching functionality and backends.
The image finding methods include finding one or all matches above the similarity defined in the configuration of each backend.
There are many parameters that could contribute for a good match. They can all be manually adjusted or automatically calibrated.
- static from_match_file(filename)[source]¶
Read the configuration from a match file with the given filename.
- Parameters:
filename (str) – match filename for the configuration
- Returns:
target finder with the parsed (and generated) settings
- Return type:
finder.Finder
- Raises:
IOError
if the respective match file couldn’t be read
The influence of the read configuration is that of an overwrite, i.e. all parameters will be generated (if not already present) and then the ones read from the configuration file will be overwritten.
- static to_match_file(finder, filename)[source]¶
Write the configuration to a match file with the given filename.
- Parameters:
finder (
finder.Finder
) – match configuration to savefilename (str) – match filename for the configuration
- configure_backend(backend=None, category='find', reset=False)[source]¶
Custom implementation of the base method.
See base method for details.
- synchronize_backend(backend=None, category='find', reset=False)[source]¶
Custom implementation of the base method.
See base method for details.
- can_calibrate(category, mark)[source]¶
Fix the parameters for a given category backend algorithm, i.e. disallow the calibrator to change them.
- Parameters:
mark (bool) – whether to mark for calibration
category (str) – backend category whose parameters are marked
- Raises:
UnsupportedBackendError
if category is not among the supported backend categories
- copy()[source]¶
Deep copy the current finder and its configuration.
- Returns:
a copy of the current finder with identical configuration
- Return type:
- find(needle, haystack)[source]¶
Find all needle targets in a haystack image.
- Parameters:
needle (
target.Target
or [target.Target
]) – image, text, pattern, or a list or chain of such to look forhaystack (
target.Image
) – image to look in
- Returns:
all found matches (one in most use cases)
- Return type:
[
match.Match
]- Raises:
NotImplementedError
if the base class method is called
- class guibot.finder.AutoPyFinder(configure=True, synchronize=True)[source]¶
Bases:
Finder
Simple matching backend provided by AutoPy.
- class guibot.finder.ContourFinder(configure=True, synchronize=True)[source]¶
Bases:
Finder
Contour matching backend provided by OpenCV.
Essentially, we will find all countours in a binary image, preprocessed with Gaussian blur and adaptive threshold and return the ones with area (size) similar to the searched image.
- configure_backend(backend=None, category='contour', reset=False)[source]¶
Custom implementation of the base method.
See base method for details.
- configure(threshold_filter=None, reset=True, **kwargs)[source]¶
Custom implementation of the base method.
- Parameters:
threshold_filter (str or None) – name of a preselected backend
- find(needle, haystack)[source]¶
Custom implementation of the base method.
- Parameters:
needle (
Image
) – target iamge to search for
See base method for details.
First extract all contours from a binary (boolean, threshold) version of the needle and haystack and then match the needle contours with one or more sets of contours in the haystack image. The number of needle matches depends on the set similarity and can be improved by requiring minimal area for the contours to be considered.
- class guibot.finder.TemplateFinder(configure=True, synchronize=True)[source]¶
Bases:
Finder
Template matching backend provided by OpenCV.
- configure_backend(backend=None, category='template', reset=False)[source]¶
Custom implementation of the base method.
See base method for details.
- class guibot.finder.FeatureFinder(configure=True, synchronize=True)[source]¶
Bases:
Finder
Feature matching backend provided by OpenCV.
Note
SURF and SIFT are proprietary algorithms and are not available by default in newer OpenCV versions (>3.0).
- configure_backend(backend=None, category='feature', reset=False)[source]¶
Custom implementation of the base method.
- Some relevant parameters are:
- detect filter - works for certain detectors and
determines how many initial features are detected in an image (e.g. hessian threshold for SURF detector)
- match filter - determines what part of all matches
returned by feature matcher remain good matches
- project filter - determines what part of the good
matches are considered inliers
ratio test - boolean for whether to perform a ratio test
symmetry test - boolean for whether to perform a symmetry test
See base method for details.
- configure(feature_detect=None, feature_extract=None, feature_match=None, reset=True, **kwargs)[source]¶
Custom implementation of the base method.
- Parameters:
feature_detect (str or None) – name of a preselected backend
feature_extract (str or None) – name of a preselected backend
feature_match (str or None) – name of a preselected backend
- synchronize_backend(backend=None, category='feature', reset=False)[source]¶
Custom implementation of the base method.
See base method for details.
- synchronize(feature_detect=None, feature_extract=None, feature_match=None, reset=True)[source]¶
Custom implementation of the base method.
- Parameters:
feature_detect (str or None) – name of a preselected backend
feature_extract (str or None) – name of a preselected backend
feature_match (str or None) – name of a preselected backend
- find(needle, haystack)[source]¶
Custom implementation of the base method.
- Parameters:
needle (
Image
) – target iamge to search for
See base method for details.
Warning
Finding multiple matches is currently not supported and this will currently only return a single match.
Available methods are: a combination of feature detector, extractor, and matcher.
- class guibot.finder.CascadeFinder(classifier_datapath='.', configure=True, synchronize=True)[source]¶
Bases:
Finder
Cascade matching backend provided by OpenCV.
This matcher uses Haar cascade for object detection. It is the most advanced method for object detection excluding convolutional neural networks. However, it requires the generation of a Haar cascade (if such is not already provided) of the needle to be found.
TODO: Currently no similarity requirement can be applied due to the cascade classifier API.
- class guibot.finder.TextFinder(configure=True, synchronize=True)[source]¶
Bases:
ContourFinder
Text matching backend provided by OpenCV.
This matcher will find a text (string) needle in the haystack, eventually relying on Tesseract or simpler kNN-based OCR, using extremal regions or contours before recognition, and returning a match if the string is among the recognized strings using string metric similar to Hamming distance.
Extremal Region Filter algorithm described in: Neumann L., Matas J.: Real-Time Scene Text Localization and Recognition, CVPR 2012
- configure_backend(backend=None, category='text', reset=False)[source]¶
Custom implementation of the base method.
See base method for details.
- configure(text_detector=None, text_recognizer=None, threshold_filter=None, threshold_filter2=None, threshold_filter3=None, reset=True, **kwargs)[source]¶
Custom implementation of the base method.
- Parameters:
text_detector (str or None) – name of a preselected backend
text_recognizer (str or None) – name of a preselected backend
threshold_filter (str or None) – threshold filter for the text detection stage
threshold_filter2 (str or None) – additional threshold filter for the OCR stage
threshold_filter3 (str or None) – additional threshold filter for distance transformation
- synchronize_backend(backend=None, category='text', reset=False)[source]¶
Custom implementation of the base method.
See base method for details.
- synchronize(text_detector=None, text_recognizer=None, threshold_filter=None, threshold_filter2=None, threshold_filter3=None, reset=True)[source]¶
Custom implementation of the base method.
- Parameters:
text_detector (str or None) – name of a preselected backend
text_recognizer (str or None) – name of a preselected backend
threshold_filter (str or None) – threshold filter for the text detection stage
threshold_filter2 (str or None) – additional threshold filter for the OCR stage
threshold_filter3 (str or None) – additional threshold filter for distance transformation
- class guibot.finder.TemplateFeatureFinder(configure=True, synchronize=True)[source]¶
Bases:
TemplateFinder
,FeatureFinder
Hybrid matcher using both OpenCV’s template and feature matching.
Feature matching is robust at small regions not too abundant of features where template matching is too picky. Template matching is good at large feature abundant regions and can be used as a heuristic for the feature matching. The current matcher will perform template matching first and then feature matching on the survived template matches to select among them one more time.
A separate (usually lower) front similarity is used for the first stage template matching in order to remove a lot of noise that would otherwise be distracting for the second stage feature matching.
- configure_backend(backend=None, category='tempfeat', reset=False)[source]¶
Custom implementation of the base method.
See base method for details.
- configure(template_match=None, feature_detect=None, feature_extract=None, feature_match=None, reset=True, **kwargs)[source]¶
Custom implementation of the base methods.
See base methods for details.
- synchronize(feature_detect=None, feature_extract=None, feature_match=None, reset=True)[source]¶
Custom implementation of the base method.
See base method for details.
- class guibot.finder.DeepFinder(classifier_datapath='.', configure=True, synchronize=True)[source]¶
Bases:
Finder
Deep learning matching backend provided by PyTorch.
The current implementation contains a basic convolutional neural network which can be trained to produce needle locations from a haystack image.
- configure_backend(backend=None, category='deep', reset=False)[source]¶
Custom implementation of the base method.
See base method for details.
- synchronize_backend(backend=None, category='deep', reset=False)[source]¶
Custom implementation of the base method.
See base method for details.
- class guibot.finder.HybridFinder(configure=True, synchronize=True)[source]¶
Bases:
Finder
Match a target through a sequence of differently configured attempts.
This matcher can work with any other matcher in the background and with unique or repeating matchers for each step. If a step fails, the matcher tries the next available along the fallback chain or fails if the end of the chain is reached.
- configure_backend(backend=None, category='hybrid', reset=False)[source]¶
Custom implementation of the base method.
See base method for details.