guibot.region module¶
SUMMARY¶
Secondary (and more advanced) interface for generic screen regions.
The main guibot interface is just a specialized region where we could match and work with subregions. Any region instance can also be a complete screen, hence the increased generality of using this as an interface and calling it directly.
INTERFACE¶
- class guibot.region.Region(xpos=0, ypos=0, width=0, height=0, dc=None, cv=None)[source]¶
Bases:
object
Region of the screen supporting vertex and nearby region selection, validation of expected images, and mouse and keyboard control.
- property x¶
Getter for readonly attribute.
- Returns:
x coordinate of the upleft vertex of the region
- Return type:
int
- property y¶
Getter for readonly attribute.
- Returns:
y coordinate of the upleft vertex of the region
- Return type:
int
- property width¶
Getter for readonly attribute.
- Returns:
width of the region (xpos+width for downright vertex x)
- Return type:
int
- property height¶
Getter for readonly attribute.
- Returns:
height of the region (ypos+height for downright vertex y)
- Return type:
int
- property center¶
Getter for readonly attribute.
- Returns:
center of the region
- Return type:
location.Location
- property top_left¶
Getter for readonly attribute.
- Returns:
upleft vertex of the region
- Return type:
location.Location
- property top_right¶
Getter for readonly attribute.
- Returns:
upright vertex of the region
- Return type:
location.Location
- property bottom_left¶
Getter for readonly attribute.
- Returns:
downleft vertex of the region
- Return type:
location.Location
- property bottom_right¶
Getter for readonly attribute.
- Returns:
downright vertex of the region
- Return type:
location.Location
- property is_empty¶
Getter for readonly attribute.
- Returns:
whether the region is empty, i.e. has zero size
- Return type:
bool
- property last_match¶
Getter for readonly attribute.
- Returns:
last match obtained from finding a target within the region
- Return type:
match.Match
- property mouse_location¶
Main region methods
- nearby(rrange=50)[source]¶
Obtain a region containing the previous one but enlarged by a number of pixels on each side.
- Parameters:
rrange (int) – number of pixels to add
- Returns:
new region enlarged by rrange on all sides
- Return type:
- above(rrange=0)[source]¶
Obtain a region containing the previous one but enlarged by a number of pixels on the upper side.
- Parameters:
rrange (int) – number of pixels to add
- Returns:
new region enlarged by rrange on upper side
- Return type:
- below(rrange=0)[source]¶
Obtain a region containing the previous one but enlarged by a number of pixels on the lower side.
- Parameters:
rrange (int) – number of pixels to add
- Returns:
new region enlarged by rrange on lower side
- Return type:
- left(rrange=0)[source]¶
Obtain a region containing the previous one but enlarged by a number of pixels on the left side.
- Parameters:
rrange (int) – number of pixels to add
- Returns:
new region enlarged by rrange on left side
- Return type:
- right(rrange=0)[source]¶
Obtain a region containing the previous one but enlarged by a number of pixels on the right side.
- Parameters:
rrange (int) – number of pixels to add
- Returns:
new region enlarged by rrange on right side
- Return type:
- find(target, timeout=10)[source]¶
Find a target on the screen.
- Parameters:
target (str or
target.Target
) – target to look fortimeout (int) – timeout before giving up
- Returns:
match obtained from finding the target within the region
- Return type:
match.Match
This method is the main entrance to all our target finding capabilities and is the milestone for all target expect methods.
- find_all(target, timeout=10, allow_zero=False)[source]¶
Find multiples of a target on the screen.
- Parameters:
target (str or
target.Target
) – target to look fortimeout (int) – timeout before giving up
allow_zero (bool) – whether to allow zero matches or raise error
- Returns:
matches obtained from finding the target within the region
- Return type:
[
match.Match
]- Raises:
errors.FindError
if no matches are found and zero matches are not allowed
This method is similar the one above but allows for more than one match.
- sample(target)[source]¶
Sample the similarity between a target and the screen, i.e. an empirical probability that the target is on the screen.
- Parameters:
target (str or
target.Target
) – target to look for- Returns:
similarity with best match on the screen
- Return type:
float
Note
Not all matchers support a ‘similarity’ value. The ones that don’t will return zero similarity (similarly to the target logging case).
- exists(target, timeout=0)[source]¶
Check if a target exists on the screen using the matching success as a threshold for the existence.
- Parameters:
target (str or
target.Target
) – target to look fortimeout (int) – timeout before giving up
- Returns:
match obtained from finding the target within the region or nothing if no match is found
- Return type:
match.Match
or None
- wait(target, timeout=30)[source]¶
Wait for a target to appear (be matched) with a given timeout as failing tolerance.
- Parameters:
target (str or
target.Target
) – target to look fortimeout (int) – timeout before giving up
- Returns:
match obtained from finding the target within the region
- Return type:
match.Match
- Raises:
errors.FindError
if no match is found
- wait_vanish(target, timeout=30)[source]¶
Wait for a target to disappear (be unmatched, i.e. matched without success) with a given timeout as failing tolerance.
- Parameters:
target (str or
target.Target
) – target to look fortimeout (int) – timeout before giving up
- Returns:
self
- Return type:
- Raises:
errors.NotFindError
if match is still found
- idle(timeout)[source]¶
Wait for a number of seconds and continue the nested call chain.
- Parameters:
timeout (int) – timeout to wait for
- Returns:
self
- Return type:
This method can be used as both a way to compactly wait for some time while not breaking the call chain. e.g.:
aregion.hover('abox').idle(1).click('aboxwithinthebox')
and as a way to conveniently perform timeout in between actions.
- hover(target_or_location)[source]¶
Hover the mouse over a target or location.
- Parameters:
target_or_location (
match.Match
orlocation.Location
or str ortarget.Target
) – target or location to hover to- Returns:
match from finding the target or nothing if hovering over a known location
- Return type:
match.Match
or None
- click(target_or_location, modifiers=None)[source]¶
Click on a target or location using the left mouse button and optionally holding special keys.
- Parameters:
target_or_location (
match.Match
orlocation.Location
or str ortarget.Target
) – target or location to click onmodifiers ([str]) – special keys to hold during clicking (see
inputmap.KeyModifier
for extensive list)
- Returns:
match from finding the target or nothing if clicking on a known location
- Return type:
match.Match
or None
The special keys refer to a list of key modifiers, e.g.:
self.click('my_target', [KeyModifier.MOD_CTRL, 'x']).
- right_click(target_or_location, modifiers=None)[source]¶
Click on a target or location using the right mouse button and optionally holding special keys.
Arguments and return values are analogical to
Region.click()
.
- middle_click(target_or_location, modifiers=None)[source]¶
Click on a target or location using the middle mouse button and optionally holding special keys.
Arguments and return values are analogical to
Region.click()
.
- double_click(target_or_location, modifiers=None)[source]¶
Double click on a target or location using the left mouse button and optionally holding special keys.
Arguments and return values are analogical to
Region.click()
.
- multi_click(target_or_location, count=3, modifiers=None)[source]¶
Click N times on a target or location using the left mouse button and optionally holding special keys.
Arguments and return values are analogical to
Region.click()
.
- click_expect(click_image_or_location, expect_target, modifiers=None, timeout=60, retries=3)[source]¶
Click on an image or location and wait for another one to appear.
- Parameters:
- Returns:
match obtained from finding the second target within the region
- Return type:
match.Match
- click_vanish(click_image_or_location, expect_target, modifiers=None, timeout=60, retries=3)[source]¶
Click on an image or location and wait for another one to disappear.
- Parameters:
- Returns:
self
- Return type:
- click_at_index(anchor, index=0, find_number=3, timeout=10)[source]¶
Find all instances of an anchor image and click on the one with the desired index given that they are horizontally then vertically sorted.
- Parameters:
anchor (str or
target.Target
) – image to find all matches ofindex (int) – index of the match to click on (assuming >=1 matches), sorted according to their (x,y) coordinates
find_number (int) – expected number of matches which is necessary for fast failure in case some elements are not visualized and/or proper matching result
timeout (int) – timeout before which the number of matches should be found
- Returns:
match from finding the target of the desired index
- Return type:
match.Match
Note
This method is a good replacement of a number of coincident limitations regarding the Windows version of autopy and PyRO and therefore the (Windows) virtual user:
autopy has an old BUG regarding capturing the screen at a region with boundaries, different than the entire screen -> subregioning which is the main way to deal with any kind of highly repeating and homogeneous interface, is totally unavailable here.
PyRO cannot serialize generators, so this is an implementation of a “generator step” involving clicking on consecutive matches.
The serialized virtual user now returns a list of proxified matches when calling find_all, but they are all essentially useless as they don’t proxify their returned objects and cannot be sent back as arguments. The special proxy interface of the virtual user was implemented only to handle the most basic case - serialize the objects returned by the main shared class by proxifying them (turning them into remote objects as well, which already have a well-defined serialization method) and nothing more.
- mouse_down(target_or_location, button=None)[source]¶
Hold down an arbitrary mouse button on a target or location.
- Parameters:
target_or_location (
match.Match
orlocation.Location
or str ortarget.Target
) – target or location to toggle onbutton (int or None) – button index depending on backend (default is left button) (see
inputmap.MouseButton
for extensive list)
- Returns:
match from finding the target or nothing if toggling on a known location
- Return type:
match.Match
or None
- mouse_up(target_or_location, button=None)[source]¶
Release an arbitrary mouse button on a target or location.
- Parameters:
target_or_location (
match.Match
orlocation.Location
or str ortarget.Target
) – target or location to toggle onbutton (int or None) – button index depending on backend (default is left button) (see
inputmap.MouseButton
for extensive list)
- Returns:
match from finding the target or nothing if toggling on a known location
- Return type:
match.Match
or None
- mouse_scroll(target_or_location, clicks=10, horizontal=False)[source]¶
Scroll the mouse for a number of clicks.
- Parameters:
target_or_location (
match.Match
orlocation.Location
or str ortarget.Target
) – target or location to scroll onclicks (int) – number of clicks to scroll up (positive) or down (negative)
horizontal (bool) – whether to perform a horizontal scroll instead (only available on some platforms)
- Returns:
match from finding the target or nothing if scrolling on a known location
- Return type:
match.Match
or None
- drag_drop(src_target_or_location, dst_target_or_location, modifiers=None)[source]¶
Drag from and drop at a target or location optionally holding special keys.
- Parameters:
src_target_or_location (
match.Match
orlocation.Location
or str ortarget.Target
) – target or location to drag fromdst_target_or_location (
match.Match
orlocation.Location
or str ortarget.Target
) – target or location to drop atmodifiers ([str]) – special keys to hold during dragging and dropping (see
inputmap.KeyModifier
for extensive list)
- Returns:
match from finding the target or nothing if dropping at a known location
- Return type:
match.Match
or None
- drag_from(target_or_location, modifiers=None)[source]¶
Drag from a target or location optionally holding special keys.
Arguments and return values are analogical to
Region.drag_drop()
but with target_or_location as src_target_or_location.
- drop_at(target_or_location, modifiers=None)[source]¶
Drop at a target or location optionally holding special keys.
Arguments and return values are analogical to
Region.drag_drop()
but with target_or_location as dst_target_or_location.
- press_keys(keys)[source]¶
Press a single key or a list of keys simultaneously.
- Parameters:
keys ([str] or str (possibly special keys in both cases)) – characters or special keys depending on the backend (see
inputmap.Key
for extensive list)- Returns:
self
- Return type:
Thus, the line
self.press_keys([Key.ENTER])
is equivalent to the lineself.press_keys(Key.ENTER)
. Other examples are:self.press_keys([Key.CTRL, 'X']) self.press_keys(['a', 'b', 3])
- press_at(keys, target_or_location)[source]¶
Press a single key or a list of keys simultaneously at a specified target or location.
This method is similar to
Region.press_keys()
but with an extra argument likeRegion.click()
.
- press_expect(keys, expect_target, timeout=60, retries=3)[source]¶
Press a key and wait for a target to appear.
- Parameters:
keys ([str] or str (possibly special keys in both cases)) – characters or special keys depending on the backend (see
inputmap.Key
for extensive list)expect_target – target to wait for
modifiers ([Key] or None) – key modifiers when clicking
timeout (int) – time in seconds to wait for
retries (int) – number of retries to reach expected target behavior
- Returns:
match obtained from finding the second target within the region
- Return type:
match.Match
- press_vanish(keys, expect_target, timeout=60, retries=3)[source]¶
Press a key and wait for a target to disappear.
- Parameters:
keys ([str] or str (possibly special keys in both cases)) – characters or special keys depending on the backend (see
inputmap.Key
for extensive list)expect_target – target to wait for
modifiers ([Key] or None) – key modifiers when clicking
timeout (int) – time in seconds to wait for
retries (int) – number of retries to reach expected target behavior
- Returns:
self
- Return type:
- type_text(text, modifiers=None)[source]¶
Type a list of consecutive character keys (without special keys).
- Parameters:
text ([str] or str (no special keys in both cases)) – characters or strings (independent of the backend)
modifiers ([str]) – special keys to hold during typing (see
inputmap.KeyModifier
for extensive list)
- Returns:
self
- Return type:
Thus, the line
self.type_text(['hello'])
is equivalent to the lineself.type_text('hello')
. Other examples are:self.type_text('ab3') # compare with press_keys() self.type_text(['Hello', ' ', 'user3614']) # in cases with appending
Special keys are only allowed as modifiers here - simply call
Region.press_keys()
multiple times for consecutively typing special keys.
- type_at(text, target_or_location, modifiers=None)[source]¶
Type a list of consecutive character keys (without special keys) at a specified target or location.
This method is similar to
Region.type_text()
but with an extra argument likeRegion.click()
.
- click_at(anchor, dx, dy, count=1)[source]¶
Clicks on a relative location using a displacement from an anchor.
- Parameters:
anchor (
Match
orLocation
orTarget
or str) – target of reference for relative locationdx (int) – displacement from the anchor in the x direction
dy (int) – displacement from the anchor in the y direction
count (int) – 0, 1, 2, … clicks on the relative location
- Returns:
self
- Return type:
- Raises:
exceptions.ValueError
if count is not acceptable value
- fill_at(anchor, text, dx, dy, del_flag=True, esc_flag=True, mark_clicks=1)[source]¶
Fills a new text at a text box using a displacement from an anchor.
- Parameters:
anchor (
Match
orLocation
orTarget
or str) – target of reference for the input fieldtext (str) – text to fill in
dx (int) – displacement from the anchor in the x direction
dy (int) – displacement from the anchor in the y direction
del_flag (bool) – whether to delete the highlighted text
esc_flag (bool) – whether to escape any possible fill suggestions
mark_clicks (int) – 0, 1, 2, … clicks to highlight previous text
- Returns:
self
- Return type:
- Raises:
exceptions.ValueError
if mark_click is not acceptable value
If the delete flag is set the previous content will be deleted or otherwise the new text will be added in the end of the current text. If the escape flag is set an escape will be pressed after typing in order to avoid any entry suggestions from a dropdown list that could cover important image matching areas.
Since different interfaces behave differently, one might need a single, double or triple click to mark the already present text that has to be replaced.
- select_at(anchor, image_or_index, dx, dy, dw=0, dh=0, ret_flag=True, mark_clicks=1, tries=3)[source]¶
Select an option at a dropdown list using either an integer index or an option image if the order cannot be easily inferred.
- Parameters:
anchor (
Match
orLocation
orTarget
or str) – target of reference for the input dropdown menuimage_or_index (str or int) – item image or item index
dx (int) – displacement from the anchor in the x direction
dy (int) – displacement from the anchor in the y direction
dw (int) – width to add to the displacement for an image search area
dh (int) – height to add to the displacement for an image search area
ret_flag (bool) – whether to press Enter after selecting
mark_clicks (int) – 0, 1, 2, … clicks to highlight previous text
tries (int) – retries if the dropdown menu doesn’t open after the initial click
- Returns:
self
- Return type:
It uses an anchor image which is rather constant and a displacement to locate the dropdown location. It moves down to the option if index is used where index 0 represents the current selection.
To avoid the limitations of the index method, an image of the option can be provided and will be matched in the area with and under the dropdown list. This also handles cases where the option coincides with the previously selected option. For more details see the really cool note in the end of this method.