Screen Reading & Scaling

ocr(regionName)

Reads text from a named screen region using OCR.

Parameter	Type	Description
regionName	string	Name of the region object defined in the visual editor

Returns string — recognized text, or "" if nothing was read

var score = ocr("score_region");
print("Score: " + score);

ocrAt(x, y, w, h)

Reads text from a screen region defined by its top-left corner and fixed size using OCR. All coordinates are in current device pixels.

Parameter	Type	Description
x, y	number	Top-left coordinates of the region in pixels
w, h	number	Width and height of the region in pixels

Returns string — recognized text, or "" if nothing was read

// Fixed position (top-left x=440, y=100, width=200, height=40)
var score = ocrAt(440, 100, 200, 40);

// Dynamic — read text next to a found element
// Match.x/y is the top-left corner; Match.w/h is the bounding box size
var pos = findText("Score:");
if (pos !== null) {
  var value = ocrAt(pos.x + pos.w + 8, pos.y, 150, pos.h);
}

readColor(x, y)

Reads the color of a single pixel at the given coordinates in current device pixels.

Parameter	Type	Description
x, y	number	Pixel coordinates

Returns string — hex color string in the format "#RRGGBB"

var color = readColor(540, 100);
if (color === "#FF0000") {
  print("Red pixel detected");
}

findColor(colorHex, options?)

Searches for the first pixel matching the given color within a per-channel tolerance. Scans top-to-bottom, left-to-right and returns the first hit.

Parameter	Type	Default	Description
colorHex	string	—	Target color in "#RRGGBB" format
options.tolerance	number	15	Maximum per-channel deviation 0–255. 0 = exact match; 15 allows ±15 on each R, G, B channel independently.
options.region	Region	—	Constrains the scan to a rectangular region. If omitted, the entire screen is searched.

Returns Match | null — a 1×1 Match at the found pixel. Call .tap() directly or read .cx/.cy for the coordinates. Returns null if not found.

// Search the whole screen and tap the result
var hit = findColor("#FF5C3D", { tolerance: 20 });
if (hit !== null) {
  hit.tap();
}

// Constrain to a fixed region
var hit = findColor("#FF5C3D", { tolerance: 20, region: new Region(100, 400, 300, 200) });

// Use a scene's stored search region (auto-scaled to current screen)
var hit = findColor("#FF5C3D", { region: Region.fromScene("battle_scene") });

// Dynamic region — search near a previously found object
var enemy = getPosition("enemy_icon");
if (enemy !== null) {
  var nearby = findColor("#FF0000", { region: new Region(enemy.cx - 60, enemy.cy - 60, 120, 120) });
}

// Exact match, no tolerance
var exact = findColor("#00FF00", { tolerance: 0 });

findColors(colorHex, options?)

Searches for all pixels matching the given color within a per-channel tolerance. Scans top-to-bottom, left-to-right and returns every hit as an array.

Parameter	Type	Default	Description
colorHex	string	—	Target color in "#RRGGBB" format
options.tolerance	number	15	Maximum per-channel deviation 0–255. 0 = exact match; 15 allows ±15 on each R, G, B channel independently.
options.region	Region	—	Constrains the scan to a rectangular region. If omitted, the entire screen is searched.
options.maxResults	number	0	Maximum number of results to return. 0 means no limit. Use this to avoid scanning unnecessarily when you only need a few matches.

Returns Match[] — array of 1×1 Match objects at each found pixel. Empty array if none found.

// Find all red pixels on screen and tap each
var hits = findColors("#FF0000", { tolerance: 20 });
print("Found " + hits.length + " matches");
hits.forEach(function(m) {
  m.tap();
  wait(100);
});

// Limit to 5 results within a region
var hits = findColors("#FFD700", {
  tolerance: 15,
  region: new Region(0, 400, 1080, 800),
  maxResults: 5
});

// Check if any match exists
if (findColors("#FF0000", { tolerance: 10, maxResults: 1 }).length > 0) {
  print("Red detected");
}

Tip: When you only need to detect presence, pass maxResults: 1 to stop scanning after the first match.

findAllText(query, options?)

Searches the screen for all occurrences of the given text using OCR. Matching is always case-insensitive.

Parameter	Type	Default	Description
query	string	—	Text to search for
options.region	Region	—	Constrains the OCR scan to a rectangular region.
options.wholeWord	boolean	false	When true, only matches the query when it appears as a standalone word — not as part of a longer word. Returns the precise bounding box of the matched word rather than the whole line.

Returns Match[] — array of matches. cx/cy is the text center; x/y/w/h is the bounding box. Empty array if none found.

findAllText("Delete").forEach(function(m) {
  m.tap();
  wait(300);
});

// Only search within the bottom half of the screen
findAllText("Collect", { region: new Region(0, 960, 1080, 960) }).forEach(function(m) {
  m.tap();
});

// Whole-word match: finds "Start" but not "Restart" or "Starting"
findAllText("Start", { wholeWord: true }).forEach(function(m) {
  m.tap();
});

findText(query, options?)

Searches the screen for the first occurrence of the given text using OCR. Convenience wrapper for findAllText(query, options)[0]. Matching is always case-insensitive.

Parameter	Type	Default	Description
query	string	—	Text to search for
options.region	Region	—	Constrains the search to a specific area
options.wholeWord	boolean	false	When true, only matches the query when it appears as a standalone word — not as part of a longer word. Returns the precise bounding box of the matched word rather than the whole line.

Returns Match | null — the first match, or null if none found. cx/cy is the text center; x/y is the top-left corner of the bounding box.

var m = findText("Score:");
if (m !== null) {
  m.tap();
}

// Read the value to the right of a label
// m.x/y is the top-left corner; m.w/h is the bounding box size
var label = findText("HP:");
if (label !== null) {
  var value = ocrAt(label.x + label.w + 8, label.y, 120, label.h);
}

// Whole-word match: finds "test" but not "testa" or "atest"
var m = findText("test", { wholeWord: true });
if (m !== null) {
  m.tap();
}

getPosition(objectName, options?)

Finds a UI object by image matching and returns its screen position without tapping it. If the object has a search region set in the editor, it is applied automatically.

Parameter	Type	Default	Description
objectName	string	—	Name of the UI object
options.threshold	number	0.70	Minimum match confidence 0.0–1.0
options.region	Region	—	Overrides the object's search region.
options.matchTimeout	number	—	Maximum milliseconds to spend on the pyramid search. The search stops early after the allotted time; any scales already completed are still used. Omit for no limit.

Returns Match | null — cx/cy is the object center, x/y/w/h is the bounding box, score is the confidence. Returns null if not found.

var pos = getPosition("enemy_icon");
if (pos !== null) {
  pos.tap(); // Tap the matched object
}

getPosition("enemy_icon", { threshold: 0.85 });
getPosition("enemy_icon", { region: new Region(0, 400, 1080, 800) });
getPosition("enemy_icon", { matchTimeout: 200 }); // stop pyramid search after 200 ms

// Use a scene's stored search region:
getPosition("enemy_icon", { region: Region.fromScene("battle_scene") });

// Use a Region to search for a color inside the matched object
var icon = getPosition("item_icon");
if (icon !== null) {
  var hit = icon.findColor("#FFD700");
}

waitForObject(objectName, options?)

Blocks until the given object appears on screen or the timeout elapses.

Parameter	Type	Default	Description
objectName	string	—	Name of the UI object
options.timeout	number	5000	Max wait time in milliseconds
options.interval	number	500	Polling interval in milliseconds
options.threshold	number	0.70	Minimum match confidence 0.0–1.0
options.region	Region	—	Constrains the search to a rectangular area
options.matchTimeout	number	—	Maximum milliseconds to spend on the pyramid search per poll iteration. Omit for no limit.

Returns Match | null — the found object, or null if the timeout elapsed

waitForObject("spinner");
waitForObject("spinner", { timeout: 10000 });
waitForObject("spinner", { threshold: 0.8 });
waitForObject("spinner", { timeout: 10000, interval: 250, threshold: 0.7 });
waitForObject("spinner", { region: new Region(200, 300, 680, 400) });

// Use the returned Match immediately
var pos = waitForObject("reward_box", { timeout: 8000 });
if (pos !== null) {
  pos.tap();
}

waitForText(query, options?)

Blocks until the given text appears anywhere on screen (via OCR) or the timeout elapses. Matching is always case-insensitive.

Parameter	Type	Default	Description
query	string	—	Text to search for
options.timeout	number	5000	Max wait time in milliseconds
options.interval	number	500	Polling interval in milliseconds
options.region	Region	—	Constrains the OCR scan to a rectangular area
options.wholeWord	boolean	false	When true, only matches when the query appears as a standalone word — not as part of a longer word

Returns boolean — true if the text appeared within the timeout

waitForText("Loading…");
waitForText("Loading…", { timeout: 10000 });
waitForText("Loading…", { timeout: 10000, interval: 250 });
waitForText("Ready", { region: new Region(0, 800, 1080, 400) });
waitForText("Start", { wholeWord: true }); // won't match "Restart"

waitUntilGone(objectName, options?)

Blocks until the given object disappears from the screen (template match returns no results) or the timeout elapses.

Parameter	Type	Default	Description
objectName	string	—	Name of the UI object
options.timeout	number	5000	Max wait time in milliseconds
options.interval	number	500	Polling interval in milliseconds
options.threshold	number	0.70	Minimum match confidence 0.0–1.0
options.region	Region	—	Constrains the search to a rectangular area
options.matchTimeout	number	—	Maximum milliseconds to spend on the pyramid search per poll iteration. Omit for no limit.

Returns boolean — true if the object disappeared within the timeout

findAndTap("confirm_button");
waitUntilGone("confirm_button");
waitUntilGone("confirm_button", { timeout: 8000 });
waitUntilGone("confirm_button", { timeout: 8000, interval: 250, threshold: 0.8 });
waitUntilGone("confirm_button", { region: new Region(200, 300, 680, 400) });

waitUntilTextGone(query, options?)

Blocks until the given text disappears from the screen (OCR returns no results) or the timeout elapses.

Parameter	Type	Default	Description
query	string	—	Text to watch for
options.timeout	number	5000	Max wait time in milliseconds
options.interval	number	500	Polling interval in milliseconds
options.region	Region	—	Constrains the OCR scan to a rectangular area

Returns boolean — true if the text disappeared within the timeout

findAndTap("close_button");
waitUntilTextGone("Loading…");
waitUntilTextGone("Loading…", { timeout: 8000 });
waitUntilTextGone("Connecting", { region: new Region(0, 0, 1080, 200) });

getPositions(objectName, options?)

Finds all image matches of a UI object on screen by template matching.

Parameter	Type	Default	Description
objectName	string	—	Name of the UI object
options.threshold	number	0.70	Minimum match confidence 0.0–1.0
options.region	Region	—	Constrains the search to a rectangular area.
options.matchTimeout	number	—	Maximum milliseconds to spend on the pyramid search. The search stops early after the allotted time; any scales already completed are still used. Omit for no limit.

Returns Match[] — array of matches sorted by confidence (highest first). Empty array if none found.

var matches = getPositions("item_icon");
for (var i = 0; i < matches.length; i++) {
  matches[i].tap();
  wait(200);
}

// only high-confidence matches
getPositions("item_icon", { threshold: 0.75 }).forEach(function(m) {
  print("Found at " + m.cx + "," + m.cy + " conf=" + m.score);
  m.tap();
  wait(200);
});

// constrain search to a region
getPositions("item_icon", { region: new Region(0, 600, 1080, 600) });

Note: nextMatch(), tapCurrentMatch(), and ocrCurrentMatch() are used internally by the visual editor's generated code and do not need to be called in handwritten scripts.