Screen Reading & Scaling

ocr(regionName, options?)

Reads text from a named screen region using OCR. Small crops are auto-upscaled before recognition to improve accuracy on small fonts.

Parameter	Type	Default	Description
regionName	string	—	Name of the region object defined in the visual editor
options.numeric	boolean	false	Read the region as a number: corrects common digit misreads (e.g. ")"→1, "O"→0, "S"→5) and normalizes locale separators to a JS-parseable number ("10.000"/"10,000"→"10000", "1,5"→"1.5"). Use for scores, counts, and timers.
options.binarize	boolean	false	Force grayscale + threshold. Helps crisp fonts; can blank out anti-aliased or low-contrast text, so leave off unless it measurably helps.
options.upscale	boolean	true	Enlarge small crops before recognition (the default and the single biggest accuracy lever on small fonts). Pass false to read the crop at its native size.

Returns string | null — recognized text; "" when the region was read successfully but holds no text; or null when the read failed (region not found, capture or OCR-engine error). The ""-vs-null split lets you tell "nothing there" (don't retry) from "couldn't read" (safe to retry).

var score = ocr("score_region", { numeric: true });
print("Score: " + score);

ocrAt(x, y, w, h, options?)

Reads text from a screen region defined by its top-left corner and fixed size using OCR. All coordinates are in current device pixels.

Parameter	Type	Default	Description
x, y	number	—	Top-left coordinates of the region in pixels
w, h	number	—	Width and height of the region in pixels
options.numeric	boolean	false	Read the region as a number — see ocr().
options.binarize	boolean	false	Force grayscale + threshold — see ocr().
options.upscale	boolean	true	Enlarge small crops before recognition — see ocr().

Returns string | null — recognized text; "" when the area was read successfully but holds no text; or null when the read failed (capture or OCR-engine error). The ""-vs-null split lets you tell "nothing there" from "couldn't read".

// Fixed position (top-left x=440, y=100, width=200, height=40)
var score = ocrAt(440, 100, 200, 40, { numeric: true });

// Dynamic — read text next to a found element
// Match.x/y is the top-left corner; Match.w/h is the bounding box size
var pos = findText("Score:");
if (pos !== null) {
  var value = ocrAt(pos.x + pos.w + 8, pos.y, 150, pos.h);
}

readColor(x, y)

Reads the color of a single pixel at the given coordinates in current device pixels.

Parameter	Type	Description
x, y	number	Pixel coordinates

Returns string — hex color string in the format "#RRGGBB"

var color = readColor(540, 100);
if (color === "#FF0000") {
  print("Red pixel detected");
}

findColor(colorHex, options?)

Searches for the first pixel matching the given color within a per-channel tolerance. Scans top-to-bottom, left-to-right and returns the first hit.

Parameter	Type	Default	Description
colorHex	string	—	Target color in "#RRGGBB" format
options.tolerance	number	15	Maximum per-channel deviation 0–255. 0 = exact match; 15 allows ±15 on each R, G, B channel independently.
options.region	Region	—	Constrains the scan to a rectangular region. If omitted, the entire screen is searched.

Returns Match | null — a 1×1 Match at the found pixel. Call .tap() directly or read .cx/.cy for the coordinates. Returns null if not found.

// Search the whole screen and tap the result
var hit = findColor("#FF5C3D", { tolerance: 20 });
if (hit !== null) {
  hit.tap();
}

// Constrain to a fixed region
var hit = findColor("#FF5C3D", { tolerance: 20, region: new Region(100, 400, 300, 200) });

// Use a scene's stored search region (auto-scaled to current screen)
var hit = findColor("#FF5C3D", { region: Region.fromScene("battle_scene") });

// Dynamic region — search near a previously found object
var enemy = findObject("enemy_icon");
if (enemy !== null) {
  var nearby = findColor("#FF0000", { region: new Region(enemy.cx - 60, enemy.cy - 60, 120, 120) });
}

// Exact match, no tolerance
var exact = findColor("#00FF00", { tolerance: 0 });

findAllColors(colorHex, options?)

Searches for all pixels matching the given color within a per-channel tolerance. Scans top-to-bottom, left-to-right and returns every hit as an array.

Parameter	Type	Default	Description
colorHex	string	—	Target color in "#RRGGBB" format
options.tolerance	number	15	Maximum per-channel deviation 0–255. 0 = exact match; 15 allows ±15 on each R, G, B channel independently.
options.region	Region	—	Constrains the scan to a rectangular region. If omitted, the entire screen is searched.
options.maxResults	number	0	Maximum number of results to return. 0 means no limit. Use this to avoid scanning unnecessarily when you only need a few matches.

Returns Match[] — array of 1×1 Match objects at each found pixel. Empty array if none found.

// Find all red pixels on screen and tap each
var hits = findAllColors("#FF0000", { tolerance: 20 });
print("Found " + hits.length + " matches");
hits.forEach(function(m) {
  m.tap();
  wait(100);
});

// Limit to 5 results within a region
var hits = findAllColors("#FFD700", {
  tolerance: 15,
  region: new Region(0, 400, 1080, 800),
  maxResults: 5
});

// Check if any match exists
if (findAllColors("#FF0000", { tolerance: 10, maxResults: 1 }).length > 0) {
  print("Red detected");
}

Tip: When you only need to detect presence, pass maxResults: 1 to stop scanning after the first match.

findColorRegions(colorHex, options?)

Like findAllColors, but groups adjacent matching pixels into connected regions (blobs) and returns one Match per region instead of one per pixel. Each match's x/y/w/h is the blob's bounding box and cx/cy is its pixel centroid. Results are ordered largest-region first. This is the right tool when a color fills an area (a button, an icon) and you want to act once per area rather than once per pixel.

Parameter	Type	Default	Description
colorHex	string	—	Target color in "#RRGGBB" format
options.tolerance	number	15	Maximum per-channel deviation 0–255. 0 = exact match.
options.region	Region	—	Constrains the scan to a rectangular region. If omitted, the entire screen is searched.
options.minSize	number	1	Minimum region size in pixels. Regions with fewer matching pixels are discarded — raise this to filter out anti-aliasing noise and stray pixels.
options.maxResults	number	0	Maximum number of regions to return (largest first). 0 means no limit.

Returns Match[] — one Match per colored region (bounding box + centroid). Empty array if none found.

// Tap the center of every red blob at least 50px large
var blobs = findColorRegions("#FF0000", { tolerance: 20, minSize: 50 });
print("Found " + blobs.length + " regions");
blobs.forEach(function(b) {
  b.tap();        // taps the blob's centroid (cx, cy)
  wait(100);
});

// Largest gold region within a panel
var panel = new Region(0, 400, 1080, 800);
var biggest = findColorRegions("#FFD700", { region: panel, minSize: 10, maxResults: 1 })[0];
if (biggest) print("biggest blob is " + biggest.w + "x" + biggest.h);

Tip: Prefer findColorRegions over findAllColors when matching solid areas — a single button can be thousands of pixels, and findAllColors would return one match per pixel.

waitForColor(colorHex, options?)

Blocks until a pixel matching the given color appears on screen, or the timeout elapses. Polls with findColor and respects a user stop between polls.

Parameter	Type	Default	Description
colorHex	string	—	Target color in "#RRGGBB" format
options.tolerance	number	15	Maximum per-channel deviation 0–255. 0 = exact match.
options.region	Region	—	Constrains the scan to a rectangular region. If omitted, the entire screen is searched.
options.timeout	number	5000	Max wait time in milliseconds
options.interval	number	500	Polling interval in milliseconds

Returns Match | null — a 1×1 Match at the found pixel if the color appeared within the timeout, otherwise null.

// Wait for a health bar to turn red, then react
var hit = waitForColor("#FF0000", { tolerance: 20, timeout: 10000 });
if (hit !== null) {
  print("low health at " + hit.cx + "," + hit.cy);
}

// Constrain to a region
waitForColor("#FFD700", { region: new Region(0, 400, 1080, 800), timeout: 8000 });

waitUntilColorGone(colorHex, options?)

Blocks until no pixel matching the given color remains on screen, or the timeout elapses. Polls with findColor and respects a user stop between polls.

Parameter	Type	Default	Description
colorHex	string	—	Target color in "#RRGGBB" format
options.tolerance	number	15	Maximum per-channel deviation 0–255. 0 = exact match.
options.region	Region	—	Constrains the scan to a rectangular region. If omitted, the entire screen is searched.
options.timeout	number	5000	Max wait time in milliseconds
options.interval	number	500	Polling interval in milliseconds

Returns boolean — true if the color disappeared within the timeout, false if the wait timed out.

// Wait for a red "loading" indicator to clear
if (waitUntilColorGone("#FF0000", { tolerance: 20, timeout: 15000 })) {
  print("indicator cleared");
}

findAllText(query, options?)

Searches the screen for all occurrences of the given text using OCR. Matching is always case-insensitive.

Parameter	Type	Default	Description
query	string	—	Text to search for
options.region	Region	—	Constrains the OCR scan to a rectangular region.
options.wholeWord	boolean	false	When true, only matches the query when it appears as a standalone word — not as part of a longer word. Returns the precise bounding box of the matched word rather than the whole line.

Returns Match[] — array of matches. cx/cy is the text center; x/y/w/h is the bounding box. Empty array if none found.

findAllText("Delete").forEach(function(m) {
  m.tap();
  wait(300);
});

// Only search within the bottom half of the screen
findAllText("Collect", { region: new Region(0, 960, 1080, 960) }).forEach(function(m) {
  m.tap();
});

// Whole-word match: finds "Start" but not "Restart" or "Starting"
findAllText("Start", { wholeWord: true }).forEach(function(m) {
  m.tap();
});

findText(query, options?)

Searches the screen for the first occurrence of the given text using OCR. Convenience wrapper for findAllText(query, options)[0]. Matching is always case-insensitive.

Parameter	Type	Default	Description
query	string	—	Text to search for
options.region	Region	—	Constrains the search to a specific area
options.wholeWord	boolean	false	When true, only matches the query when it appears as a standalone word — not as part of a longer word. Returns the precise bounding box of the matched word rather than the whole line.

Returns Match | null — the first match, or null if none found. cx/cy is the text center; x/y is the top-left corner of the bounding box.

var m = findText("Score:");
if (m !== null) {
  m.tap();
}

// Read the value to the right of a label
// m.x/y is the top-left corner; m.w/h is the bounding box size
var label = findText("HP:");
if (label !== null) {
  var value = ocrAt(label.x + label.w + 8, label.y, 120, label.h);
}

// Whole-word match: finds "test" but not "testa" or "atest"
var m = findText("test", { wholeWord: true });
if (m !== null) {
  m.tap();
}

findObject(objectName, options?)

Finds a UI object by image matching and returns its screen position without tapping it. If the object has a search region set in the editor, it is applied automatically.

Parameter	Type	Default	Description
objectName	string	—	Name of the UI object
options.threshold	number	0.70	Minimum match confidence 0.0–1.0
options.region	Region	—	Overrides the object's search region.
options.matchTimeout	number	—	Maximum milliseconds to spend on the pyramid search. The search stops early after the allotted time; any scales already completed are still used. Omit for no limit.

Returns Match | null — cx/cy is the object center, x/y/w/h is the bounding box, score is the confidence. Returns null if not found.

var pos = findObject("enemy_icon");
if (pos !== null) {
  pos.tap(); // Tap the matched object
}

findObject("enemy_icon", { threshold: 0.85 });
findObject("enemy_icon", { region: new Region(0, 400, 1080, 800) });
findObject("enemy_icon", { matchTimeout: 200 }); // stop pyramid search after 200 ms

// Use a scene's stored search region:
findObject("enemy_icon", { region: Region.fromScene("battle_scene") });

// Use a Region to search for a color inside the matched object
var icon = findObject("item_icon");
if (icon !== null) {
  var hit = icon.findColor("#FFD700");
}

waitForObject(objectName, options?)

Blocks until the given object appears on screen or the timeout elapses.

Parameter	Type	Default	Description
objectName	string	—	Name of the UI object
options.timeout	number	5000	Max wait time in milliseconds
options.interval	number	500	Polling interval in milliseconds
options.threshold	number	0.70	Minimum match confidence 0.0–1.0
options.region	Region	—	Constrains the search to a rectangular area
options.matchTimeout	number	—	Maximum milliseconds to spend on the pyramid search per poll iteration. Omit for no limit.

Returns Match | null — the found object, or null if the timeout elapsed

waitForObject("spinner");
waitForObject("spinner", { timeout: 10000 });
waitForObject("spinner", { threshold: 0.8 });
waitForObject("spinner", { timeout: 10000, interval: 250, threshold: 0.7 });
waitForObject("spinner", { region: new Region(200, 300, 680, 400) });

// Use the returned Match immediately
var pos = waitForObject("reward_box", { timeout: 8000 });
if (pos !== null) {
  pos.tap();
}

waitForText(query, options?)

Blocks until the given text appears anywhere on screen (via OCR) or the timeout elapses. Matching is always case-insensitive.

Parameter	Type	Default	Description
query	string	—	Text to search for
options.timeout	number	5000	Max wait time in milliseconds
options.interval	number	500	Polling interval in milliseconds
options.region	Region	—	Constrains the OCR scan to a rectangular area
options.wholeWord	boolean	false	When true, only matches when the query appears as a standalone word — not as part of a longer word

Returns Match | null — a Match at the found text (cx/cy center, x/y/w/h bounding box) if it appeared within the timeout, otherwise null. A returned Match is truthy, so existing if (waitForText(…)) checks keep working.

waitForText("Loading…");
waitForText("Loading…", { timeout: 10000 });
waitForText("Loading…", { timeout: 10000, interval: 250 });
waitForText("Ready", { region: new Region(0, 800, 1080, 400) });
waitForText("Start", { wholeWord: true }); // won't match "Restart"

waitUntilGone(objectName, options?)

Blocks until the given object disappears from the screen (template match returns no results) or the timeout elapses.

Parameter	Type	Default	Description
objectName	string	—	Name of the UI object
options.timeout	number	5000	Max wait time in milliseconds
options.interval	number	500	Polling interval in milliseconds
options.threshold	number	0.70	Minimum match confidence 0.0–1.0
options.region	Region	—	Constrains the search to a rectangular area
options.matchTimeout	number	—	Maximum milliseconds to spend on the pyramid search per poll iteration. Omit for no limit.

Returns boolean — true if the object disappeared within the timeout

findAndTap("confirm_button");
waitUntilGone("confirm_button");
waitUntilGone("confirm_button", { timeout: 8000 });
waitUntilGone("confirm_button", { timeout: 8000, interval: 250, threshold: 0.8 });
waitUntilGone("confirm_button", { region: new Region(200, 300, 680, 400) });

waitUntilTextGone(query, options?)

Blocks until the given text disappears from the screen (OCR returns no results) or the timeout elapses.

Parameter	Type	Default	Description
query	string	—	Text to watch for
options.timeout	number	5000	Max wait time in milliseconds
options.interval	number	500	Polling interval in milliseconds
options.region	Region	—	Constrains the OCR scan to a rectangular area

Returns boolean — true if the text disappeared within the timeout

findAndTap("close_button");
waitUntilTextGone("Loading…");
waitUntilTextGone("Loading…", { timeout: 8000 });
waitUntilTextGone("Connecting", { region: new Region(0, 0, 1080, 200) });

findAllObjects(objectName, options?)

Finds all image matches of a UI object on screen by template matching.

Parameter	Type	Default	Description
objectName	string	—	Name of the UI object
options.threshold	number	0.70	Minimum match confidence 0.0–1.0
options.region	Region	—	Constrains the search to a rectangular area.
options.matchTimeout	number	—	Maximum milliseconds to spend on the pyramid search. The search stops early after the allotted time; any scales already completed are still used. Omit for no limit.

Returns Match[] — array of matches sorted by confidence (highest first). Empty array if none found.

var matches = findAllObjects("item_icon");
for (var i = 0; i < matches.length; i++) {
  matches[i].tap();
  wait(200);
}

// only high-confidence matches
findAllObjects("item_icon", { threshold: 0.75 }).forEach(function(m) {
  print("Found at " + m.cx + "," + m.cy + " conf=" + m.score);
  m.tap();
  wait(200);
});

// constrain search to a region
findAllObjects("item_icon", { region: new Region(0, 600, 1080, 600) });